mirror of
https://github.com/simplex-chat/simplex-chat.git
synced 2026-11-22 19:58:26 +00:00
store: Prometheus metrics of orders, badges, providers, chain and names (#8143)
* store: Prometheus metrics of name orders and registrar funds * store: keep the service running when the metrics port fails * store: describe held balances and the funds read time exactly * store: export badge, payment provider and chain node metrics * store: rename a test binding that shadowed hspec's before * store: count the names on chain in the metrics * store: check the scraped samples one by one * store: answer 0 for metrics labels with no rows --------- Co-authored-by: sh-agents <350651734+sh-agents@users.noreply.github.com>
This commit is contained in:
1 parent
2a87d8dc2e
commit
caf0caa54d
27 files changed
+1070
-43
No files matched your search
@@ -701,7 +701,8 @@ With `serve_webapp = on` (all in one), the service listens on both ports, and ea
|
||||
own site: every path outside `/api/` and `/webhooks/` is a file of that site, `/` and `/index.html`
|
||||
its shell. Both ports answer `/api/` and `/webhooks/`, whatever the `Host`. A reverse proxy sends
|
||||
each domain to its own port, as the all-in-one example in [Caddy](#caddy) does. With
|
||||
`serve_webapp = off`, the service listens on `badges_port` alone, for the API and the webhooks.
|
||||
`serve_webapp = off`, the service listens on `badges_port` alone, for the API and the webhooks. With
|
||||
`metrics_port`, it also listens on `127.0.0.1` on that port, for `/metrics` alone.
|
||||
|
||||
### Development with a local chain
|
||||
|
||||
@@ -863,6 +864,106 @@ simplex.domains {
|
||||
The service sets each file's type and `Cache-Control` itself: `no-cache`, and a year for
|
||||
`/assets/*`, so the names site's fonts and images are revalidated rather than kept for a day.
|
||||
|
||||
### Metrics
|
||||
|
||||
`[listener] metrics_port` turns on the store's Prometheus metrics. It is off when the key is
|
||||
absent and must differ from `badges_port` and `names_port`. The service answers `GET /metrics` on
|
||||
`127.0.0.1:<metrics_port>`, whatever `host` is, so only the machine itself reaches it:
|
||||
|
||||
```ini
|
||||
[listener]
|
||||
metrics_port = 9300
|
||||
```
|
||||
|
||||
Each scrape counts invoices, badges and, with `[names]`, name orders, transactions, balances and
|
||||
record edits in the database. With `[names]`, the registrar key's ETH balance, its sponsor
|
||||
allowance, the node's latest block and the terms links are read from the node at start and then
|
||||
every minute, and a scrape answers the last read, so a slow or failing node never delays a scrape.
|
||||
A part that fails keeps its last value, sets `simplex_store_names_chain_read_ok` to 0 and logs a
|
||||
warning. A port that cannot be listened on is logged as an error and leaves the rest of the
|
||||
service running.
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| `simplex_store_invoices_open{provider}` | open invoices of badges and names, by provider: `crypto` (BTCPay) or `stripe`; 0 for a configured provider with none |
|
||||
| `simplex_store_invoices_overdue{provider}` | those of them past their expiry and its 10-minute grace with no money received; the poller expires each once the provider answers, so a count that stays up means the provider cannot be read or the invoice cancelled there |
|
||||
| `simplex_store_provider_list_ok_timestamp_seconds{provider}` | when the poller last listed the provider's open invoices successfully, every one or two minutes at the default `idle_seconds`; absent until the first list |
|
||||
| `simplex_store_provider_webhook_timestamp_seconds{provider}` | when the provider last delivered a webhook that verified; absent until the first one |
|
||||
| `simplex_store_badges_purchases{status}` | badge purchases by status: `acquiring`, `issued`, `superseded` or `failed`, each 0 when it has none |
|
||||
| `simplex_store_badges_subscriptions{state}` | App Store and Google Play subscriptions paying for badges, by state: `renewing`, `cancelled`, `billingRetry` or `ended`, each 0 when it has none |
|
||||
| `simplex_store_badges_codes_open` | badge codes paid or free, not revoked or expired, with redemptions left |
|
||||
| `simplex_store_names_orders{phase}` | name orders in the phase: `unpaid`, `queued`, `committing`, `committed`, `registering`, `registered`, `taken` or `failed`; the queue is `queued` to `registering`, while `unpaid` also counts closed orders kept for 30 days and `failed` every failed order not tried again |
|
||||
| `simplex_store_names_queue_oldest_change_timestamp_seconds` | the last change, in Unix seconds, of the order unchanged longest among those the service moves on: `queued`, `committing`, `registering`, and `committed` with the page's reveal stored; absent when there is none |
|
||||
| `simplex_store_names_transactions_pending` | registrar transactions sent and not settled, one per nonce, so a fee replacement counts once |
|
||||
| `simplex_store_names_transactions_oldest_pending_timestamp_seconds` | when the oldest of them was first signed; absent when none is pending |
|
||||
| `simplex_store_names_invoices_open` | open invoices of name orders |
|
||||
| `simplex_store_names_balances_held` | balances reserved by an order paying the difference, until that order is paid or closed |
|
||||
| `simplex_store_names_balances_held_cents` | the cents of those balances |
|
||||
| `simplex_store_names_record_edits_pending` | record edits relayed and not settled |
|
||||
| `simplex_store_names_record_edits_oldest_pending_timestamp_seconds` | when the oldest of them was made; absent when none is pending |
|
||||
| `simplex_store_names_registrar_balance_wei` | the registrar key's ETH balance, as last read |
|
||||
| `simplex_store_names_sponsor_allowance_cents` | its allowance left on the sponsor, in cents, rounded down, as last read |
|
||||
| `simplex_store_names_chain_head_block` | the node's latest block number, as last read |
|
||||
| `simplex_store_names_chain_head_timestamp_seconds` | that block's time |
|
||||
| `simplex_store_names_chain_base_fee_wei` | that block's base fee per gas |
|
||||
| `simplex_store_names_terms_held{kind}` | 1 when the service holds the text of the `general` or `community` terms the name policy links, 0 while it cannot fetch it, so names of that kind cannot be ordered; absent while the policy links none |
|
||||
| `simplex_store_names_chain_read_timestamp_seconds` | when every part of the read last succeeded |
|
||||
| `simplex_store_names_chain_read_ok` | 1 when the last read succeeded in every part, 0 when a part failed or none has finished |
|
||||
|
||||
With `[names]`, the service also counts every name on chain, however it was registered, at start and
|
||||
then every 5 minutes, under the metric names of the former `simplex-names-exporter`. At the latest
|
||||
block it reads the base registrar's names (`totalSupply`, `tokenByIndex`), keeps those whose
|
||||
`nameExpires` is after the block's time, takes each label from `labelOf`, then walks the subname
|
||||
registrar (`subnameHook()`) level by level with `getChildren`, keeping subnames whose `ownerOf` is
|
||||
not zero. For each name it reads `resolver(node)` on the registry and, on the configured resolver,
|
||||
its `simplex.contact` and `simplex.channel` texts. The reads run 16 at a time at that one block. A
|
||||
failed census keeps the last one and is counted.
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| `simplex_names{level,records}` | live names at `level` 2 (`alice.simplex`), 3 (`shop.alice.simplex`) and below, by `records`: `none`, `contact`, `channel`, `both`, or `other_resolver` for a name whose resolver is not the configured one, whose records are not read; names in the 90-day grace period are not counted |
|
||||
| `simplex_name_length` | histogram of label lengths in characters, by `level`, a bucket for each length 1 to 63 |
|
||||
| `simplex_names_block` | the block the census read |
|
||||
| `simplex_names_refresh_timestamp_seconds` | when the census last succeeded |
|
||||
| `simplex_names_refresh_failures_total` | counter of failed censuses since the service started |
|
||||
|
||||
Every metric is a gauge, but for the census's histogram and counter. A stuck order, transaction or record edit, a node that cannot be read or
|
||||
has stopped following the chain, terms that cannot be shown and a payment provider that cannot be
|
||||
read show as:
|
||||
|
||||
```
|
||||
time() - simplex_store_names_queue_oldest_change_timestamp_seconds > 1200
|
||||
time() - simplex_store_names_transactions_oldest_pending_timestamp_seconds > 900
|
||||
time() - simplex_store_names_record_edits_oldest_pending_timestamp_seconds > 900
|
||||
simplex_store_names_chain_read_ok == 0
|
||||
time() - simplex_store_names_chain_head_timestamp_seconds > 120
|
||||
simplex_store_names_terms_held == 0
|
||||
time() - simplex_store_provider_list_ok_timestamp_seconds > 600
|
||||
simplex_store_invoices_overdue > 0
|
||||
```
|
||||
|
||||
The overdue count can be above 0 for a few seconds between an invoice's grace ending and the next
|
||||
poller pass, so its alert waits, as with `for: 10m`. Mainnet makes a block every 12 seconds, so a
|
||||
head older than 2 minutes means the node has fallen behind.
|
||||
|
||||
An order moves on within minutes: its commit and register are each mined and confirmed, and the
|
||||
register waits about a minute after the commit. An order tried again can wait unchanged up to
|
||||
`reveal_margin_seconds` (900 by default) for its old commitment to lapse, so the queue alert's
|
||||
threshold is above it. Size the alerts on the balance and the allowance to
|
||||
the orders expected before a top-up, with the figures in [Fund the key](#fund-the-key).
|
||||
|
||||
Caddy proxies it on the monitoring site of the host, beside node_exporter, inside that site's
|
||||
`handle_path /metrics/*` block:
|
||||
|
||||
```
|
||||
handle_path /store_service* {
|
||||
rewrite * /metrics
|
||||
reverse_proxy http://127.0.0.1:9300
|
||||
}
|
||||
```
|
||||
|
||||
Prometheus then scrapes `/metrics/store_service` on that site.
|
||||
|
||||
### Terms and community names
|
||||
|
||||
The controller's name policy (`SimplexNamePolicy`, read from `namePolicy()` once at startup and
|
||||
@@ -1234,6 +1335,7 @@ psql "postgresql://simplex@127.0.0.1:5432/badge_service" -c "SELECT registrar_no
|
||||
|
||||
#### When funds or the allowance run out
|
||||
|
||||
With `metrics_port` set, both are exported before they run out (see [Metrics](#metrics)).
|
||||
`docker compose logs store-service` shows:
|
||||
|
||||
- `[ERROR …] names sender: nonce <n>: not sent: …`, with the node's `insufficient funds` message,
|
||||
|
||||
@@ -93,7 +93,7 @@ How a request flows:
|
||||
| Part | Role | Source |
|
||||
|---|---|---|
|
||||
| Startup | `store_service.ini` is parsed, the database lock taken, and the node's chain id and the issuer key checked; then the chat core is started. In its callback the providers, the names environment and the web environment are built, the sites exported, the managed group resolved, and all lanes started together. Without `--service-config` only the chat side and the badge request lanes run. | [`Main.hs:15-28`](../Main.hs#L15-L28), [`Service.hs:280-370`](../src/StoreService/Service.hs#L280-L370) |
|
||||
| Web API | Warp listeners on `[listener] host`: `badges_port` (8080) always, `names_port` (8081) with `[names]` and `serve_webapp = on`. `/api/*` and both webhooks are answered on every port. Per-client limits a minute: read 60, search 30, preview 20, code 10, records 10, create 5, find 5. Checkouts that reach a provider: 5 a minute per IPv4 /24 or IPv6 /48, and 60 card and 30 BTCPay a minute for all clients together. At most 500 long polls are held at once, each up to 30 s. | [`Web/Server.hs:121-151`](../src/StoreService/Web/Server.hs#L121-L151), [`Web/Server.hs:563-586`](../src/StoreService/Web/Server.hs#L563-L586), [`Config.hs:511-520`](../src/StoreService/Config.hs#L511-L520), [`Web/Names.hs:28-51`](../src/StoreService/Web/Names.hs#L28-L51), [`Web/Http.hs:111-161`](../src/StoreService/Web/Http.hs#L111-L161), [`Web/Http.hs:227-230`](../src/StoreService/Web/Http.hs#L227-L230) |
|
||||
| Web API | Warp listeners on `[listener] host`: `badges_port` (8080) always, `names_port` (8081) with `[names]` and `serve_webapp = on`; with `metrics_port`, a listener on `127.0.0.1` for `/metrics` alone. `/api/*` and both webhooks are answered on every port but the metrics one. Per-client limits a minute: read 60, search 30, preview 20, code 10, records 10, create 5, find 5. Checkouts that reach a provider: 5 a minute per IPv4 /24 or IPv6 /48, and 60 card and 30 BTCPay a minute for all clients together. At most 500 long polls are held at once, each up to 30 s. | [`Web/Server.hs:121-151`](../src/StoreService/Web/Server.hs#L121-L151), [`Web/Server.hs:563-586`](../src/StoreService/Web/Server.hs#L563-L586), [`Config.hs:511-520`](../src/StoreService/Config.hs#L511-L520), [`Web/Names.hs:28-51`](../src/StoreService/Web/Names.hs#L28-L51), [`Web/Http.hs:111-161`](../src/StoreService/Web/Http.hs#L111-L161), [`Web/Http.hs:227-230`](../src/StoreService/Web/Http.hs#L227-L230) |
|
||||
| Order handlers | An order is placed in one of three ways: a paid invoice, a prepaid code, a spend of another order's balance. The owner's signature, the timestamp, the duration, the terms and an available name are checked, then the chain price for a paid or spent order and the balance for a spend. The order is written in one transaction, in which the same wallet's overdue orders for that name are closed. | [`Web/Names/Orders.hs:86-306`](../src/StoreService/Web/Names/Orders.hs#L86-L306), [`Names/Orders.hs:201-310`](../src/StoreService/Names/Orders.hs#L201-L310), [`Names/Balances.hs:79-106`](../src/StoreService/Names/Balances.hs#L79-L106) |
|
||||
| Registration and owner handlers | The registration view, held on its version; the reveal; Try again; find by the owner's signature; a move to another name. | [`Web/Names/Registration.hs:45-140`](../src/StoreService/Web/Names/Registration.hs#L45-L140), [`Web/Names/Owner.hs:44-152`](../src/StoreService/Web/Names/Owner.hs#L44-L152), [`Names/Reveal.hs:103-145`](../src/StoreService/Names/Reveal.hs#L103-L145), [`Names/View.hs:52-112`](../src/StoreService/Names/View.hs#L52-L112) |
|
||||
| Search handlers | Name status, price, terms, and records with link profiles; the terms text; the link preview. These handlers only read, except that a terms text fetched for the first time is stored. | [`Web/Names/Search.hs:53-154`](../src/StoreService/Web/Names/Search.hs#L53-L154), [`Names/Profiles.hs:236-274`](../src/StoreService/Names/Profiles.hs#L236-L274) |
|
||||
|
||||
@@ -35,7 +35,7 @@ where
|
||||
import qualified Control.Exception as E
|
||||
import Control.Applicative (many, (<|>))
|
||||
import Control.Logger.Simple (LogLevel (..), logNote, logWarn)
|
||||
import Control.Monad (forM_, mfilter, unless, when)
|
||||
import Control.Monad (forM, forM_, mfilter, unless, when)
|
||||
import Data.Attoparsec.Combinator (lookAhead)
|
||||
import Data.Attoparsec.Text (Parser, endOfInput, isEndOfLine, parseOnly, satisfy, skipMany, skipSpace, skipWhile)
|
||||
import Data.Bifunctor (first)
|
||||
@@ -80,7 +80,8 @@ data ListenerConfig = ListenerConfig
|
||||
lNamesStaticDir :: Maybe FilePath,
|
||||
lNamesExportDir :: Maybe FilePath,
|
||||
-- | @Nothing@ unless the names site is served, which takes @[names]@ and @serve_webapp@ on.
|
||||
lNamesPort :: Maybe Int
|
||||
lNamesPort :: Maybe Int,
|
||||
lMetricsPort :: Maybe Int
|
||||
}
|
||||
deriving (Eq, Show)
|
||||
|
||||
@@ -425,7 +426,7 @@ trailingNoise = skipSpace *> skipMany (comment *> skipSpace)
|
||||
knownSettings :: [(Text, [Text])]
|
||||
knownSettings =
|
||||
-- The retired keys are known, so parseConfig refuses them rather than a warning calling them ignored.
|
||||
[ ("listener", ["host", "badges_port", "badges_static_dir", "serve_webapp", "badges_export_dir", "trust_forwarded_for"] <> namesListenerKeys <> map fst retiredListenerKeys),
|
||||
[ ("listener", ["host", "badges_port", "badges_static_dir", "serve_webapp", "badges_export_dir", "trust_forwarded_for", "metrics_port"] <> namesListenerKeys <> map fst retiredListenerKeys),
|
||||
("btcpay", ["host", "api_key", "store_id", "webhook_secret", "expiry_minutes", "speed_policy", "payment_tolerance"]),
|
||||
("stripe", ["secret_key", "publishable_key", "webhook_secret", "session_minutes"]),
|
||||
("poll", ["waiting_seconds", "idle_seconds"]),
|
||||
@@ -471,6 +472,9 @@ parseConfig ini = do
|
||||
let lBadgesExportDir = T.unpack <$> present "listener" "badges_export_dir"
|
||||
lTrustForwardedFor <- bool "listener" "trust_forwarded_for" False
|
||||
(lNamesStaticDir, lNamesExportDir, lNamesPort) <- namesSite lServeWebapp lBadgesPort
|
||||
lMetricsPort <- optionalPort "metrics_port"
|
||||
when (any (\p -> p == lBadgesPort || Just p == lNamesPort) lMetricsPort) $
|
||||
Left "listener.metrics_port must differ from listener.badges_port and listener.names_port"
|
||||
btc <- btcpaySection
|
||||
str <- stripeSection
|
||||
iss <- issuerSection
|
||||
@@ -510,7 +514,7 @@ parseConfig ini = do
|
||||
Left "[dev] check_store_during_coverage needs accept_unverified_store_receipts or credit_test_purchases = on, as only a dev deployment may ask a store while coverage runs"
|
||||
pure
|
||||
ServiceConfig
|
||||
{ listener = ListenerConfig {lHost, lBadgesPort, lBadgesStaticDir, lServeWebapp, lBadgesExportDir, lTrustForwardedFor, lNamesStaticDir, lNamesExportDir, lNamesPort},
|
||||
{ listener = ListenerConfig {lHost, lBadgesPort, lBadgesStaticDir, lServeWebapp, lBadgesExportDir, lTrustForwardedFor, lNamesStaticDir, lNamesExportDir, lNamesPort, lMetricsPort},
|
||||
btcpay = btc,
|
||||
stripe = str,
|
||||
poll = PollConfig {pWaitingSeconds, pIdleSeconds},
|
||||
@@ -531,9 +535,9 @@ parseConfig ini = do
|
||||
present s k = mfilter (not . T.null) (T.strip <$> look s k)
|
||||
required s k = maybe (Left (T.unpack s <> "." <> T.unpack k <> " is required")) Right (present s k)
|
||||
optional s k d = Right (fromMaybe d (present s k))
|
||||
listenerPort k d = do
|
||||
p <- num "listener" k d
|
||||
if 1 <= p && p <= 65535 then Right p else Left ("listener." <> T.unpack k <> " must be between 1 and 65535")
|
||||
listenerPort k d = fromMaybe d <$> optionalPort k
|
||||
optionalPort k =
|
||||
optionalNum "listener" k >>= mapM (\p -> if 1 <= p && p <= 65535 then Right p else Left ("listener." <> T.unpack k <> " must be between 1 and 65535"))
|
||||
unusedListenerKey k why = Left ("listener." <> T.unpack k <> " is set but " <> why <> ", so nothing would use it")
|
||||
namesSite serve badgesPort
|
||||
| not (hasSection "names") = case filter (isJust . look "listener") namesListenerKeys of
|
||||
@@ -545,12 +549,11 @@ parseConfig ini = do
|
||||
when (namesPort == Just badgesPort) $ Left "listener.names_port must differ from listener.badges_port"
|
||||
let staticDir = maybe defaultNamesStaticDir T.unpack (present "listener" "names_static_dir")
|
||||
Right (Just staticDir, T.unpack <$> present "listener" "names_export_dir", namesPort)
|
||||
num s k d = fromMaybe d <$> optionalNum s k
|
||||
-- Integer, because readMaybe at Int wraps silently, reading 2^64+4 as 4.
|
||||
num s k d = case look s k of
|
||||
Nothing -> Right d
|
||||
Just v -> case readMaybe (T.unpack (T.strip v)) of
|
||||
Just n | n >= toInteger (minBound :: Int), n <= toInteger (maxBound :: Int) -> Right (fromInteger n)
|
||||
_ -> Left (T.unpack k <> " must be a whole number")
|
||||
optionalNum s k = forM (look s k) $ \v -> case readMaybe (T.unpack (T.strip v)) of
|
||||
Just n | n >= toInteger (minBound :: Int), n <= toInteger (maxBound :: Int) -> Right (fromInteger n)
|
||||
_ -> Left (T.unpack k <> " must be a whole number")
|
||||
bool s k d = case fmap (T.toLower . T.strip) (look s k) of
|
||||
Nothing -> Right d
|
||||
Just "on" -> Right True
|
||||
|
||||
@@ -0,0 +1,267 @@
|
||||
{-# LANGUAGE NamedFieldPuns #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
|
||||
module StoreService.Metrics
|
||||
( MetricsEnv (..),
|
||||
NamesSource (..),
|
||||
ChainRead (..),
|
||||
ProviderTimes (..),
|
||||
CensusRead (..),
|
||||
noChainRead,
|
||||
noCensusRead,
|
||||
readChain,
|
||||
runChainReads,
|
||||
readCensus,
|
||||
serviceText,
|
||||
namesText,
|
||||
censusText,
|
||||
metricsApp,
|
||||
runMetrics,
|
||||
)
|
||||
where
|
||||
|
||||
import Control.Concurrent (threadDelay)
|
||||
import Control.Concurrent.STM
|
||||
import Control.Logger.Simple (logWarn)
|
||||
import Control.Monad (forever, unless, void)
|
||||
import qualified Data.ByteString.Lazy as LB
|
||||
import Data.Either (isRight)
|
||||
import Data.List (nub, sort)
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Map.Strict as M
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Data.String (fromString)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Data.Text.Encoding (encodeUtf8)
|
||||
import Data.Time.Clock (UTCTime, addUTCTime, getCurrentTime)
|
||||
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
|
||||
import Network.HTTP.Types (hContentType, status200)
|
||||
import Network.Wai (Application, pathInfo, responseLBS)
|
||||
import qualified Network.Wai.Handler.Warp as Warp
|
||||
import Simplex.Chat.Badges.Code (maxNameLength)
|
||||
import Simplex.Chat.Badges.Types (BadgePurchaseStatus (..), SubscriptionState (..))
|
||||
import Simplex.Messaging.Agent.Store.Common (DBStore, withTransaction)
|
||||
import Simplex.Messaging.Encoding.String (textEncode)
|
||||
import Simplex.Messaging.Util (raceAny_, tshow)
|
||||
import StoreService.Config (ListenerConfig)
|
||||
import StoreService.Names.Census (Census (..), CensusName (..), nameCensus, recordsLabel)
|
||||
import StoreService.Names.Chain (Block (..), ChainClient, latestBlock)
|
||||
import StoreService.Names.Contracts (Contracts, RegistrarFunds (..), registrarFunds, termsHeld)
|
||||
import StoreService.Names.Store (NamesCounts (..), namesCounts)
|
||||
import StoreService.Poller (expiryGrace)
|
||||
import StoreService.Safely (safely)
|
||||
import StoreService.Store (BadgeCounts (..), badgeCounts)
|
||||
import StoreService.Store.Invoices (openInvoiceCounts)
|
||||
import StoreService.Web.Http (notFound, only)
|
||||
import StoreService.Web.Server (webSettings)
|
||||
|
||||
data MetricsEnv = MetricsEnv
|
||||
{ meStore :: DBStore,
|
||||
meProviders :: [Text],
|
||||
meListedOk :: TVar (Map Text UTCTime),
|
||||
meWebhookAt :: TVar (Map Text UTCTime),
|
||||
-- | @Nothing@ when the ini has no @[names]@ section.
|
||||
meNames :: Maybe NamesSource
|
||||
}
|
||||
|
||||
data NamesSource = NamesSource {nsChain :: ChainClient, nsContracts :: Contracts}
|
||||
|
||||
-- | Each part keeps its last value while its read fails.
|
||||
data ChainRead = ChainRead
|
||||
{ crFunds :: Maybe RegistrarFunds,
|
||||
crHead :: Maybe Block,
|
||||
crTerms :: [(Text, Bool)],
|
||||
-- | the last read in which every part succeeded
|
||||
crReadAt :: Maybe UTCTime,
|
||||
crOk :: Bool
|
||||
}
|
||||
|
||||
noChainRead :: ChainRead
|
||||
noChainRead = ChainRead {crFunds = Nothing, crHead = Nothing, crTerms = [], crReadAt = Nothing, crOk = False}
|
||||
|
||||
-- | By each provider's text encoding.
|
||||
data ProviderTimes = ProviderTimes {ptListedOk :: Map Text UTCTime, ptWebhookAt :: Map Text UTCTime}
|
||||
|
||||
chainReadInterval :: Int
|
||||
chainReadInterval = 60 * 1000000
|
||||
|
||||
metricsHost :: String
|
||||
metricsHost = "127.0.0.1"
|
||||
|
||||
readChain :: NamesSource -> TVar ChainRead -> IO ()
|
||||
readChain NamesSource {nsChain, nsContracts} v = do
|
||||
funds <- registrarFunds nsContracts
|
||||
head' <- latestBlock nsChain
|
||||
terms <- termsHeld nsContracts
|
||||
now <- getCurrentTime
|
||||
let ok = isRight funds && isRight head' && isRight terms
|
||||
atomically $ modifyTVar' v $ \ChainRead {crFunds, crHead, crTerms, crReadAt} ->
|
||||
ChainRead
|
||||
{ crFunds = either (const crFunds) Just funds,
|
||||
crHead = either (const crHead) Just head',
|
||||
crTerms = either (const crTerms) id terms,
|
||||
crReadAt = if ok then Just now else crReadAt,
|
||||
crOk = ok
|
||||
}
|
||||
warnIfFailed "the registrar's funds" funds
|
||||
warnIfFailed "the latest block" head'
|
||||
warnIfFailed "the terms links" terms
|
||||
where
|
||||
warnIfFailed :: Show e => Text -> Either e a -> IO ()
|
||||
warnIfFailed what = either (\e -> logWarn $ "store metrics: " <> what <> " could not be read: " <> tshow e) (const $ pure ())
|
||||
|
||||
runChainReads :: NamesSource -> TVar ChainRead -> IO ()
|
||||
runChainReads ns v = forever $ do
|
||||
done <- safely "store metrics: the chain read" (readChain ns v)
|
||||
unless done $ atomically $ modifyTVar' v $ \cr -> cr {crOk = False}
|
||||
threadDelay chainReadInterval
|
||||
|
||||
-- | A failed census keeps the last one and is counted.
|
||||
data CensusRead = CensusRead {crCensus :: Maybe (Census, UTCTime), crFailures :: Int}
|
||||
|
||||
noCensusRead :: CensusRead
|
||||
noCensusRead = CensusRead {crCensus = Nothing, crFailures = 0}
|
||||
|
||||
censusInterval :: Int
|
||||
censusInterval = 300 * 1000000
|
||||
|
||||
readCensus :: NamesSource -> TVar CensusRead -> IO ()
|
||||
readCensus NamesSource {nsChain, nsContracts} v = do
|
||||
counted <- nameCensus nsChain nsContracts
|
||||
case counted of
|
||||
Right census -> do
|
||||
now <- getCurrentTime
|
||||
atomically $ modifyTVar' v $ \cr -> cr {crCensus = Just (census, now)}
|
||||
Left e -> do
|
||||
atomically $ modifyTVar' v $ \cr -> cr {crFailures = crFailures cr + 1}
|
||||
logWarn $ "store metrics: the names census failed: " <> tshow e
|
||||
|
||||
runCensus :: NamesSource -> TVar CensusRead -> IO ()
|
||||
runCensus ns v = forever $ do
|
||||
done <- safely "store metrics: the names census" (readCensus ns v)
|
||||
unless done $ atomically $ modifyTVar' v $ \cr -> cr {crFailures = crFailures cr + 1}
|
||||
threadDelay censusInterval
|
||||
|
||||
-- | Open invoices come as (provider, open, overdue).
|
||||
serviceText :: [Text] -> [(Text, Int, Int)] -> BadgeCounts -> ProviderTimes -> Text
|
||||
serviceText providers invoices BadgeCounts {bcPurchases, bcSubscriptions, bcOpenCodes} ProviderTimes {ptListedOk, ptWebhookAt} =
|
||||
T.concat
|
||||
[ gauge "simplex_store_invoices_open" "Open invoices by provider" [labelled "provider" p (value n) | (p, (n, _)) <- invoiceCounts],
|
||||
gauge "simplex_store_invoices_overdue" "Open invoices past their expiry and its grace with no money received, which the poller expires once the provider answers" [labelled "provider" p (value n) | (p, (_, n)) <- invoiceCounts],
|
||||
gauge "simplex_store_provider_list_ok_timestamp_seconds" "When the poller last listed the provider's open invoices successfully" [labelled "provider" p (timestamp t) | (p, t) <- M.toList ptListedOk],
|
||||
gauge "simplex_store_provider_webhook_timestamp_seconds" "When the provider last delivered a webhook that verified" [labelled "provider" p (timestamp t) | (p, t) <- M.toList ptWebhookAt],
|
||||
gauge "simplex_store_badges_purchases" "Badge purchases by status" [labelled "status" s (value n) | (s, n) <- withZeros purchaseStatuses bcPurchases],
|
||||
gauge "simplex_store_badges_subscriptions" "App store subscriptions paying for badges, by state" [labelled "state" s (value n) | (s, n) <- withZeros subscriptionStates bcSubscriptions],
|
||||
gauge "simplex_store_badges_codes_open" "Badge codes paid or free, not revoked or expired, with redemptions left" (Just $ value bcOpenCodes)
|
||||
]
|
||||
where
|
||||
invoiceCounts = M.toList $ M.union (M.fromList [(p, (n, overdue)) | (p, n, overdue) <- invoices]) (M.fromList [(p, (0, 0)) | p <- providers])
|
||||
|
||||
withZeros :: [Text] -> [(Text, Int)] -> [(Text, Int)]
|
||||
withZeros known counts = M.toList $ M.union (M.fromList counts) (M.fromList [(k, 0) | k <- known])
|
||||
|
||||
purchaseStatuses :: [Text]
|
||||
purchaseStatuses = map textEncode [PSAcquiring, PSIssued, PSSuperseded, PSFailed]
|
||||
|
||||
subscriptionStates :: [Text]
|
||||
subscriptionStates = map textEncode [SSRenewing, SSCancelled, SSBillingRetry, SSEnded]
|
||||
|
||||
namesText :: NamesCounts -> ChainRead -> Text
|
||||
namesText NamesCounts {ncPhases, ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit} ChainRead {crFunds, crHead, crTerms, crReadAt, crOk} =
|
||||
T.concat
|
||||
[ gauge "simplex_store_names_orders" "Name orders by phase" [labelled "phase" (textEncode p) (value $ fromMaybe 0 $ lookup p ncPhases) | p <- [minBound .. maxBound]],
|
||||
gauge "simplex_store_names_queue_oldest_change_timestamp_seconds" "Last change of the order longest unmoved among those the service moves on: queued, committing, registering, or committed with its reveal" (timestamp <$> ncQueueOldestChange),
|
||||
gauge "simplex_store_names_transactions_pending" "Registrar transactions sent and not settled, one per nonce" (Just $ value ncPendingTransactions),
|
||||
gauge "simplex_store_names_transactions_oldest_pending_timestamp_seconds" "When the oldest pending registrar transaction was first signed" (timestamp <$> ncOldestPending),
|
||||
gauge "simplex_store_names_invoices_open" "Open invoices of name orders" (Just $ value ncOpenInvoices),
|
||||
gauge "simplex_store_names_balances_held" "Balances reserved by an order paying the difference, until that order is paid or closed" (Just $ value ncHeldBalances),
|
||||
gauge "simplex_store_names_balances_held_cents" "Cents of the balances orders hold" (Just $ value ncHeldCents),
|
||||
gauge "simplex_store_names_record_edits_pending" "Record edits relayed and not settled" (Just $ value ncPendingEdits),
|
||||
gauge "simplex_store_names_record_edits_oldest_pending_timestamp_seconds" "When the oldest pending record edit was made" (timestamp <$> ncOldestEdit),
|
||||
gauge "simplex_store_names_registrar_balance_wei" "The registrar key's ETH balance, as last read" (value . rfBalanceWei <$> crFunds),
|
||||
gauge "simplex_store_names_sponsor_allowance_cents" "The registrar key's allowance left on the sponsor, as last read" (value . rfAllowanceCents <$> crFunds),
|
||||
gauge "simplex_store_names_chain_head_block" "The node's latest block number, as last read" (value . blockNumber <$> crHead),
|
||||
gauge "simplex_store_names_chain_head_timestamp_seconds" "The node's latest block time, as last read" (value . blockTimestamp <$> crHead),
|
||||
gauge "simplex_store_names_chain_base_fee_wei" "The node's latest block base fee per gas, as last read" (value . blockBaseFee <$> crHead),
|
||||
gauge "simplex_store_names_terms_held" "1 when the service holds the text of the terms the name policy links, by kind, as last read" [labelled "kind" k (value $ fromEnum held) | (k, held) <- crTerms],
|
||||
gauge "simplex_store_names_chain_read_timestamp_seconds" "When every part of the chain read last succeeded" (timestamp <$> crReadAt),
|
||||
gauge "simplex_store_names_chain_read_ok" "1 when the last chain read succeeded in every part, 0 when a part failed or none was made" (Just $ value $ fromEnum crOk)
|
||||
]
|
||||
|
||||
value :: Show a => a -> (Text, Text)
|
||||
value n = ("", tshow n)
|
||||
|
||||
timestamp :: UTCTime -> (Text, Text)
|
||||
timestamp t = value (floor $ utcTimeToPOSIXSeconds t :: Integer)
|
||||
|
||||
labelled :: Text -> Text -> (Text, Text) -> (Text, Text)
|
||||
labelled name v = withLabels [(name, v)]
|
||||
|
||||
withLabels :: [(Text, Text)] -> (Text, Text) -> (Text, Text)
|
||||
withLabels ls (_, sample) = (labelSet ls, sample)
|
||||
|
||||
labelSet :: [(Text, Text)] -> Text
|
||||
labelSet ls = "{" <> T.intercalate "," [name <> "=\"" <> escape v <> "\"" | (name, v) <- ls] <> "}"
|
||||
where
|
||||
escape = T.replace "\n" "\\n" . T.replace "\"" "\\\"" . T.replace "\\" "\\\\"
|
||||
|
||||
gauge :: Foldable f => Text -> Text -> f (Text, Text) -> Text
|
||||
gauge = family "gauge"
|
||||
|
||||
family :: Foldable f => Text -> Text -> Text -> f (Text, Text) -> Text
|
||||
family kind name help samples
|
||||
| null samples = ""
|
||||
| otherwise = "# HELP " <> name <> " " <> help <> "\n# TYPE " <> name <> " " <> kind <> "\n" <> foldMap sample samples <> "\n"
|
||||
where
|
||||
sample (labels, v) = name <> labels <> " " <> v <> "\n"
|
||||
|
||||
-- | The census of #8114's exporter, under its metric names: names and subnames by level and records set, and their
|
||||
-- label lengths. Before the first census only the failures are answered.
|
||||
censusText :: CensusRead -> Text
|
||||
censusText CensusRead {crCensus, crFailures} =
|
||||
family "counter" "simplex_names_refresh_failures_total" "Refreshes that failed" (Just $ value crFailures)
|
||||
<> foldMap counted crCensus
|
||||
where
|
||||
counted (Census {cBlock, cNames}, refreshedAt) =
|
||||
T.concat
|
||||
[ gauge "simplex_names" "Registered names by level, and by simplex.contact and simplex.channel set" [withLabels [("level", tshow l), ("records", recordsLabel r)] (value $ count l r) | l <- levelsSeen, r <- [minBound .. maxBound]],
|
||||
lengthHistogram,
|
||||
gauge "simplex_names_block" "Block the names were read at" (Just $ value cBlock),
|
||||
gauge "simplex_names_refresh_timestamp_seconds" "Time of the last refresh" (Just $ timestamp refreshedAt)
|
||||
]
|
||||
where
|
||||
levelsSeen = sort $ nub $ 2 : map cnLevel cNames
|
||||
count l r = length [() | CensusName {cnLevel, cnRecords} <- cNames, cnLevel == l, cnRecords == r]
|
||||
lengthHistogram =
|
||||
"# HELP simplex_name_length Label lengths in characters, by level\n# TYPE simplex_name_length histogram\n"
|
||||
<> foldMap lengthsAt levelsSeen
|
||||
<> "\n"
|
||||
lengthsAt l =
|
||||
let lengths = [cnLength | CensusName {cnLevel, cnLength} <- cNames, cnLevel == l]
|
||||
level = [("level", tshow l)]
|
||||
line suffix ls n = "simplex_name_length" <> suffix <> labelSet ls <> " " <> tshow n <> "\n"
|
||||
in foldMap (\b -> line "_bucket" (level <> [("le", tshow b <> ".0")]) (length $ filter (<= b) lengths)) [1 .. maxNameLength]
|
||||
<> line "_bucket" (level <> [("le", "+Inf")]) (length lengths)
|
||||
<> line "_sum" level (sum lengths)
|
||||
<> line "_count" level (length lengths)
|
||||
|
||||
metricsApp :: MetricsEnv -> TVar ChainRead -> TVar CensusRead -> Application
|
||||
metricsApp MetricsEnv {meStore, meProviders, meListedOk, meWebhookAt, meNames} v census req respond = case pathInfo req of
|
||||
["metrics"] -> only "GET" req respond $ do
|
||||
now <- getCurrentTime
|
||||
(invoices, badges, names) <- withTransaction meStore $ \db ->
|
||||
(,,) <$> openInvoiceCounts db (addUTCTime (negate expiryGrace) now) <*> badgeCounts db now <*> mapM (const $ namesCounts db) meNames
|
||||
times <- ProviderTimes <$> readTVarIO meListedOk <*> readTVarIO meWebhookAt
|
||||
chain <- readTVarIO v
|
||||
counted <- readTVarIO census
|
||||
let text = serviceText meProviders invoices badges times <> foldMap (\n -> namesText n chain <> censusText counted) names
|
||||
respond $ responseLBS status200 [(hContentType, "text/plain; version=0.0.4; charset=utf-8")] $ LB.fromStrict $ encodeUtf8 text
|
||||
_ -> respond notFound
|
||||
|
||||
runMetrics :: ListenerConfig -> Int -> MetricsEnv -> IO ()
|
||||
runMetrics lc port env@MetricsEnv {meNames} = do
|
||||
v <- newTVarIO noChainRead
|
||||
census <- newTVarIO noCensusRead
|
||||
let serve = safely ("store metrics: the endpoint on port " <> tshow port) $ Warp.runSettings (Warp.setPort port . Warp.setHost (fromString metricsHost) $ webSettings lc) (metricsApp env v census)
|
||||
raceAny_ $ (void serve >> forever (threadDelay maxBound)) : concat [[runChainReads ns v, runCensus ns census] | Just ns <- [meNames]]
|
||||
@@ -1,5 +1,6 @@
|
||||
{-# LANGUAGE LambdaCase #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
{-# LANGUAGE TupleSections #-}
|
||||
|
||||
module StoreService.Names.Abi
|
||||
( AbiValue (..),
|
||||
@@ -21,6 +22,7 @@ module StoreService.Names.Abi
|
||||
decodeBytes32,
|
||||
decodeBytes,
|
||||
decodeString,
|
||||
decodeLabels,
|
||||
)
|
||||
where
|
||||
|
||||
@@ -126,6 +128,18 @@ decodeBytes ret = offsetAt 0 ret >>= bytesAt ret
|
||||
decodeString :: ByteString -> Either AbiError Text
|
||||
decodeString ret = decodeBytes ret >>= first (const AENotUtf8) . decodeUtf8'
|
||||
|
||||
-- | A return of a bytes32[] and a string[] of the same length, paired.
|
||||
decodeLabels :: ByteString -> Either AbiError [(ByteString, Text)]
|
||||
decodeLabels ret = do
|
||||
(hashCount, hashesStart) <- offsetAt 0 ret >>= arrayAt
|
||||
(labelCount, labelsStart) <- offsetAt abiWordSize ret >>= arrayAt
|
||||
unless (hashCount == labelCount) $ Left AEOffset
|
||||
hashes <- mapM (\i -> wordAt (hashesStart + i * abiWordSize) ret) [0 .. hashCount - 1]
|
||||
labels <- mapM (\i -> offsetAt (labelsStart + i * abiWordSize) ret >>= bytesAt ret . (labelsStart +) >>= first (const AENotUtf8) . decodeUtf8') [0 .. labelCount - 1]
|
||||
pure $ zip hashes labels
|
||||
where
|
||||
arrayAt o = (,o + abiWordSize) <$> offsetAt o ret
|
||||
|
||||
wordAt :: Int -> ByteString -> Either AbiError ByteString
|
||||
wordAt o ret
|
||||
| o + abiWordSize <= B.length ret = Right $ B.take abiWordSize $ B.drop o ret
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
{-# LANGUAGE NamedFieldPuns #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
|
||||
module StoreService.Names.Census
|
||||
( Census (..),
|
||||
CensusName (..),
|
||||
Records (..),
|
||||
recordsLabel,
|
||||
nameCensus,
|
||||
)
|
||||
where
|
||||
|
||||
import Control.Concurrent.Async (mapConcurrently)
|
||||
import Control.Monad.Except (ExceptT (..), runExceptT)
|
||||
import Data.ByteString (ByteString)
|
||||
import qualified Data.ByteString as B
|
||||
import Data.List (unfoldr)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Data.Word (Word64)
|
||||
import Simplex.Messaging.Crypto (keccak256)
|
||||
import Simplex.Messaging.Eth.Address (Address, unAddress)
|
||||
import StoreService.Config (ChainConfig (..))
|
||||
import StoreService.Names.Abi
|
||||
import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError, latestBlock)
|
||||
import StoreService.Names.Contracts (Contracts, callAt)
|
||||
|
||||
data Records = RNone | RContact | RChannel | RBoth | ROtherResolver
|
||||
deriving (Eq, Ord, Show, Enum, Bounded)
|
||||
|
||||
recordsLabel :: Records -> Text
|
||||
recordsLabel r = case r of
|
||||
RNone -> "none"
|
||||
RContact -> "contact"
|
||||
RChannel -> "channel"
|
||||
RBoth -> "both"
|
||||
ROtherResolver -> "other_resolver"
|
||||
|
||||
-- | Level 2 is a name such as alice.simplex, 3 its subname; the length is in characters.
|
||||
data CensusName = CensusName {cnLevel :: Int, cnLength :: Int, cnRecords :: Records}
|
||||
deriving (Eq, Show)
|
||||
|
||||
data Census = Census {cBlock :: Word64, cNames :: [CensusName]}
|
||||
deriving (Eq, Show)
|
||||
|
||||
data CensusEnv = CensusEnv
|
||||
{ ceContracts :: Contracts,
|
||||
ceAt :: BlockTag,
|
||||
ceRegistry :: Address,
|
||||
ceSubnames :: Address,
|
||||
ceResolver :: Address
|
||||
}
|
||||
|
||||
type Node = (ByteString, Text)
|
||||
|
||||
readsAtOnce :: Int
|
||||
readsAtOnce = 16
|
||||
|
||||
childrenPage :: Integer
|
||||
childrenPage = 100
|
||||
|
||||
contactKey, channelKey :: ByteString
|
||||
contactKey = "simplex.contact"
|
||||
channelKey = "simplex.channel"
|
||||
|
||||
-- | Every live name and subname at the latest block: names while nameExpires is after the block's time, subnames while
|
||||
-- the subname registrar names an owner. A name whose resolver is not the configured one is counted without its records.
|
||||
nameCensus :: ChainClient -> Contracts -> IO (Either ChainError Census)
|
||||
nameCensus c@ChainClient {ccConfig = ChainConfig {cBaseRegistrar, cResolver}} ct = runExceptT $ do
|
||||
Block {blockNumber, blockTimestamp} <- ExceptT $ latestBlock c
|
||||
let at = AtBlock blockNumber
|
||||
registry <- ExceptT $ callAt ct at cBaseRegistrar ("ens()", "") decodeAddress
|
||||
subnames <- ExceptT $ callAt ct at cBaseRegistrar ("subnameHook()", "") decodeAddress
|
||||
base <- ExceptT $ callAt ct at cBaseRegistrar ("baseNode()", "") decodeBytes32
|
||||
total <- ExceptT $ callAt ct at cBaseRegistrar ("totalSupply()", "") decodeUint256
|
||||
let env = CensusEnv {ceContracts = ct, ceAt = at, ceRegistry = registry, ceSubnames = subnames, ceResolver = cResolver}
|
||||
registrar :: ByteString -> Integer -> (ByteString -> Either AbiError a) -> IO (Either ChainError a)
|
||||
registrar signature i = readAt env cBaseRegistrar signature (uint256 i)
|
||||
ids <- readAll [registrar "tokenByIndex(uint256)" i decodeUint256 | i <- [0 .. total - 1]]
|
||||
expiries <- readAll [registrar "nameExpires(uint256)" i decodeUint256 | i <- ids]
|
||||
let live = [i | (i, expiry) <- zip ids expiries, expiry > toInteger blockTimestamp]
|
||||
labels <- readAll [registrar "labelOf(uint256)" i decodeString | i <- live]
|
||||
Census blockNumber <$> levels env 2 [(keccak256 (base <> uint256 i), label) | (i, label) <- zip live labels]
|
||||
|
||||
levels :: CensusEnv -> Int -> [Node] -> ExceptT ChainError IO [CensusName]
|
||||
levels env level nodes
|
||||
| null nodes = pure []
|
||||
| otherwise = do
|
||||
records <- recordsOf env nodes
|
||||
children <- childrenOf env nodes
|
||||
(zipWith (\(_, label) r -> CensusName level (T.length label) r) nodes records <>) <$> levels env (level + 1) children
|
||||
|
||||
recordsOf :: CensusEnv -> [Node] -> ExceptT ChainError IO [Records]
|
||||
recordsOf env@CensusEnv {ceRegistry, ceResolver} nodes = do
|
||||
resolvers <- readAll [readAt env ceRegistry "resolver(bytes32)" node decodeAddress | (node, _) <- nodes]
|
||||
readAll [if r == ceResolver then textsOf node else pure (Right ROtherResolver) | ((node, _), r) <- zip nodes resolvers]
|
||||
where
|
||||
textsOf node = do
|
||||
contact <- readAt env ceResolver "text(bytes32,string)" (textArgs node contactKey) decodeString
|
||||
channel <- readAt env ceResolver "text(bytes32,string)" (textArgs node channelKey) decodeString
|
||||
pure $ recordsOfTexts <$> contact <*> channel
|
||||
textArgs node key = abiTuple [Static node, Dynamic (abiBytes key)]
|
||||
|
||||
recordsOfTexts :: Text -> Text -> Records
|
||||
recordsOfTexts contact channel = case (T.null contact, T.null channel) of
|
||||
(True, True) -> RNone
|
||||
(False, True) -> RContact
|
||||
(True, False) -> RChannel
|
||||
(False, False) -> RBoth
|
||||
|
||||
-- | A subname the subname registrar no longer names an owner of, as of a lapsed name or a previous owner, is left out.
|
||||
childrenOf :: CensusEnv -> [Node] -> ExceptT ChainError IO [Node]
|
||||
childrenOf env@CensusEnv {ceSubnames} parents = do
|
||||
counts <- readAll [readAt env ceSubnames "childrenLength(bytes32)" node decodeUint256 | (node, _) <- parents]
|
||||
let pages = [(node, start) | ((node, _), count) <- zip parents counts, start <- [0, childrenPage .. count - 1]]
|
||||
listed <- readAll [readAt env ceSubnames "getChildren(bytes32,uint256,uint256)" (node <> uint256 start <> uint256 childrenPage) decodeLabels | (node, start) <- pages]
|
||||
let found = [(keccak256 (node <> labelHash), label) | ((node, _), children) <- zip pages listed, (labelHash, label) <- children]
|
||||
owners <- readAll [readAt env ceSubnames "ownerOf(uint256)" node decodeAddress | (node, _) <- found]
|
||||
pure [child | (child, owner) <- zip found owners, not (B.all (== 0) (unAddress owner))]
|
||||
|
||||
readAt :: CensusEnv -> Address -> ByteString -> ByteString -> (ByteString -> Either AbiError a) -> IO (Either ChainError a)
|
||||
readAt CensusEnv {ceContracts, ceAt} to signature args = callAt ceContracts ceAt to (signature, args)
|
||||
|
||||
readAll :: [IO (Either ChainError a)] -> ExceptT ChainError IO [a]
|
||||
readAll calls = concat <$> mapM (ExceptT . fmap sequence . mapConcurrently id) (groupsOf readsAtOnce calls)
|
||||
|
||||
groupsOf :: Int -> [a] -> [[a]]
|
||||
groupsOf n = unfoldr (\xs -> if null xs then Nothing else Just (splitAt n xs))
|
||||
@@ -22,6 +22,7 @@ module StoreService.Names.Chain
|
||||
transactionReceipt,
|
||||
transactionKnown,
|
||||
transactionCount,
|
||||
registrarBalance,
|
||||
estimateGas,
|
||||
recentTip,
|
||||
signTransaction,
|
||||
@@ -180,6 +181,9 @@ transactionKnown c (TxHash h) =
|
||||
transactionCount :: ChainClient -> Address -> BlockTag -> IO (Either ChainError Word64)
|
||||
transactionCount c a tag = rpc c "eth_getTransactionCount" [J.toJSON a, blockTagParam tag] quantity
|
||||
|
||||
registrarBalance :: ChainClient -> IO (Either ChainError Integer)
|
||||
registrarBalance c@ChainClient {ccRegistrar} = rpc c "eth_getBalance" [J.toJSON ccRegistrar, blockTagParam Latest] natural
|
||||
|
||||
blockTagParam :: BlockTag -> J.Value
|
||||
blockTagParam = \case
|
||||
Latest -> "latest"
|
||||
@@ -386,8 +390,13 @@ nullable p = \case
|
||||
v -> Just <$> p v
|
||||
|
||||
quantity :: forall a. (Integral a, Bounded a) => J.Value -> JT.Parser a
|
||||
quantity = J.withText "quantity" $ \t -> case T.stripPrefix "0x" t of
|
||||
Just h | T.length h <= maxQuantityDigits, [(n, "")] <- readHex (T.unpack h), n <= toInteger (maxBound :: a) -> pure $ fromInteger n
|
||||
quantity =
|
||||
natural >=> \n ->
|
||||
if n <= toInteger (maxBound :: a) then pure $ fromInteger n else fail "not a quantity"
|
||||
|
||||
natural :: J.Value -> JT.Parser Integer
|
||||
natural = J.withText "quantity" $ \t -> case T.stripPrefix "0x" t of
|
||||
Just h | T.length h <= maxQuantityDigits, [(n, "")] <- readHex (T.unpack h) -> pure n
|
||||
_ -> fail "not a quantity"
|
||||
|
||||
bytes :: J.Value -> JT.Parser ByteString
|
||||
|
||||
@@ -11,6 +11,7 @@ module StoreService.Names.Contracts
|
||||
CommitmentAges (..),
|
||||
Terms (..),
|
||||
TermsRefusal (..),
|
||||
RegistrarFunds (..),
|
||||
newContracts,
|
||||
secondsPerYear,
|
||||
nameStatus,
|
||||
@@ -18,6 +19,7 @@ module StoreService.Names.Contracts
|
||||
textRecord,
|
||||
chainPriceCents,
|
||||
sponsorNonce,
|
||||
registrarFunds,
|
||||
resolverNonce,
|
||||
namePolicy,
|
||||
forgetNamePolicy,
|
||||
@@ -28,6 +30,7 @@ module StoreService.Names.Contracts
|
||||
termsText,
|
||||
fetchCurrentTerms,
|
||||
runTermsFetch,
|
||||
termsHeld,
|
||||
policyNonce,
|
||||
acceptedTerms,
|
||||
confirmedTag,
|
||||
@@ -38,6 +41,7 @@ module StoreService.Names.Contracts
|
||||
commitmentAges,
|
||||
together,
|
||||
concurrent,
|
||||
callAt,
|
||||
)
|
||||
where
|
||||
|
||||
@@ -72,7 +76,7 @@ import Simplex.Messaging.SimplexName (LabelHash (..), SimplexTLD)
|
||||
import Simplex.Messaging.Util (catchOwn', tshow)
|
||||
import StoreService.Config (ChainConfig (..), localChainId)
|
||||
import StoreService.Names.Abi
|
||||
import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError (..), confirmedNumber, ethCallAt, fetchText, latestBlock, nodeChainId)
|
||||
import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError (..), confirmedNumber, ethCallAt, fetchText, latestBlock, nodeChainId, registrarBalance)
|
||||
import StoreService.Names.Registration (channelRecordKey, contactRecordKey, labelNode, nodeBytes)
|
||||
import StoreService.Names.Revert (RevertName (..))
|
||||
import StoreService.Names.Store (getTermsText, setTermsText)
|
||||
@@ -125,6 +129,9 @@ data Terms = Terms {tmLink :: Text, tmHash :: ByteString}
|
||||
data TermsRefusal = TRLink | TRWait Int
|
||||
deriving (Eq, Show)
|
||||
|
||||
data RegistrarFunds = RegistrarFunds {rfBalanceWei :: Integer, rfAllowanceCents :: Integer}
|
||||
deriving (Eq, Show)
|
||||
|
||||
secondsPerYear :: Integer
|
||||
secondsPerYear = 31536000
|
||||
|
||||
@@ -210,6 +217,12 @@ chainPriceCents ct@Contracts {ctMinPriceCentsPerYear} len years = fmap (>>= chec
|
||||
sponsorNonce :: Contracts -> Address -> IO (Either ChainError Word64)
|
||||
sponsorNonce ct owner = call ct (cSponsor $ chainConfig ct) ("nonces(address)", addressWord owner) decodeWord64
|
||||
|
||||
registrarFunds :: Contracts -> IO (Either ChainError RegistrarFunds)
|
||||
registrarFunds ct@Contracts {ctChain = c@ChainClient {ccRegistrar}} =
|
||||
together $ RegistrarFunds <$> concurrent (registrarBalance c) <*> concurrent allowance
|
||||
where
|
||||
allowance = call ct (cSponsor $ chainConfig ct) ("registrarAllowance(address)", addressWord ccRegistrar) $ fmap (`div` attoUsdPerCent) . decodeUint256
|
||||
|
||||
resolverNonce :: Contracts -> Address -> IO (Either ChainError Word64)
|
||||
resolverNonce ct owner = call ct (resolver ct) ("nonces(address)", addressWord owner) decodeWord64
|
||||
|
||||
@@ -254,6 +267,14 @@ termsText ct@Contracts {ctTermsTexts, ctFetching} t@Terms {tmHash} =
|
||||
where
|
||||
kept = M.lookup tmHash <$> readIORef ctTermsTexts
|
||||
|
||||
-- | Each kind of terms the name policy links, with whether its text is held; nothing is fetched.
|
||||
termsHeld :: Contracts -> IO (Either ChainError [(Text, Bool)])
|
||||
termsHeld ct@Contracts {ctTermsTexts} = termsLinks ct >>= mapM held
|
||||
where
|
||||
held (generalLink, communityLink) = do
|
||||
texts <- readIORef ctTermsTexts
|
||||
pure [(kind, M.member tmHash texts) | (kind, Just Terms {tmHash}) <- [("general", termsOf generalLink), ("community", termsOf communityLink)]]
|
||||
|
||||
fetchCurrentTerms :: Contracts -> IO ()
|
||||
fetchCurrentTerms ct =
|
||||
termsLinks ct >>= \case
|
||||
|
||||
@@ -39,6 +39,7 @@ module StoreService.Names.Store
|
||||
EditState (..),
|
||||
RecordEdit (..),
|
||||
NewRecordEdit (..),
|
||||
NamesCounts (..),
|
||||
findLimit,
|
||||
orderHash,
|
||||
orderLive,
|
||||
@@ -99,6 +100,7 @@ module StoreService.Names.Store
|
||||
relayedEditTimes,
|
||||
getTermsText,
|
||||
setTermsText,
|
||||
namesCounts,
|
||||
)
|
||||
where
|
||||
|
||||
@@ -487,6 +489,19 @@ data NewRecordEdit = NewRecordEdit
|
||||
nreBodyHash :: ByteString
|
||||
}
|
||||
|
||||
data NamesCounts = NamesCounts
|
||||
{ ncPhases :: [(NamePhase, Int)],
|
||||
ncQueueOldestChange :: Maybe UTCTime,
|
||||
ncPendingTransactions :: Int,
|
||||
ncOldestPending :: Maybe UTCTime,
|
||||
ncOpenInvoices :: Int,
|
||||
ncHeldBalances :: Int,
|
||||
ncHeldCents :: Int,
|
||||
ncPendingEdits :: Int,
|
||||
ncOldestEdit :: Maybe UTCTime
|
||||
}
|
||||
deriving (Eq, Show)
|
||||
|
||||
findLimit :: Int
|
||||
findLimit = 50
|
||||
|
||||
@@ -1568,6 +1583,28 @@ setTermsText db termsHash text =
|
||||
|]
|
||||
(Binary termsHash, Binary text)
|
||||
|
||||
namesCounts :: DB.Connection -> IO NamesCounts
|
||||
namesCounts db = do
|
||||
ncPhases <- DB.query_ db "SELECT phase, COUNT(*) FROM sx_badge_service_name_registrations GROUP BY phase"
|
||||
DB.query
|
||||
db
|
||||
[sql|
|
||||
SELECT
|
||||
(SELECT MIN(updated_at) FROM sx_badge_service_name_registrations WHERE phase IN (?,?,?) OR (phase = ? AND label IS NOT NULL)),
|
||||
(SELECT COUNT(DISTINCT registrar_nonce) FROM sx_badge_service_name_transactions WHERE status = ?),
|
||||
(SELECT MIN(sent_at) FROM sx_badge_service_name_transactions WHERE status = ?),
|
||||
(SELECT COUNT(*) FROM sx_badge_service_name_invoices ni JOIN sx_badge_service_invoices i ON i.invoice_id = ni.invoice_id WHERE i.status = ?),
|
||||
(SELECT COUNT(*) FROM sx_badge_service_name_balances WHERE reserved_by IS NOT NULL),
|
||||
(SELECT COALESCE(SUM(amount), 0) FROM sx_badge_service_name_balances WHERE reserved_by IS NOT NULL),
|
||||
(SELECT COUNT(*) FROM sx_badge_service_name_record_edits WHERE state = ?),
|
||||
(SELECT MIN(created_at) FROM sx_badge_service_name_record_edits WHERE state = ?)
|
||||
|]
|
||||
(PQueued, PCommitting, PRegistering, PCommitted, TSPending, TSPending, invoiceStatusText ISOpen, pendingEdit, pendingEdit)
|
||||
>>= \case
|
||||
[(ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit)] ->
|
||||
pure NamesCounts {ncPhases, ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit}
|
||||
_ -> E.throwIO $ StoreDecodeError "names counts: not one row"
|
||||
|
||||
int64 :: Word64 -> Int64
|
||||
int64 = fromIntegral
|
||||
|
||||
|
||||
@@ -126,6 +126,8 @@ data PollerEnv = PollerEnv
|
||||
pePoll :: PollConfig,
|
||||
peSkipped :: TVar (Map Text UTCTime),
|
||||
peListedAt :: TVar (Maybe UTCTime),
|
||||
-- | When each provider's list last succeeded, by its text encoding.
|
||||
peListedOk :: TVar (Map Text UTCTime),
|
||||
peStrayEvery :: NominalDiffTime,
|
||||
-- When the cancel first failed, for orders older than 72 hours.
|
||||
peCancelFailed :: TVar (Map Text UTCTime),
|
||||
@@ -142,13 +144,14 @@ newPollerEnv :: DBStore -> Waiters -> Maybe Phases -> ReadHints -> [Provider] ->
|
||||
newPollerEnv peStore peWaiters pePhases peHints peProviders pePoll = do
|
||||
peSkipped <- newTVarIO M.empty
|
||||
peListedAt <- newTVarIO Nothing
|
||||
peListedOk <- newTVarIO M.empty
|
||||
peCancelFailed <- newTVarIO M.empty
|
||||
peReadFailed <- newTVarIO M.empty
|
||||
peCleanedAt <- newTVarIO Nothing
|
||||
peCleanupLeft <- newTVarIO []
|
||||
peCreditedAt <- newTVarIO Nothing
|
||||
peRates <- newTVarIO M.empty
|
||||
pure PollerEnv {peStore, peWaiters, pePhases, peHints, peProviders, pePoll, peSkipped, peListedAt, peStrayEvery = strayListInterval, peCancelFailed, peReadFailed, peCleanedAt, peCleanupLeft, peCreditedAt, peRates}
|
||||
pure PollerEnv {peStore, peWaiters, pePhases, peHints, peProviders, pePoll, peSkipped, peListedAt, peListedOk, peStrayEvery = strayListInterval, peCancelFailed, peReadFailed, peCleanedAt, peCleanupLeft, peCreditedAt, peRates}
|
||||
|
||||
-- | Read providers before the sweep, and leave out of the sweep every invoice whose read or settlement failed,
|
||||
-- so an invoice with money in it is never expired. After a list that cannot account for every invoice, only overdue
|
||||
@@ -241,13 +244,14 @@ readRows env now rows =
|
||||
-- | The refs of ours the list did not account for, so the sweep does not expire one over money it missed;
|
||||
-- Nothing when it cannot name them all.
|
||||
listPass :: PollerEnv -> UTCTime -> Provider -> IO (Maybe (Set Text))
|
||||
listPass env now p =
|
||||
listPass env@PollerEnv {peListedOk} now p =
|
||||
pListOpen p >>= \case
|
||||
Left (ProviderError e) -> do
|
||||
due <- dueToWarn env now ("list failed: " <> tshow (pProvider p))
|
||||
when due $ logWarn ("badge poller: " <> tshow (pProvider p) <> " list failed; every invoice waits for the next pass: " <> e)
|
||||
pure Nothing
|
||||
Right ListPass {lpMoved, lpSkipped} -> do
|
||||
atomically $ modifyTVar' peListedOk $ M.insert (textEncode $ pProvider p) now
|
||||
owners <- mapM (\s -> safelyWith (skipWhat s) SkipUnaccounted (reportSkip env (pProvider p) now s)) lpSkipped
|
||||
reportStrangers env (pProvider p) now [reason | (SkipStranger, (_, reason)) <- zip owners lpSkipped]
|
||||
unsettled <- filterM (\m -> not <$> safely (settleWhat m) (settleMoved env (pProvider p) now m)) lpMoved
|
||||
|
||||
@@ -47,9 +47,10 @@ where
|
||||
|
||||
import StoreService.Catalog (StoreProduct (..), defaultCatalog, storeProduct)
|
||||
import StoreService.Codes (issueFailedText, issueOneCode, singleUse)
|
||||
import StoreService.Config (AppleStoreConfig (..), BadgeIssuerKey (..), ChainConfig (..), GroupConfig (..), NamesConfig (..), PlayStoreConfig (..), ServiceConfig (..), minNameYears, readServiceConfig)
|
||||
import StoreService.Config (AppleStoreConfig (..), BadgeIssuerKey (..), ChainConfig (..), GroupConfig (..), ListenerConfig (..), NamesConfig (..), PlayStoreConfig (..), ServiceConfig (..), minNameYears, readServiceConfig)
|
||||
import StoreService.Group (GroupEvent, ensureManagedGroup, fmtTime, groupEvent, hasTracker, joinerAcceptance, refreshTracker, revokeWithTracker, runGroupLane)
|
||||
import StoreService.Group.Command (MoneyCmd (..), badgeTypeP, balanceArgsP, closeArgsP, codeP, maxMonths, maxNameYears, nameCodeP, prepaidCodeP, recordRefundArgsP, textTokenP)
|
||||
import StoreService.Metrics (MetricsEnv (..), NamesSource (..), runMetrics)
|
||||
import StoreService.Names.Abi (hex)
|
||||
import StoreService.Names.Balances (RefundRecord (..), RefundRefusal (..), recordRefund)
|
||||
import StoreService.Names.Chain (ChainClient, newChainClient, nodeChainId)
|
||||
@@ -65,8 +66,8 @@ import StoreService.Names.Turns (Turns, newTurns)
|
||||
import StoreService.Names.View (addressText)
|
||||
import StoreService.Names.Worker (CloseRefusal (..), Closed (..), closeRegistration, newWorker, runCodeSweep, runWorkers)
|
||||
import StoreService.Options
|
||||
import StoreService.Poller (expiryGrace, newPollerEnv, newReadHints, runPoller)
|
||||
import StoreService.Providers (Provider)
|
||||
import StoreService.Poller (PollerEnv (peListedOk), expiryGrace, newPollerEnv, newReadHints, runPoller)
|
||||
import StoreService.Providers (Provider (pProvider))
|
||||
import StoreService.Providers.BTCPay (btcpayProvider)
|
||||
import StoreService.Providers.Stripe (stripeProvider)
|
||||
import StoreService.Store
|
||||
@@ -78,7 +79,7 @@ import StoreService.StoreReceipts.Google (PlayVerifier (..), playStoreVerifier)
|
||||
import StoreService.StoreReceipts.Mock (newMockStoreVerifier)
|
||||
import StoreService.Waiters (Waiters, newWaiters)
|
||||
import StoreService.Web.Http (Bucket, takeFrom)
|
||||
import StoreService.Web.Server (checkSiteDirs, exportWebapp, newWebEnv, runWebListeners)
|
||||
import StoreService.Web.Server (WebEnv (weWebhookAt), checkSiteDirs, exportWebapp, newWebEnv, runWebListeners)
|
||||
import Control.Applicative (optional, (<|>))
|
||||
import Control.Concurrent.STM
|
||||
import Control.Logger.Simple (logError, logNote, logWarn)
|
||||
@@ -119,7 +120,7 @@ import Simplex.Messaging.Agent.Store.Common (DBStore)
|
||||
import qualified Simplex.Messaging.Agent.Store.DB as DB
|
||||
import qualified Simplex.Messaging.Crypto as C
|
||||
import Simplex.Messaging.Crypto.BBS (bbsPublicKey)
|
||||
import Simplex.Messaging.Encoding.String (strEncode)
|
||||
import Simplex.Messaging.Encoding.String (strEncode, textEncode)
|
||||
import Simplex.Messaging.SimplexName (LabelHash (..), SimplexTLD)
|
||||
import Simplex.Messaging.Util (catchOwn, catchOwn', raceAny_, safeDecodeUtf8, tshow)
|
||||
import Simplex.Messaging.Version (isCompatible)
|
||||
@@ -328,7 +329,10 @@ storeService opts serviceCfg baseCfg env = do
|
||||
exportWebapp sc
|
||||
pollerEnv <- newPollerEnv chatStore ws (nePhases <$> namesEnv) hints providers (poll sc)
|
||||
nameLanes <- maybe (pure []) namesLanes namesEnv
|
||||
pure $ [runWebListeners webEnv, runPoller pollerEnv] <> nameLanes
|
||||
let metricsEnv = MetricsEnv {meStore = chatStore, meProviders = map (textEncode . pProvider) providers, meListedOk = peListedOk pollerEnv, meWebhookAt = weWebhookAt webEnv, meNames = namesSource <$> namesEnv}
|
||||
namesSource NamesEnv {neChain, neContracts} = NamesSource {nsChain = neChain, nsContracts = neContracts}
|
||||
metricsLane = [runMetrics (listener sc) port metricsEnv | Just port <- [lMetricsPort $ listener sc]]
|
||||
pure $ [runWebListeners webEnv, runPoller pollerEnv] <> nameLanes <> metricsLane
|
||||
|
||||
startNames :: ChatController -> Turns -> Waiters -> [Provider] -> NamesConfig -> IO NamesEnv
|
||||
startNames cc@ChatController {chatStore} turns waiters providers nc@NamesConfig {nTLD, nCodesOnly, nMinPriceCentsPerYear, nChain = ChainConfig {cChainId}} =
|
||||
|
||||
@@ -50,6 +50,8 @@ module StoreService.Store
|
||||
getEditableTrackers,
|
||||
RevokeResult (..),
|
||||
revokeCode,
|
||||
BadgeCounts (..),
|
||||
badgeCounts,
|
||||
)
|
||||
where
|
||||
|
||||
@@ -65,7 +67,7 @@ import Data.Time.Clock (UTCTime)
|
||||
import Simplex.Chat.Badges (BadgeCredential, BadgeMasterKey (..), BadgeType)
|
||||
import Simplex.Chat.Badges.Ledger
|
||||
import Simplex.Chat.StoreService (StatementCreditType (..), StatementDebitType (..), StatementEntry (..), StatementEntryType (..), SubscriptionState)
|
||||
import Simplex.Chat.Badges.Types (BadgeCodePaymentStatus, BadgePurchaseStatus (..))
|
||||
import Simplex.Chat.Badges.Types (BadgeCodePaymentStatus (..), BadgePurchaseStatus (..))
|
||||
import Simplex.Chat.PaymentService.Types (CurrencyAmount (..), InvoiceId (..), PaymentProvider (..), PaymentStatus (..))
|
||||
import Simplex.Chat.Store.Shared (insertedRowId)
|
||||
import Simplex.Messaging.Agent.Store.DB (Binary (..), BoolInt (..))
|
||||
@@ -647,3 +649,26 @@ getEditableTrackers db sentAfter =
|
||||
ORDER BY badge_code_id
|
||||
|]
|
||||
(Only sentAfter)
|
||||
|
||||
data BadgeCounts = BadgeCounts
|
||||
{ bcPurchases :: [(Text, Int)],
|
||||
bcSubscriptions :: [(Text, Int)],
|
||||
bcOpenCodes :: Int
|
||||
}
|
||||
deriving (Eq, Show)
|
||||
|
||||
-- | Open codes are paid or free, not revoked, not expired, and have redemptions left.
|
||||
badgeCounts :: DB.Connection -> UTCTime -> IO BadgeCounts
|
||||
badgeCounts db now = do
|
||||
bcPurchases <- DB.query_ db "SELECT status, COUNT(*) FROM sx_badge_service_badge_purchases GROUP BY status ORDER BY status"
|
||||
bcSubscriptions <- DB.query_ db "SELECT subscription_state, COUNT(*) FROM sx_badge_service_payments WHERE subscription_state IS NOT NULL GROUP BY subscription_state ORDER BY subscription_state"
|
||||
bcOpenCodes <-
|
||||
maybeFirstRow' 0 fromOnly $
|
||||
DB.query
|
||||
db
|
||||
[sql|
|
||||
SELECT COUNT(*) FROM sx_badge_service_badge_codes
|
||||
WHERE code_payment_status IN (?,?) AND redeem_count < redeem_limit AND revoked_at IS NULL AND (expires_at IS NULL OR expires_at > ?)
|
||||
|]
|
||||
(textEncode CPSPaid, textEncode CPSFree, now)
|
||||
pure BadgeCounts {bcPurchases, bcSubscriptions, bcOpenCodes}
|
||||
@@ -25,6 +25,7 @@ module StoreService.Store.Invoices
|
||||
codeHashExists,
|
||||
OverdueInvoice (..),
|
||||
overdueInvoices,
|
||||
openInvoiceCounts,
|
||||
expireOverdueInvoice,
|
||||
lockInvoice,
|
||||
unfundedQuery,
|
||||
@@ -267,6 +268,13 @@ qOverdueInvoices =
|
||||
<> unfundedOnly
|
||||
<> "ORDER BY @invoices.created_at"
|
||||
|
||||
qOpenInvoiceCounts :: Query
|
||||
qOpenInvoiceCounts =
|
||||
mkQuery $
|
||||
"SELECT provider, COUNT(*), COALESCE(SUM(CASE WHEN expires_at < ? "
|
||||
<> unfundedOnly
|
||||
<> "THEN 1 ELSE 0 END), 0) FROM @invoices WHERE status = 'open' GROUP BY provider ORDER BY provider"
|
||||
|
||||
qExpireOverdue :: Query
|
||||
qExpireOverdue =
|
||||
mkQuery $
|
||||
@@ -617,6 +625,10 @@ overdueInvoices st cutoff = do
|
||||
logError ("store: overdue invoice " <> oiProviderRef <> " names provider " <> providerTxt <> ", which this build does not know, so it is left open")
|
||||
pure Nothing
|
||||
|
||||
-- | Open invoices by provider, each with how many of them the sweep would expire at this cutoff.
|
||||
openInvoiceCounts :: DB.Connection -> UTCTime -> IO [(Text, Int, Int)]
|
||||
openInvoiceCounts db cutoff = DB.query db qOpenInvoiceCounts (Only (truncateToSecond cutoff))
|
||||
|
||||
-- | Expires the invoice if it is still open, has received no money, and is past the cutoff.
|
||||
expireOverdueInvoice :: DB.Connection -> UTCTime -> InvoiceId -> IO Bool
|
||||
expireOverdueInvoice db cutoff' (InvoiceId i) = (> 0) <$> executeChanging db qExpireOverdue (cutoff, cutoff, i)
|
||||
|
||||
@@ -34,6 +34,7 @@ import StoreService.Web.Names (namesApp, namesRoutes)
|
||||
import StoreService.Web.Names.Orders (nameInvoiceHandler, nameInvoiceView)
|
||||
import StoreService.Web.Names.Registration (heldFor, registrationChangedResponse, versionParam)
|
||||
import Control.Concurrent.Async (race)
|
||||
import Control.Concurrent.STM (TVar, atomically, modifyTVar', newTVarIO)
|
||||
import qualified Control.Exception as E
|
||||
import Control.Logger.Simple (logError, logNote, logWarn)
|
||||
import Control.Monad (filterM, forM_, void, when)
|
||||
@@ -46,6 +47,8 @@ import qualified Data.ByteString.Lazy as LB
|
||||
import Data.Char (toLower)
|
||||
import Data.Functor ((<&>))
|
||||
import Data.List (isPrefixOf)
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Map.Strict as M
|
||||
import Data.Maybe (catMaybes, fromMaybe, isJust)
|
||||
import Data.String (fromString)
|
||||
import Data.Text (Text)
|
||||
@@ -80,7 +83,9 @@ data WebEnv = WebEnv
|
||||
-- | @Nothing@ when the ini has no @[names]@ section.
|
||||
weNames :: Maybe NamesEnv,
|
||||
-- | The name orders' part of weNames, which tests set without a chat controller.
|
||||
weOrders :: Maybe OrdersEnv
|
||||
weOrders :: Maybe OrdersEnv,
|
||||
-- | When each provider last delivered a webhook that verified, by its text encoding.
|
||||
weWebhookAt :: TVar (Map Text UTCTime)
|
||||
}
|
||||
|
||||
-- | A built site, served when @serve_webapp@ is on.
|
||||
@@ -95,7 +100,8 @@ newWebEnv weStore weConfig weWaiters weHints weProviders weNames = do
|
||||
weHttp <- newHttpEnv lc
|
||||
weBadgesSite <- if lServeWebapp then Just <$> loadSite (keyFills weConfig) lBadgesStaticDir else pure Nothing
|
||||
weNamesSite <- mapM (loadSite (namesFills weConfig)) (lNamesPort *> lNamesStaticDir)
|
||||
pure WebEnv {weStore, weConfig, weWaiters, weProviders, weHoldMicros = holdMicros, weHints, weHttp, weBadgesSite, weNamesSite, weNames, weOrders = ordersEnv <$> weNames}
|
||||
weWebhookAt <- newTVarIO M.empty
|
||||
pure WebEnv {weStore, weConfig, weWaiters, weProviders, weHoldMicros = holdMicros, weHints, weHttp, weBadgesSite, weNamesSite, weNames, weOrders = ordersEnv <$> weNames, weWebhookAt}
|
||||
where
|
||||
lc@ListenerConfig {lServeWebapp, lBadgesStaticDir, lNamesStaticDir, lNamesPort} = listener weConfig
|
||||
loadSite fills dir = Site dir <$> prepareShell dir fills
|
||||
@@ -523,8 +529,9 @@ webhookHandler env@WebEnv {weStore, weHints} provider route req respond =
|
||||
case pVerifyWebhook p now (requestHeaders req) (LB.toStrict body) of
|
||||
Left (WebhookError e) -> refuse e
|
||||
Left (WebhookStale e) -> logRefusal logWarn (route <> " refused: " <> e) >> respond webhookRefused
|
||||
Right Nothing -> ignore body "nothing this service acts on"
|
||||
Right (Just ref) -> resolve body ref
|
||||
Right ref_ -> do
|
||||
atomically $ modifyTVar' (weWebhookAt env) $ M.insert (textEncode provider) now
|
||||
maybe (ignore body "nothing this service acts on") (resolve body) ref_
|
||||
refuse :: Text -> IO ResponseReceived
|
||||
refuse why = logRefusal logNote (route <> " refused: " <> why) >> respond webhookRefused
|
||||
-- anyone can post here, so the lines a flood would write are capped, not the refusals
|
||||
|
||||
@@ -23,6 +23,8 @@ trust_forwarded_for = on
|
||||
; names_static_dir = ./apps/simplex-domains/webapp/dist
|
||||
; names_export_dir = /srv/names-export
|
||||
; names_port = 8081
|
||||
; Prometheus metrics of the store, on 127.0.0.1 whatever host is
|
||||
; metrics_port = 9300
|
||||
|
||||
; omit this whole section to disable Bitcoin and Monero (the browser shows the provider-unavailable screen)
|
||||
[btcpay]
|
||||
|
||||
@@ -15,6 +15,9 @@ badges_export_dir = /srv/badges-export
|
||||
; the names site (simplex.domains), with [names] only: the folder it is exported to, which compose
|
||||
; bind-mounts to ./names for Caddy
|
||||
;names_export_dir = /srv/names-export
|
||||
; Prometheus metrics of the store, on 127.0.0.1 whatever host is; Caddy proxies them on the host's
|
||||
; monitoring site
|
||||
;metrics_port = 9300
|
||||
; on because Caddy proxies /api and /webhooks to this service; without it the rate limiter keys on
|
||||
; Caddy's address and shares one bucket across all callers
|
||||
trust_forwarded_for = on
|
||||
|
||||
@@ -460,8 +460,10 @@ executable simplex-store-service
|
||||
StoreService.Config
|
||||
StoreService.Group
|
||||
StoreService.Group.Command
|
||||
StoreService.Metrics
|
||||
StoreService.Names.Abi
|
||||
StoreService.Names.Balances
|
||||
StoreService.Names.Census
|
||||
StoreService.Names.Chain
|
||||
StoreService.Names.Codes
|
||||
StoreService.Names.Community
|
||||
@@ -801,6 +803,7 @@ test-suite simplex-chat-test
|
||||
Bots.StoreService.FakeStripe
|
||||
Bots.StoreService.GroupIntegrationTests
|
||||
Bots.StoreService.GroupTests
|
||||
Bots.StoreService.MetricsTests
|
||||
Bots.StoreService.NameRegistrationTests
|
||||
Bots.StoreService.NamesBalancesTests
|
||||
Bots.StoreService.NamesChainTests
|
||||
@@ -844,8 +847,10 @@ test-suite simplex-chat-test
|
||||
StoreService.Config
|
||||
StoreService.Group
|
||||
StoreService.Group.Command
|
||||
StoreService.Metrics
|
||||
StoreService.Names.Abi
|
||||
StoreService.Names.Balances
|
||||
StoreService.Names.Census
|
||||
StoreService.Names.Chain
|
||||
StoreService.Names.Codes
|
||||
StoreService.Names.Community
|
||||
|
||||
@@ -712,6 +712,7 @@ namesConfigTests = describe "StoreService names: config" $ do
|
||||
it "reads the names keys of [listener], defaulting to the image's build and port 8081" testNamesListenerKeys
|
||||
it "refuses the names keys of [listener] without [names]" testNamesSiteOnlyWithNames
|
||||
it "refuses a names_port outside 1-65535, on the badges_port, or with serve_webapp off" testNamesPortRules
|
||||
it "reads metrics_port, off when absent, also without [names], and refuses one outside 1-65535 or on the badges_port or names_port" testMetricsPort
|
||||
it "leaves the names side off when [names] is absent" testNamesAbsent
|
||||
it "refuses [names] without [chain], and [chain] without [names]" testNamesNeedsChain
|
||||
it "refuses years below 2" testNamesYearsBelowTwo
|
||||
@@ -844,6 +845,19 @@ testNamesPortRules = do
|
||||
where
|
||||
refused ks e = parseIni (namesIniWith ks) >>= (`shouldBe` e) . either id (const "parsed")
|
||||
|
||||
testMetricsPort :: IO ()
|
||||
testMetricsPort = do
|
||||
Right off <- parseIni (namesIni namesSection chainSection)
|
||||
lMetricsPort (listener off) `shouldBe` Nothing
|
||||
Right on <- parseIni (namesIniWith ["metrics_port = 9300"])
|
||||
lMetricsPort (listener on) `shouldBe` Just 9300
|
||||
Right badgesOnly <- parseIni (withListenerKeys ["metrics_port = 9300"])
|
||||
lMetricsPort (listener badgesOnly) `shouldBe` Just 9300
|
||||
forM_ ["0", "65536"] $ \v -> refused ["metrics_port = " <> v] "listener.metrics_port must be between 1 and 65535"
|
||||
forM_ ["8080", "8081"] $ \v -> refused ["metrics_port = " <> v] "listener.metrics_port must differ from listener.badges_port and listener.names_port"
|
||||
where
|
||||
refused ks e = parseIni (namesIniWith ks) >>= (`shouldBe` e) . either id (const "parsed")
|
||||
|
||||
-- | namesIni with these lines first in [listener].
|
||||
namesIniWith :: [T.Text] -> T.Text
|
||||
namesIniWith ks = withListenerKeys ks <> T.unlines (namesSection <> chainSection)
|
||||
|
||||
@@ -62,6 +62,7 @@ module Bots.StoreService.FakeEthNode
|
||||
fakeTip,
|
||||
fakeBaseFee,
|
||||
fakeMaxFeePerGas,
|
||||
genesisTime,
|
||||
)
|
||||
where
|
||||
|
||||
|
||||
@@ -0,0 +1,349 @@
|
||||
{-# LANGUAGE NamedFieldPuns #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
|
||||
module Bots.StoreService.MetricsTests (storeMetricsTests) where
|
||||
|
||||
import Bots.StoreService.FakeEthNode
|
||||
import Bots.StoreService.NamesContractsTests (minPriceCentsPerYear)
|
||||
import Bots.StoreService.NamesStoreTests (at, btc, insertInvoiceRow, insertProviderInvoice, newEdit, newReg, order, ownerA, ownerB, paidOrder, price, reveal, setChannel, t0)
|
||||
import Bots.StoreService.WebTests (digestFixture, statusOf, webGet, webRequestBody, withApp, withServiceStore)
|
||||
import Control.Concurrent.Async (cancel)
|
||||
import Control.Concurrent.STM (TVar, newTVarIO, readTVarIO)
|
||||
import Control.Exception (finally)
|
||||
import Control.Monad (forM_)
|
||||
import Crypto.Random (ChaChaDRG)
|
||||
import qualified Data.Aeson as J
|
||||
import Data.ByteString (ByteString)
|
||||
import qualified Data.ByteString as B
|
||||
import qualified Data.ByteString.Lazy as LB
|
||||
import qualified Data.Map.Strict as M
|
||||
import Data.Maybe (fromJust, isJust)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Data.Text.Encoding (decodeUtf8, encodeUtf8)
|
||||
import Data.Time.Clock (UTCTime, getCurrentTime)
|
||||
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
|
||||
import Data.Word (Word64, Word8)
|
||||
import Network.HTTP.Client (Response, responseBody, responseHeaders)
|
||||
import Network.HTTP.Types (hContentType)
|
||||
import Simplex.Chat.Badges.Types (BadgePurchaseStatus (..), SubscriptionState (..))
|
||||
import Simplex.Messaging.Agent.Store.Common (withTransaction)
|
||||
import Simplex.Messaging.Agent.Store.DB (Binary (..))
|
||||
import qualified Simplex.Messaging.Agent.Store.DB as DB
|
||||
import Simplex.Messaging.Crypto (keccak256)
|
||||
import qualified Simplex.Messaging.Crypto as C
|
||||
import Simplex.Messaging.Encoding.String (textEncode)
|
||||
import Simplex.Messaging.Eth.Address (Address, addressFromPrivateKey)
|
||||
import Simplex.Messaging.SimplexName (SimplexTLD (..))
|
||||
import Simplex.Messaging.Util (tshow)
|
||||
import StoreService.Metrics
|
||||
import StoreService.Names.Abi (AbiValue (..), abiArray, abiBytes, abiTuple, addressFromBytes, addressWord, selector, uint256)
|
||||
import StoreService.Names.Census (Census (..))
|
||||
import StoreService.Names.Chain (TxHash (..))
|
||||
import StoreService.Names.Contracts (newContracts, termsOf, termsText)
|
||||
import StoreService.Names.Phases (RegistrationId (..))
|
||||
import StoreService.Names.Store
|
||||
import StoreService.Store (badgeCounts)
|
||||
import StoreService.Store.Invoices (openInvoiceCounts)
|
||||
import System.Timeout (timeout)
|
||||
import Test.Hspec
|
||||
|
||||
storeMetricsTests :: Spec
|
||||
storeMetricsTests = describe "StoreService: metrics" $ do
|
||||
g <- runIO C.newRandom
|
||||
it "counts open invoices by provider with those the sweep would expire, badge purchases by status, subscriptions by state and open badge codes, with the providers' last list and webhook, and 0 for a provider, status or state without rows" testServiceCounts
|
||||
it "counts name orders by phase, the oldest change among the orders the service moves on, pending transactions by nonce, open name invoices, held balances and pending record edits" testNamesCounts
|
||||
it "reads the registrar's funds, the latest block and which terms texts are held, and keeps each part while a read fails" $ testChainRead g
|
||||
it "answers GET /metrics with the service metrics alone without names, and with the names metrics while the node holds the chain read" $ testScrape g
|
||||
it "counts live names and owned subnames by level and records, with their label lengths, and keeps the last census while one fails" $ testCensus g
|
||||
|
||||
testCensus :: TVar ChaChaDRG -> IO ()
|
||||
testCensus g = withServiceStore $ \st -> withFakeEthNode $ \node -> do
|
||||
c <- fakeChainClient g node
|
||||
ct <- newContracts st TLDSimplex minPriceCentsPerYear c
|
||||
let ns = NamesSource {nsChain = c, nsContracts = ct}
|
||||
answer to signature args = answerCall node to (selector signature <> args) . Returns
|
||||
str t = abiTuple [Dynamic (abiBytes $ encodeUtf8 t)]
|
||||
texts n (contact, channel) =
|
||||
forM_ [("simplex.contact", contact), ("simplex.channel", channel)] $ \(key, text) ->
|
||||
answer fakeResolver "text(bytes32,string)" (abiTuple [Static n, Dynamic (abiBytes key)]) (str text)
|
||||
live = genesisTime + 1000
|
||||
names :: [(Text, Word64, Maybe (Text, Text))]
|
||||
names =
|
||||
[ ("alicebakery", live, Just ("", "")),
|
||||
("bobsbakery", live, Just ("https://smp8.simplex.im/a#bob", "")),
|
||||
("carolshop", live, Nothing),
|
||||
("expiredname", genesisTime, Nothing)
|
||||
]
|
||||
aliceNode = keccak256 (censusBase <> keccak256 "alicebakery")
|
||||
shopNode = keccak256 (aliceNode <> keccak256 "shop")
|
||||
oldNode = keccak256 (aliceNode <> keccak256 "old")
|
||||
answer fakeBaseRegistrar "ens()" "" $ addressWord censusRegistry
|
||||
answer fakeBaseRegistrar "subnameHook()" "" $ addressWord censusSubnames
|
||||
answer fakeBaseRegistrar "baseNode()" "" censusBase
|
||||
answer fakeBaseRegistrar "totalSupply()" "" $ uint256 (length names)
|
||||
forM_ (zip [0 :: Int ..] names) $ \(i, (label, expiry, records)) -> do
|
||||
let labelHash = keccak256 (encodeUtf8 label)
|
||||
n = keccak256 (censusBase <> labelHash)
|
||||
answer fakeBaseRegistrar "tokenByIndex(uint256)" (uint256 i) labelHash
|
||||
answer fakeBaseRegistrar "nameExpires(uint256)" labelHash (uint256 expiry)
|
||||
answer fakeBaseRegistrar "labelOf(uint256)" labelHash (str label)
|
||||
answer censusRegistry "resolver(bytes32)" n $ addressWord (maybe censusOtherResolver (const fakeResolver) records)
|
||||
mapM_ (texts n) records
|
||||
answer censusSubnames "childrenLength(bytes32)" n $ uint256 (if n == aliceNode then 2 else 0 :: Int)
|
||||
answer censusSubnames "getChildren(bytes32,uint256,uint256)" (aliceNode <> uint256 (0 :: Int) <> uint256 (100 :: Int)) $
|
||||
abiTuple [Dynamic (uint256 (2 :: Int) <> keccak256 "shop" <> keccak256 "old"), Dynamic (abiArray [abiBytes "shop", abiBytes "old"])]
|
||||
answer censusSubnames "ownerOf(uint256)" shopNode $ addressWord censusOwner
|
||||
answer censusSubnames "ownerOf(uint256)" oldNode $ B.replicate 32 0
|
||||
answer censusRegistry "resolver(bytes32)" shopNode $ addressWord fakeResolver
|
||||
texts shopNode ("https://smp8.simplex.im/a#shop", "https://smp8.simplex.im/c#shop")
|
||||
answer censusSubnames "childrenLength(bytes32)" shopNode $ uint256 (0 :: Int)
|
||||
v <- newTVarIO noCensusRead
|
||||
readCensus ns v
|
||||
fmap (cBlock . fst) . crCensus <$> readTVarIO v `shouldReturn` Just 1
|
||||
let counted =
|
||||
[ "simplex_names{level=\"2\",records=\"none\"} 1",
|
||||
"simplex_names{level=\"2\",records=\"contact\"} 1",
|
||||
"simplex_names{level=\"2\",records=\"channel\"} 0",
|
||||
"simplex_names{level=\"2\",records=\"both\"} 0",
|
||||
"simplex_names{level=\"2\",records=\"other_resolver\"} 1",
|
||||
"simplex_names{level=\"3\",records=\"none\"} 0",
|
||||
"simplex_names{level=\"3\",records=\"contact\"} 0",
|
||||
"simplex_names{level=\"3\",records=\"channel\"} 0",
|
||||
"simplex_names{level=\"3\",records=\"both\"} 1",
|
||||
"simplex_names{level=\"3\",records=\"other_resolver\"} 0"
|
||||
]
|
||||
lengths =
|
||||
[ "simplex_name_length_bucket{level=\"2\",le=\"8.0\"} 0",
|
||||
"simplex_name_length_bucket{level=\"2\",le=\"9.0\"} 1",
|
||||
"simplex_name_length_bucket{level=\"2\",le=\"10.0\"} 2",
|
||||
"simplex_name_length_bucket{level=\"2\",le=\"11.0\"} 3",
|
||||
"simplex_name_length_bucket{level=\"2\",le=\"63.0\"} 3",
|
||||
"simplex_name_length_bucket{level=\"2\",le=\"+Inf\"} 3",
|
||||
"simplex_name_length_sum{level=\"2\"} 30",
|
||||
"simplex_name_length_count{level=\"2\"} 3",
|
||||
"simplex_name_length_bucket{level=\"3\",le=\"3.0\"} 0",
|
||||
"simplex_name_length_bucket{level=\"3\",le=\"4.0\"} 1",
|
||||
"simplex_name_length_sum{level=\"3\"} 4",
|
||||
"simplex_name_length_count{level=\"3\"} 1",
|
||||
"simplex_names_block 1",
|
||||
"simplex_names_refresh_failures_total 0"
|
||||
]
|
||||
censusLines = T.lines . censusText <$> readTVarIO v
|
||||
filter ("simplex_names{" `T.isPrefixOf`) <$> censusLines `shouldReturn` counted
|
||||
censusLines >>= \ls -> forM_ lengths $ \l -> (l, l `elem` ls) `shouldBe` (l, True)
|
||||
setFault node "eth_getBlockByNumber" HttpFailure
|
||||
readCensus ns v
|
||||
crFailures <$> readTVarIO v `shouldReturn` 1
|
||||
filter ("simplex_names{" `T.isPrefixOf`) <$> censusLines `shouldReturn` counted
|
||||
|
||||
censusRegistry, censusSubnames, censusOtherResolver, censusOwner :: Address
|
||||
censusRegistry = censusAddress 0x21
|
||||
censusSubnames = censusAddress 0x22
|
||||
censusOtherResolver = censusAddress 0x23
|
||||
censusOwner = censusAddress 0x24
|
||||
|
||||
censusAddress :: Word8 -> Address
|
||||
censusAddress = fromJust . addressFromBytes . B.replicate 20
|
||||
|
||||
censusBase :: ByteString
|
||||
censusBase = B.replicate 32 0x11
|
||||
|
||||
testServiceCounts :: IO ()
|
||||
testServiceCounts = withServiceStore $ \st -> do
|
||||
withTransaction st $ \db -> do
|
||||
insertProviderInvoice db "crypto" "c-overdue" (at 1) "open"
|
||||
insertProviderInvoice db "crypto" "c-open" (at 60) "open"
|
||||
insertProviderInvoice db "crypto" "c-paid" (at 1) "paid"
|
||||
insertProviderInvoice db "stripe" "s-funded" (at 1) "open"
|
||||
DB.execute
|
||||
db
|
||||
"INSERT INTO sx_badge_service_payments (payment_id, invoice_id, provider, amount, currency, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?)"
|
||||
("pay-s" :: Text, "s-funded" :: Text, "stripe" :: Text, price, "usd" :: Text, "pending" :: Text, t0, t0)
|
||||
forM_ [(1, PSIssued), (2, PSIssued), (3, PSSuperseded)] $ \(n, status) ->
|
||||
DB.execute
|
||||
db
|
||||
"INSERT INTO sx_badge_service_badge_purchases (purchase_key, master_key, initial_badge_type, current_badge_type, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?)"
|
||||
(Binary $ digestFixture n, Binary $ digestFixture (n + 100), "supporter" :: Text, "supporter" :: Text, textEncode status, t0, t0)
|
||||
forM_ [("sub-1", SSRenewing), ("sub-2", SSRenewing), ("sub-3", SSBillingRetry)] $ \(paymentId, state) ->
|
||||
DB.execute
|
||||
db
|
||||
"INSERT INTO sx_badge_service_payments (payment_id, provider, status, subscription_state, created_at, updated_at) VALUES (?,?,?,?,?,?)"
|
||||
(paymentId :: Text, "google" :: Text, "paid" :: Text, textEncode state, t0, t0)
|
||||
forM_ codes $ \(n, paymentStatus, (redeemLimit, redeemCount), revokedAt, expiresAt) ->
|
||||
DB.execute
|
||||
db
|
||||
"INSERT INTO sx_badge_service_badge_codes (code_hash, badge_type, months, code_payment_status, redeem_limit, redeem_count, revoked_at, expires_at, created_at) VALUES (?,?,?,?,?,?,?,?,?)"
|
||||
(Binary $ digestFixture n, "supporter" :: Text, 1 :: Int, paymentStatus, redeemLimit, redeemCount, revokedAt, expiresAt, t0)
|
||||
(invoices, badges) <- withTransaction st $ \db -> (,) <$> openInvoiceCounts db (at 10) <*> badgeCounts db (at 10)
|
||||
invoices `shouldBe` [("crypto", 2, 1), ("stripe", 1, 0)]
|
||||
let times = ProviderTimes {ptListedOk = M.fromList [("crypto", at 11), ("stripe", at 12)], ptWebhookAt = M.fromList [("stripe", at 13)]}
|
||||
typedSamples (serviceText ["crypto", "stripe"] invoices badges times)
|
||||
`shouldBe` [ "simplex_store_invoices_open{provider=\"crypto\"} 2",
|
||||
"simplex_store_invoices_open{provider=\"stripe\"} 1",
|
||||
"simplex_store_invoices_overdue{provider=\"crypto\"} 1",
|
||||
"simplex_store_invoices_overdue{provider=\"stripe\"} 0",
|
||||
"simplex_store_provider_list_ok_timestamp_seconds{provider=\"crypto\"} " <> seconds (at 11),
|
||||
"simplex_store_provider_list_ok_timestamp_seconds{provider=\"stripe\"} " <> seconds (at 12),
|
||||
"simplex_store_provider_webhook_timestamp_seconds{provider=\"stripe\"} " <> seconds (at 13),
|
||||
"simplex_store_badges_purchases{status=\"acquiring\"} 0",
|
||||
"simplex_store_badges_purchases{status=\"failed\"} 0",
|
||||
"simplex_store_badges_purchases{status=\"issued\"} 2",
|
||||
"simplex_store_badges_purchases{status=\"superseded\"} 1",
|
||||
"simplex_store_badges_subscriptions{state=\"billingRetry\"} 1",
|
||||
"simplex_store_badges_subscriptions{state=\"cancelled\"} 0",
|
||||
"simplex_store_badges_subscriptions{state=\"ended\"} 0",
|
||||
"simplex_store_badges_subscriptions{state=\"renewing\"} 2",
|
||||
"simplex_store_badges_codes_open 2"
|
||||
]
|
||||
closed <- withTransaction st $ \db -> do
|
||||
DB.execute_ db "UPDATE sx_badge_service_invoices SET status = 'expired' WHERE status = 'open'"
|
||||
openInvoiceCounts db (at 10)
|
||||
closed `shouldBe` []
|
||||
filter ("simplex_store_invoices_" `T.isPrefixOf`) (typedSamples (serviceText ["crypto", "stripe"] closed badges times))
|
||||
`shouldBe` [ "simplex_store_invoices_open{provider=\"crypto\"} 0",
|
||||
"simplex_store_invoices_open{provider=\"stripe\"} 0",
|
||||
"simplex_store_invoices_overdue{provider=\"crypto\"} 0",
|
||||
"simplex_store_invoices_overdue{provider=\"stripe\"} 0"
|
||||
]
|
||||
where
|
||||
-- open: the first two; then unpaid, used up, revoked, expired
|
||||
codes :: [(Word8, Text, (Int, Int), Maybe UTCTime, Maybe UTCTime)]
|
||||
codes =
|
||||
[ (1, "paid", (1, 0), Nothing, Nothing),
|
||||
(2, "free", (3, 1), Nothing, Just (at 120)),
|
||||
(3, "unpaid", (1, 0), Nothing, Nothing),
|
||||
(4, "paid", (2, 2), Nothing, Nothing),
|
||||
(5, "paid", (1, 0), Just (at 5), Nothing),
|
||||
(6, "paid", (1, 0), Nothing, Just (at 5))
|
||||
]
|
||||
|
||||
testNamesCounts :: IO ()
|
||||
testNamesCounts = withServiceStore $ \st -> do
|
||||
let placedAs status rid n terms = withTransaction st $ \db -> do
|
||||
insertInvoiceRow db ("inv-" <> rid) (at 60) status
|
||||
createPaidRegistration db (newReg rid ownerA $ order (rid <> "bakery") n) terms (paidOrder $ "inv-" <> rid) t0 `shouldReturn` Right ()
|
||||
moved rid minute change = withTransaction st $ \db -> do
|
||||
Just r <- getRegistration db (RegistrationId rid)
|
||||
updateRegistration db (change r) (at minute) `shouldReturn` Written 2
|
||||
placedAs "open" "u" 1 (OTInvoice btc price)
|
||||
mapM_ (\(rid, n) -> placedAs "paid" rid n (OTInvoice btc price)) [("src", 2), ("q", 3), ("c1", 4), ("c2", 5), ("rg", 6), ("f", 7)]
|
||||
moved "src" 1 $ \r -> r {nrPhase = PTaken}
|
||||
moved "f" 1 $ \r -> r {nrPhase = PFailed}
|
||||
moved "c1" 2 $ \r -> r {nrPhase = PCommitted}
|
||||
moved "c2" 3 $ \r -> r {nrPhase = PCommitted, nrReveal = Just reveal}
|
||||
moved "rg" 4 $ \r -> r {nrPhase = PRegistering}
|
||||
moved "q" 5 $ \r -> r {nrPhase = PQueued}
|
||||
withTransaction st $ \db -> do
|
||||
creditBalance db (RegistrationId "src") 6000 t0
|
||||
creditBalance db (RegistrationId "f") 2500 t0
|
||||
insertInvoiceRow db "inv-badge" (at 60) "open"
|
||||
placedAs "open" "h" 8 (OTBalance (RegistrationId "src") price 6000 (Just btc))
|
||||
withTransaction st $ \db -> do
|
||||
forM_ [(40, 0, 6), (41, 1, 7), (42, 1, 8), (43, 2, 9)] $ \(n, nonce, minute) ->
|
||||
insertTransaction db (TxCommit $ RegistrationId "q") NewTransaction {ntxRaw = "raw", ntxHash = TxHash $ digestFixture n, ntxNonce = nonce} (at minute)
|
||||
recordTransactionOutcome db (TxHash $ digestFixture 40) TSPending TSMined (Just 120) `shouldReturn` True
|
||||
Just mined <- insertRecordEdit db (newEdit ownerA 1 setChannel) (at 8)
|
||||
insertRecordEdit db (newEdit ownerB 3 setChannel) (at 9) >>= (`shouldSatisfy` isJust)
|
||||
settleRecordEdit db mined (ESMined Nothing) `shouldReturn` True
|
||||
counts <- withTransaction st namesCounts
|
||||
typedSamples (namesText counts noChainRead)
|
||||
`shouldBe` [ "simplex_store_names_orders{phase=\"unpaid\"} 2",
|
||||
"simplex_store_names_orders{phase=\"queued\"} 1",
|
||||
"simplex_store_names_orders{phase=\"committing\"} 0",
|
||||
"simplex_store_names_orders{phase=\"committed\"} 2",
|
||||
"simplex_store_names_orders{phase=\"registering\"} 1",
|
||||
"simplex_store_names_orders{phase=\"registered\"} 0",
|
||||
"simplex_store_names_orders{phase=\"taken\"} 1",
|
||||
"simplex_store_names_orders{phase=\"failed\"} 1",
|
||||
"simplex_store_names_queue_oldest_change_timestamp_seconds " <> seconds (at 3),
|
||||
"simplex_store_names_transactions_pending 2",
|
||||
"simplex_store_names_transactions_oldest_pending_timestamp_seconds " <> seconds (at 7),
|
||||
"simplex_store_names_invoices_open 2",
|
||||
"simplex_store_names_balances_held 1",
|
||||
"simplex_store_names_balances_held_cents 6000",
|
||||
"simplex_store_names_record_edits_pending 1",
|
||||
"simplex_store_names_record_edits_oldest_pending_timestamp_seconds " <> seconds (at 9),
|
||||
"simplex_store_names_chain_read_ok 0"
|
||||
]
|
||||
|
||||
testChainRead :: TVar ChaChaDRG -> IO ()
|
||||
testChainRead g = withServiceStore $ \st -> withFakeEthNode $ \node -> do
|
||||
c <- fakeChainClient g node
|
||||
ct <- newContracts st TLDSimplex minPriceCentsPerYear c
|
||||
let ns = NamesSource {nsChain = c, nsContracts = ct}
|
||||
registrar <- addressFromPrivateKey g fakeRegistrarKey
|
||||
answerCall node fakeSponsor (selector "registrarAllowance(address)" <> addressWord registrar) $ Returns $ uint256 (123456 * 10 ^ (16 :: Int) + 5 :: Integer)
|
||||
setFault node "eth_getBalance" $ Answers $ J.String "0x15af1d78b58c40000"
|
||||
v <- newTVarIO noChainRead
|
||||
started <- getCurrentTime
|
||||
readChain ns v
|
||||
ended <- getCurrentTime
|
||||
ChainRead {crReadAt} <- readTVarIO v
|
||||
readAt <- maybe (fail "no chain read succeeded") pure crReadAt
|
||||
(started <= readAt && readAt <= ended) `shouldBe` True
|
||||
let held general = ["simplex_store_names_terms_held{kind=\"general\"} " <> general, "simplex_store_names_terms_held{kind=\"community\"} 0"]
|
||||
chainSamples <$> readTVarIO v
|
||||
`shouldReturn` [ "simplex_store_names_registrar_balance_wei 25000000000000000000",
|
||||
"simplex_store_names_sponsor_allowance_cents 123456",
|
||||
"simplex_store_names_chain_head_block 1",
|
||||
"simplex_store_names_chain_head_timestamp_seconds " <> tshow genesisTime,
|
||||
"simplex_store_names_chain_base_fee_wei " <> tshow (fakeBaseFee :: Integer)
|
||||
]
|
||||
<> held "0"
|
||||
<> ["simplex_store_names_chain_read_timestamp_seconds " <> seconds readAt, "simplex_store_names_chain_read_ok 1"]
|
||||
termsText ct (fromJust $ termsOf $ fakeTermsLink fakeGeneralTermsText) `shouldReturn` Right "SimpleX general terms v1"
|
||||
readChain ns v
|
||||
fetched <- chainSamples <$> readTVarIO v
|
||||
filter ("simplex_store_names_terms_held" `T.isPrefixOf`) fetched `shouldBe` held "1"
|
||||
setFault node "eth_getBalance" HttpFailure
|
||||
readChain ns v
|
||||
chainSamples <$> readTVarIO v `shouldReturn` init fetched <> ["simplex_store_names_chain_read_ok 0"]
|
||||
|
||||
testScrape :: TVar ChaChaDRG -> IO ()
|
||||
testScrape g = withServiceStore $ \st -> withFakeEthNode $ \node -> do
|
||||
c <- fakeChainClient g node
|
||||
ct <- newContracts st TLDSimplex minPriceCentsPerYear c
|
||||
listedOk <- newTVarIO M.empty
|
||||
webhookAt <- newTVarIO M.empty
|
||||
v <- newTVarIO noChainRead
|
||||
census <- newTVarIO noCensusRead
|
||||
let badgesOnly = MetricsEnv {meStore = st, meProviders = ["crypto"], meListedOk = listedOk, meWebhookAt = webhookAt, meNames = Nothing}
|
||||
ns = NamesSource {nsChain = c, nsContracts = ct}
|
||||
withApp (metricsApp badgesOnly v census) $ \client -> do
|
||||
r <- webGet client "/metrics"
|
||||
(statusOf r, lookup hContentType (responseHeaders r)) `shouldBe` (200, Just "text/plain; version=0.0.4; charset=utf-8")
|
||||
let samples = typedSamples $ bodyText r
|
||||
samples `shouldContain` ["simplex_store_badges_codes_open 0"]
|
||||
samples `shouldContain` ["simplex_store_invoices_open{provider=\"crypto\"} 0"]
|
||||
filter (\l -> any (`T.isPrefixOf` l) ["simplex_store_names_", "simplex_names", "simplex_name_"]) samples `shouldBe` []
|
||||
statusOf <$> webGet client "/" `shouldReturn` 404
|
||||
statusOf <$> webRequestBody client "POST" "/metrics" [] "" `shouldReturn` 405
|
||||
(reads', _) <- pausedAt node "eth_getBalance" (runChainReads ns v)
|
||||
(`finally` cancel reads') $ withApp (metricsApp badgesOnly {meNames = Just ns} v census) $ \client -> do
|
||||
r <- timeout scrapeMicros (webGet client "/metrics") >>= maybe (fail "the scrape waited for the node") pure
|
||||
let samples = typedSamples $ bodyText r
|
||||
forM_ ["simplex_store_badges_codes_open 0", "simplex_store_names_orders{phase=\"unpaid\"} 0", "simplex_names_refresh_failures_total 0"] $ \s ->
|
||||
samples `shouldContain` [s]
|
||||
filter ("simplex_store_names_chain_" `T.isPrefixOf`) samples `shouldBe` ["simplex_store_names_chain_read_ok 0"]
|
||||
filter ("simplex_names{" `T.isPrefixOf`) samples `shouldBe` []
|
||||
|
||||
scrapeMicros :: Int
|
||||
scrapeMicros = 2000000
|
||||
|
||||
bodyText :: Response LB.ByteString -> Text
|
||||
bodyText = decodeUtf8 . LB.toStrict . responseBody
|
||||
|
||||
typedSamples :: Text -> [Text]
|
||||
typedSamples text = [l | l <- ls, not (T.null l || "#" `T.isPrefixOf` l), described (T.takeWhile (`notElem` [' ', '{']) l)]
|
||||
where
|
||||
ls = T.lines text
|
||||
described name = any (("# TYPE " <> name <> " ") `T.isPrefixOf`) ls && any (("# HELP " <> name <> " ") `T.isPrefixOf`) ls
|
||||
|
||||
chainSamples :: ChainRead -> [Text]
|
||||
chainSamples = filter (\l -> any (`T.isPrefixOf` l) ["simplex_store_names_registrar_", "simplex_store_names_sponsor_", "simplex_store_names_chain_", "simplex_store_names_terms_"]) . typedSamples . namesText noCounts
|
||||
where
|
||||
noCounts = NamesCounts {ncPhases = [], ncQueueOldestChange = Nothing, ncPendingTransactions = 0, ncOldestPending = Nothing, ncOpenInvoices = 0, ncHeldBalances = 0, ncHeldCents = 0, ncPendingEdits = 0, ncOldestEdit = Nothing}
|
||||
|
||||
seconds :: UTCTime -> Text
|
||||
seconds t = tshow (floor $ utcTimeToPOSIXSeconds t :: Integer)
|
||||
@@ -104,7 +104,7 @@ testCodeStates = do
|
||||
-- The route as namesRoutes registers it, since a NamesEnv needs a running chat controller.
|
||||
withCodeCheck :: (DBStore -> Client -> IO a) -> IO a
|
||||
withCodeCheck action = withServiceStore $ \st -> do
|
||||
http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing}
|
||||
http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing}
|
||||
let routes = \case
|
||||
["api", "code", "check"] -> Just Route {rMethod = "POST", rLimit = codeLimit, rHandler = codeCheckHandler st}
|
||||
_ -> Nothing
|
||||
|
||||
@@ -102,7 +102,7 @@ withSearch :: TVar ChaChaDRG -> (NamesConfig -> NamesConfig) -> (Contracts -> Co
|
||||
withSearch g adjustConfig adjust action = withServiceStore $ \store -> withContracts g $ \node ct -> do
|
||||
fetched <- newTVarIO []
|
||||
searchesAtFetch <- newTVarIO []
|
||||
http@HttpEnv {heSearches} <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing}
|
||||
http@HttpEnv {heSearches} <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing}
|
||||
let fetch sl = atomically (readTVar heSearches >>= modifyTVar' searchesAtFetch . (:)) >> fakeFetch fetched sl
|
||||
ps <- newProfiles defaultProfileLimits {fetchTimeout = fetchMicros} monotonicSeconds fetch stubResolver
|
||||
let cfg = adjustConfig $ testNamesConfig node
|
||||
|
||||
@@ -119,6 +119,12 @@ namesSignatureTests = describe "StoreService names: signatures" $ do
|
||||
decodeString (uint256 (4096 :: Int) <> B.drop 32 retString) `shouldBe` Left AEOffset
|
||||
decodeString (B.take 80 retString) `shouldBe` Left AEShort
|
||||
decodeString (abiTuple [Dynamic $ abiBytes "\xff"]) `shouldBe` Left AENotUtf8
|
||||
it "pairs the bytes32[] and string[] of a subname registrar's children, as viem encodes them, and refuses arrays of different lengths" $ do
|
||||
decodeLabels (hx "000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000000000000000295b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea69513447e2d376aacd434727887590dd448cda8f2d30c4ace903d31fe209f8ad8000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000473686f70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005636166c3a9000000000000000000000000000000000000000000000000000000")
|
||||
`shouldBe` Right [(hx "95b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea6", "shop"), (hx "9513447e2d376aacd434727887590dd448cda8f2d30c4ace903d31fe209f8ad8", "café")]
|
||||
decodeLabels (abiTuple [Dynamic (uint256 (0 :: Int)), Dynamic (uint256 (0 :: Int))]) `shouldBe` Right []
|
||||
decodeLabels (hx "00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000195b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea6000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000473686f70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005636166c3a9000000000000000000000000000000000000000000000000000000")
|
||||
`shouldBe` Left AEOffset
|
||||
|
||||
testMessage :: ByteString -> ByteString -> Text -> Text -> IO ()
|
||||
testMessage msg expected digest signature = do
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
{-# LANGUAGE NamedFieldPuns #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
|
||||
module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, invoiceStatus, atNonce, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where
|
||||
module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, insertProviderInvoice, invoiceStatus, atNonce, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where
|
||||
|
||||
import Bots.StoreService.FakeEthNode (plainTerms)
|
||||
import Bots.StoreService.WebTests (columnsOf, digestFixture, runMigrations, withServiceStore)
|
||||
@@ -132,11 +132,14 @@ paidOrder :: Text -> PaidOrder
|
||||
paidOrder invId = PaidOrder {poMethod = btc, poPrice = price, poInvoiceId = InvoiceId invId, poProviderRef = "ref-" <> invId}
|
||||
|
||||
insertInvoiceRow :: DB.Connection -> Text -> UTCTime -> Text -> IO ()
|
||||
insertInvoiceRow db invId expiresAt status =
|
||||
insertInvoiceRow db = insertProviderInvoice db "crypto"
|
||||
|
||||
insertProviderInvoice :: DB.Connection -> Text -> Text -> UTCTime -> Text -> IO ()
|
||||
insertProviderInvoice db provider invId expiresAt status =
|
||||
DB.execute
|
||||
db
|
||||
"INSERT INTO sx_badge_service_invoices (invoice_id, provider, price, amount, currency, payment_address, payment_crypto_currency, payment_crypto_amount, expires_at, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)"
|
||||
((invId, "crypto" :: Text, price, price, "usd" :: Text, "bc1qexampleaddress" :: Text, "btc" :: Text, "0.00312" :: Text) :. (expiresAt, status, t0, t0))
|
||||
((invId, provider, price, price, "usd" :: Text, "bc1qexampleaddress" :: Text, "btc" :: Text, "0.00312" :: Text) :. (expiresAt, status, t0, t0))
|
||||
|
||||
-- | A paid order whose invoice, named after the registration, expires at the given time.
|
||||
placePaid :: DBStore -> Text -> Address -> OrderFields -> UTCTime -> IO (Either RegistrationRefused ())
|
||||
|
||||
@@ -118,7 +118,7 @@ probeLimit = Limit {lmName = "probe", lmPerMinute = 1}
|
||||
|
||||
withProbeRouter :: (Client -> IO a) -> IO a
|
||||
withProbeRouter action = do
|
||||
http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing}
|
||||
http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing}
|
||||
let routes = \case
|
||||
["api", "probe", _] -> Just Route {rMethod = "GET", rLimit = probeLimit, rHandler = \_ respond -> respond (jsonResponse status200 (J.object []))}
|
||||
_ -> Nothing
|
||||
|
||||
@@ -304,7 +304,7 @@ badgeWebTests = do
|
||||
it "refuses a settlement instant in the future, as a millisecond clock gives" testFutureSettledInstantIsRefused
|
||||
describe "store service poller" $ do
|
||||
it "settles a payment with no webhook delivered anywhere" testSettlesWithNoWebhookAtAll
|
||||
it "reads only the invoices it is waiting on, and lists on the stray cadence" testPassReadsWhatItAwaits
|
||||
it "reads only the invoices it is waiting on, and lists on the stray cadence, recording when the provider's list succeeded" testPassReadsWhatItAwaits
|
||||
it "asks the provider nothing when it is waiting on nothing" testIdlePassAsksNothing
|
||||
it "lists instead of reading once too many invoices are open" testManyOpenInvoicesList
|
||||
it "sweeps nothing when no provider can account for the rows" testNoProviderAccountsForNothing
|
||||
@@ -314,7 +314,7 @@ badgeWebTests = do
|
||||
it "passes over a listed invoice this service does not hold" testForeignRefIsPassedOver
|
||||
it "changes nothing when the provider fails, and settles on the next pass" testProviderFailureLosesNothing
|
||||
it "asks a stalled BTCPay once in a pass and again in the next, settling a card invoice listed after its invoices" testStalledBTCPayHoldsNoCardSettlement
|
||||
it "reads the overdue invoices on their own after a list that timed out" testListTimeoutKeepsTheReads
|
||||
it "reads the overdue invoices on their own after a list that timed out, recording no successful list" testListTimeoutKeepsTheReads
|
||||
it "reads every invoice of a provider whose reads fail without a timeout" testFailedReadKeepsTheOtherReads
|
||||
it "reads the cadence fresh from the waiters each pass" testCadenceFollowsTheWaiters
|
||||
it "cuts an idle sleep short when a browser arrives during it" testWaiterCutsTheIdleSleepShort
|
||||
@@ -350,7 +350,7 @@ badgeWebTests = do
|
||||
it "pages past a hundred open invoices, a request per hundred" testListPaginates
|
||||
it "stops paging at its ceiling rather than looping on a server that ignores take" testPagingStopsAtTheCeiling
|
||||
describe "store service webhook" $ do
|
||||
it "verifies a real BTCPay signature over the bytes as received, and queues" testWebhookVerifiesARealSignature
|
||||
it "verifies a real BTCPay signature over the bytes as received, and queues, recording when only a verified delivery came" testWebhookVerifiesARealSignature
|
||||
it "hands the adapter the raw body, byte for byte" testWebhookPassesTheRawBytes
|
||||
it "queues a read and answers 200 empty, reaching no provider" testWebhookQueuesARead
|
||||
it "settles by the poller's own path, so there is one settlement lane" testWebhookSettlesByThePollerPath
|
||||
@@ -1257,7 +1257,7 @@ prepareStaticDir root = do
|
||||
testServiceConfig :: FilePath -> Bool -> ServiceConfig
|
||||
testServiceConfig staticDir trustForwarded =
|
||||
ServiceConfig
|
||||
{ listener = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = True, lBadgesExportDir = Nothing, lTrustForwardedFor = trustForwarded, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing},
|
||||
{ listener = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = True, lBadgesExportDir = Nothing, lTrustForwardedFor = trustForwarded, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing},
|
||||
btcpay = Nothing,
|
||||
stripe = Nothing,
|
||||
poll = PollConfig {pWaitingSeconds = 3, pIdleSeconds = 60},
|
||||
@@ -1294,7 +1294,7 @@ testExportWebapp = do
|
||||
withTempDirectory "tests/tmp" "badge-export" $ \root -> do
|
||||
staticDir <- prepareStaticDir root
|
||||
let outDir = root </> "out"
|
||||
lc = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = False, lBadgesExportDir = Just outDir, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing}
|
||||
lc = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = False, lBadgesExportDir = Just outDir, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing}
|
||||
stripeCfg = Just StripeConfig {sSecretKey = "rk_test_x", sPublishableKey = "pk_test_injected", sWebhookSecret = "whsec_x", sSessionMinutes = 60, sHost = "https://api.stripe.com", sTimeoutMicros = providerTimeoutMicros, sCheckoutTimeoutMicros = providerCheckoutTimeoutMicros}
|
||||
exportWebapp (testServiceConfig staticDir False) {listener = lc, stripe = stripeCfg}
|
||||
idx <- LB.readFile (outDir </> "index.html")
|
||||
@@ -3030,9 +3030,13 @@ testPassReadsWhatItAwaits = bounded "reads what it awaits" $ withStubPoller race
|
||||
b <- seedOpenRef (weStore env) 2 "p-2" someExpiry
|
||||
c <- seedOpenRef (weStore env) 3 "p-3" someExpiry
|
||||
setSignals ref [("p-1", settledSignal), ("p-2", settledSignal), ("p-3", settledSignal)]
|
||||
passStarted <- getCurrentTime
|
||||
runOnePass poller
|
||||
stubCalls ref `shouldReturn` [StubRead "p-1", StubRead "p-2", StubRead "p-3", StubListOpen]
|
||||
mapM (invoiceStatus (weStore env)) [a, b, c] `shouldReturn` [ISPaid, ISPaid, ISPaid]
|
||||
listed <- readTVarIO (peListedOk poller)
|
||||
Map.keys listed `shouldBe` ["crypto"]
|
||||
all (>= passStarted) listed `shouldBe` True
|
||||
|
||||
testIdlePassAsksNothing :: IO ()
|
||||
testIdlePassAsksNothing = bounded "idle pass" $ withStubPoller raceHold $ \ref poller env _ -> do
|
||||
@@ -3175,6 +3179,7 @@ testListTimeoutKeepsTheReads = bounded "list timeout" $ withStubPoller raceHold
|
||||
runOnePass poller
|
||||
stubCalls ref `shouldReturn` [StubListOpen, StubRead "p-overdue"]
|
||||
invoiceStatus (weStore env) overdue `shouldReturn` ISExpired
|
||||
readTVarIO (peListedOk poller) `shouldReturn` Map.empty
|
||||
|
||||
testFailedReadKeepsTheOtherReads :: IO ()
|
||||
testFailedReadKeepsTheOtherReads = bounded "failed read" $ withStubPoller raceHold $ \ref poller env _ -> do
|
||||
@@ -3779,14 +3784,18 @@ testWebhookVerifiesARealSignature = bounded "real signature" $ withFakePoller $
|
||||
Just row <- getInvoice (weStore env) iid
|
||||
let secret = bWebhookSecret (fbConfig fake)
|
||||
body = webhookEvent "InvoiceSettled" (irProviderRef row)
|
||||
unsigned <- postWebhook client [] body
|
||||
statusOf unsigned `shouldBe` 400
|
||||
readTVarIO (weWebhookAt env) `shouldReturn` Map.empty
|
||||
delivered <- postWebhook client (webhookSigHeader secret body) body
|
||||
statusOf delivered `shouldBe` 200
|
||||
responseBody delivered `shouldBe` ""
|
||||
verifiedAt <- readTVarIO (weWebhookAt env)
|
||||
Map.keys verifiedAt `shouldBe` ["crypto"]
|
||||
tampered <- postWebhook client (webhookSigHeader secret body) (body <> " ")
|
||||
statusOf tampered `shouldBe` 400
|
||||
responseBody tampered `shouldBe` ""
|
||||
unsigned <- postWebhook client [] body
|
||||
statusOf unsigned `shouldBe` 400
|
||||
readTVarIO (weWebhookAt env) `shouldReturn` verifiedAt
|
||||
invoiceStatus (weStore env) iid `shouldReturn` ISOpen
|
||||
setInvoiceState fake (irProviderRef row) ["status" .= ("Settled" :: Text), "paymentMethodPaid" .= ("0.32095000" :: Text)]
|
||||
drainHints poller
|
||||
|
||||
@@ -9,6 +9,7 @@ import Bots.StoreService.CatalogTests
|
||||
import Bots.StoreService.ConfigTests
|
||||
import Bots.StoreService.GroupIntegrationTests
|
||||
import Bots.StoreService.GroupTests
|
||||
import Bots.StoreService.MetricsTests
|
||||
import Bots.StoreService.NameRegistrationTests
|
||||
import Bots.StoreService.NamesBalancesTests
|
||||
import Bots.StoreService.NamesLateCreditTests
|
||||
@@ -110,6 +111,7 @@ main = do
|
||||
badgeStripeTests
|
||||
badgeNameRegistrationTests
|
||||
badgeStoreVerifierTests
|
||||
storeMetricsTests
|
||||
namesConfigTests
|
||||
namesBalancesTests
|
||||
namesLateCreditTests
|
||||
|
||||
Reference in new issue
Block a user