mirror of
https://github.com/simplex-chat/simplexmq.git
synced 2026-08-29 03:28:42 +00:00
router diagrams
This commit is contained in:
Evgeny @ SimpleX Chat
+5
-32
@@ -2,11 +2,11 @@
|
||||
|
||||
SimpleX routers are the network infrastructure of the [SimpleX Network](../protocol/overview-tjr.md). They accept, buffer, and deliver data packets between endpoints. Each router operates independently and can be run by any party on standard computing hardware.
|
||||
|
||||
This document covers deployment and advanced configuration. For an overview of the router architecture and trust model, see the [SimpleX Network overview](../protocol/overview-tjr.md).
|
||||
This document covers deployment and advanced configuration. For an overview of the router architecture and trust model, see the [SimpleX Network overview](../protocol/overview-tjr.md). For internal architecture diagrams (thread topology, command processing flows), see [`spec/routers.md`](../spec/routers.md).
|
||||
|
||||
## SMP Router
|
||||
|
||||
The SMP router provides messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes each). It implements the [SimpleX Messaging Protocol](../protocol/simplex-messaging.md). **Module spec**: [`spec/modules/Simplex/Messaging/Server.md`](../spec/modules/Simplex/Messaging/Server.md).
|
||||
The SMP router provides messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes each). It implements the [SimpleX Messaging Protocol](../protocol/simplex-messaging.md). For architecture and module specs, see [SMP Router](../spec/routers.md#smp-router).
|
||||
|
||||
### Advanced configuration
|
||||
|
||||
@@ -35,13 +35,13 @@ echo 'PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zprofile
|
||||
|
||||
## XFTP Router
|
||||
|
||||
The XFTP router accepts and delivers data packets over HTTP/2 — individually addressed blocks in fixed sizes (64KB, 256KB, 1MB, 4MB). It implements the [XFTP protocol](../protocol/xftp.md). Data packets are used for larger payload delivery (files, media) where SMP queue packet sizes would be inefficient. The use of HTTP/2 simplifies browser integration. **Module spec**: [`spec/modules/Simplex/FileTransfer/Server.md`](../spec/modules/Simplex/FileTransfer/Server.md).
|
||||
The XFTP router accepts and delivers data packets over HTTP/2 — individually addressed blocks in fixed sizes (64KB, 256KB, 1MB, 4MB). It implements the [XFTP protocol](../protocol/xftp.md). Data packets are used for larger payload delivery (files, media) where SMP queue packet sizes would be inefficient. The use of HTTP/2 simplifies browser integration. For architecture and module specs, see [XFTP Router](../spec/routers.md#xftp-router).
|
||||
|
||||
Initialize with `xftp-server init` and configure storage quota in `xftp-server.ini`.
|
||||
|
||||
## NTF Router
|
||||
|
||||
The NTF router bridges SimpleX Network to platform push notification services (APNS). It implements the [Push Notifications protocol](../protocol/push-notifications.md). Mobile clients register push tokens with the NTF router, which subscribes to their SMP queues and sends push notifications when messages arrive. The push notification contains only a notification ID, not message content. **Module spec**: [`spec/modules/Simplex/Messaging/Notifications/Server.md`](../spec/modules/Simplex/Messaging/Notifications/Server.md).
|
||||
The NTF router bridges SimpleX Network to platform push notification services (APNS). It implements the [Push Notifications protocol](../protocol/push-notifications.md). Mobile clients register push tokens with the NTF router, which subscribes to their SMP queues and sends push notifications when messages arrive. The push notification contains only a notification ID, not message content. For architecture and module specs, see [NTF Router](../spec/routers.md#ntf-router).
|
||||
|
||||
Initialize with `ntf-server init` and configure APNS credentials in `ntf-server.ini`.
|
||||
|
||||
@@ -184,7 +184,7 @@ smp-server init [-l] -n <fqdn>
|
||||
|
||||
## Monitoring
|
||||
|
||||
SMP and XFTP routers expose Prometheus metrics via a control port. The control port also supports commands for runtime inspection (queue counts, client counts, statistics). See [SMP Server Prometheus](../spec/modules/Simplex/Messaging/Server/Prometheus.md), [SMP Server Control](../spec/modules/Simplex/Messaging/Server/Control.md), and [NTF Server Control](../spec/modules/Simplex/Messaging/Notifications/Server/Control.md) module specs for available metrics and control commands.
|
||||
SMP and XFTP routers expose Prometheus metrics via a control port. The control port also supports commands for runtime inspection (queue counts, client counts, statistics). See module specs linked from each router section in [`spec/routers.md`](../spec/routers.md) (Control, Prometheus, Stats).
|
||||
|
||||
## Protocol references
|
||||
|
||||
@@ -192,30 +192,3 @@ SMP and XFTP routers expose Prometheus metrics via a control port. The control p
|
||||
- [XFTP Protocol](../protocol/xftp.md) — data packet protocol
|
||||
- [Push Notifications Protocol](../protocol/push-notifications.md) — NTF protocol
|
||||
- [SimpleX Network overview](../protocol/overview-tjr.md) — architecture and trust model
|
||||
|
||||
## Module specs
|
||||
|
||||
### SMP Router
|
||||
- [Server](../spec/modules/Simplex/Messaging/Server.md) — main server module, client handling, message routing
|
||||
- [Server Main](../spec/modules/Simplex/Messaging/Server/Main.md) — server startup, initialization
|
||||
- [QueueStore](../spec/modules/Simplex/Messaging/Server/QueueStore.md) — queue persistence abstraction
|
||||
- [QueueStore Postgres](../spec/modules/Simplex/Messaging/Server/QueueStore/Postgres.md) — PostgreSQL queue store
|
||||
- [MsgStore](../spec/modules/Simplex/Messaging/Server/MsgStore.md) — message storage abstraction
|
||||
- [StoreLog](../spec/modules/Simplex/Messaging/Server/StoreLog.md) — append-only store log for queue persistence
|
||||
- [Server Control](../spec/modules/Simplex/Messaging/Server/Control.md) — control port commands
|
||||
- [Server Prometheus](../spec/modules/Simplex/Messaging/Server/Prometheus.md) — metrics export
|
||||
- [Server Stats](../spec/modules/Simplex/Messaging/Server/Stats.md) — statistics collection
|
||||
|
||||
### XFTP Router
|
||||
- [Server](../spec/modules/Simplex/FileTransfer/Server.md) — main server module, data packet handling
|
||||
- [Server Main](../spec/modules/Simplex/FileTransfer/Server/Main.md) — server startup
|
||||
- [Server Store](../spec/modules/Simplex/FileTransfer/Server/Store.md) — data packet storage
|
||||
- [Server StoreLog](../spec/modules/Simplex/FileTransfer/Server/StoreLog.md) — store log for packet persistence
|
||||
- [Server Stats](../spec/modules/Simplex/FileTransfer/Server/Stats.md) — statistics
|
||||
|
||||
### NTF Router
|
||||
- [Server](../spec/modules/Simplex/Messaging/Notifications/Server.md) — main server module
|
||||
- [Server Main](../spec/modules/Simplex/Messaging/Notifications/Server/Main.md) — server startup
|
||||
- [Server Store Postgres](../spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) — PostgreSQL store for tokens and subscriptions
|
||||
- [APNS Push](../spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md) — Apple push notification delivery
|
||||
- [Server Control](../spec/modules/Simplex/Messaging/Notifications/Server/Control.md) — control port commands
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 830 610" font-family="monospace" font-size="12">
|
||||
<defs>
|
||||
<marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
|
||||
</marker>
|
||||
<marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- Title -->
|
||||
<text x="415" y="22" text-anchor="middle" font-size="15" font-weight="bold">NTF Router -- Component Topology</text>
|
||||
|
||||
<!-- ===== PER-CLIENT GROUP (left) ===== -->
|
||||
<rect x="25" y="40" width="300" height="170" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="55" fill="#888" font-size="10">per client (raceAny_)</text>
|
||||
|
||||
<!-- network in -->
|
||||
<text x="13" y="78" font-size="9" fill="#999">net</text>
|
||||
<line x1="12" y1="82" x2="40" y2="82"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
|
||||
<!-- receive -->
|
||||
<rect x="42" y="66" width="78" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="81" y="88" text-anchor="middle">receive</text>
|
||||
|
||||
<!-- rcvQ -->
|
||||
<line x1="120" y1="83" x2="158" y2="83"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="139" y="77" text-anchor="middle" font-size="10" fill="#555">rcvQ</text>
|
||||
|
||||
<!-- client -->
|
||||
<rect x="160" y="66" width="78" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="199" y="88" text-anchor="middle">client</text>
|
||||
|
||||
<!-- sndQ -->
|
||||
<line x1="238" y1="83" x2="268" y2="83"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="253" y="77" text-anchor="middle" font-size="10" fill="#555">sndQ</text>
|
||||
|
||||
<!-- send -->
|
||||
<rect x="270" y="66" width="48" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="294" y="88" text-anchor="middle">send</text>
|
||||
|
||||
<!-- network out -->
|
||||
<text x="326" y="88" font-size="9" fill="#999">net</text>
|
||||
|
||||
<!-- commands label -->
|
||||
<text x="52" y="120" font-size="10" fill="#666">TNEW, TVFY, TRPL, TDEL</text>
|
||||
<text x="52" y="134" font-size="10" fill="#666">SNEW, SCHK, SDEL</text>
|
||||
|
||||
<!-- client -> store (down) -->
|
||||
<line x1="199" y1="100" x2="199" y2="280"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="211" y="230" font-size="10" fill="#555">store</text>
|
||||
|
||||
<!-- ===== SMP CLIENT AGENT GROUP (right) ===== -->
|
||||
<rect x="370" y="40" width="435" height="220" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="380" y="55" fill="#888" font-size="10">SMP Client Agent (connects to SMP routers)</text>
|
||||
|
||||
<!-- SMP routers external label -->
|
||||
<text x="620" y="38" font-size="9" fill="#999">SMP routers</text>
|
||||
<line x1="620" y1="40" x2="620" y2="62"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
|
||||
<!-- SMPClientAgent -->
|
||||
<rect x="530" y="64" width="220" height="34" rx="4"
|
||||
fill="#fce8e6" stroke="#ea4335" />
|
||||
<text x="640" y="86" text-anchor="middle">SMPClientAgent</text>
|
||||
|
||||
<!-- msgQ arrow: SMPClientAgent -> receiveSMP -->
|
||||
<line x1="580" y1="98" x2="510" y2="128"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="530" y="112" font-size="10" fill="#555">msgQ</text>
|
||||
|
||||
<!-- agentQ arrow: SMPClientAgent -> receiveAgent -->
|
||||
<line x1="690" y1="98" x2="690" y2="128"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="702" y="117" font-size="10" fill="#555">agentQ</text>
|
||||
|
||||
<!-- ntfSubscriber/receiveSMP -->
|
||||
<rect x="390" y="130" width="195" height="34" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="487" y="152" text-anchor="middle" font-size="11">ntfSubscriber/receiveSMP</text>
|
||||
|
||||
<!-- ntfSubscriber/receiveAgent -->
|
||||
<rect x="605" y="130" width="185" height="34" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="697" y="152" text-anchor="middle" font-size="11">receiveAgent</text>
|
||||
|
||||
<!-- race_ label between them -->
|
||||
<text x="597" y="144" font-size="9" fill="#888">race_</text>
|
||||
|
||||
<!-- receiveSMP -> pushQ (down) -->
|
||||
<line x1="487" y1="164" x2="487" y2="195"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="500" y="183" font-size="10" fill="#555">pushQ</text>
|
||||
|
||||
<!-- receiveAgent -> store (down) -->
|
||||
<line x1="697" y1="164" x2="697" y2="195"
|
||||
stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
|
||||
<text x="709" y="183" font-size="10" fill="#555">store</text>
|
||||
|
||||
<!-- per-SMP-server subscriber -->
|
||||
<rect x="390" y="200" width="200" height="44" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="490" y="219" text-anchor="middle" font-size="11">runSMPSubscriber</text>
|
||||
<text x="490" y="236" text-anchor="middle" font-size="9" fill="#666">(one per SMP router)</text>
|
||||
|
||||
<!-- subscriberSubQ label -->
|
||||
<text x="408" y="196" font-size="9" fill="#555">subscriberSubQ</text>
|
||||
|
||||
<!-- ===== CENTRAL STORE ===== -->
|
||||
<rect x="100" y="280" width="600" height="50" rx="4"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="400" y="301" text-anchor="middle" font-weight="bold">tokens / subscriptions / tokenLastNtfs</text>
|
||||
<text x="400" y="319" text-anchor="middle" font-size="10">(in-memory TMap + PostgreSQL)</text>
|
||||
|
||||
<!-- ===== PUSH DELIVERY ===== -->
|
||||
<rect x="25" y="355" width="780" height="90" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="370" fill="#888" font-size="10">push delivery pipeline</text>
|
||||
|
||||
<!-- pushQ hub: store -> ntfPush -->
|
||||
<line x1="400" y1="330" x2="400" y2="388"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="412" y="365" font-size="10" fill="#555">pushQ</text>
|
||||
|
||||
<!-- ntfPush -->
|
||||
<rect x="350" y="390" width="110" height="34" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="405" y="412" text-anchor="middle">ntfPush</text>
|
||||
|
||||
<!-- APNS provider -->
|
||||
<line x1="460" y1="407" x2="528" y2="407"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<rect x="530" y="390" width="135" height="34" rx="4"
|
||||
fill="#fce8e6" stroke="#ea4335" />
|
||||
<text x="597" y="412" text-anchor="middle">APNS provider</text>
|
||||
|
||||
<!-- periodicNtfsThread -->
|
||||
<rect x="45" y="390" width="175" height="34" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="132" y="412" text-anchor="middle" font-size="11">periodicNtfsThread</text>
|
||||
|
||||
<!-- periodic -> pushQ -->
|
||||
<line x1="220" y1="407" x2="348" y2="407"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="284" y="401" font-size="10" fill="#555">pushQ</text>
|
||||
|
||||
<!-- periodic reads store -->
|
||||
<line x1="132" y1="390" x2="132" y2="330"
|
||||
stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
|
||||
<text x="117" y="365" font-size="9" fill="#555">reads</text>
|
||||
|
||||
<!-- ===== OPTIONAL ===== -->
|
||||
<rect x="25" y="462" width="780" height="45" rx="6"
|
||||
fill="none" stroke="#bbb" stroke-dasharray="4,3" />
|
||||
<text x="35" y="477" fill="#bbb" font-size="10">optional</text>
|
||||
|
||||
<rect x="45" y="484" width="128" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="109" y="497" text-anchor="middle" font-size="9">logServerStats</text>
|
||||
|
||||
<rect x="190" y="484" width="108" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="244" y="497" text-anchor="middle" font-size="9">prometheus</text>
|
||||
|
||||
<rect x="315" y="484" width="108" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="369" y="497" text-anchor="middle" font-size="9">controlPort</text>
|
||||
|
||||
<rect x="440" y="484" width="108" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="494" y="497" text-anchor="middle" font-size="9">resubscribe</text>
|
||||
|
||||
<!-- ===== LEGEND ===== -->
|
||||
<rect x="25" y="525" width="780" height="55" rx="4"
|
||||
fill="none" stroke="#ddd" />
|
||||
|
||||
<rect x="40" y="540" width="14" height="11" rx="2"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="60" y="550" font-size="10">per-client thread</text>
|
||||
|
||||
<rect x="195" y="540" width="14" height="11" rx="2"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="215" y="550" font-size="10">singleton thread</text>
|
||||
|
||||
<rect x="350" y="540" width="14" height="11" rx="2"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="370" y="550" font-size="10">storage</text>
|
||||
|
||||
<rect x="445" y="540" width="14" height="11" rx="2"
|
||||
fill="#fce8e6" stroke="#ea4335" />
|
||||
<text x="465" y="550" font-size="10">external connection</text>
|
||||
|
||||
<text x="40" y="572" font-size="10" fill="#666">
|
||||
Solid arrows: TBQueue connections. Dashed: store access.
|
||||
</text>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 8.3 KiB |
@@ -0,0 +1,191 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 590" font-family="monospace" font-size="12">
|
||||
<defs>
|
||||
<marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
|
||||
</marker>
|
||||
<marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- Title -->
|
||||
<text x="430" y="22" text-anchor="middle" font-size="15" font-weight="bold">SMP Router -- Component Topology</text>
|
||||
|
||||
<!-- ===== PER-CLIENT GROUP ===== -->
|
||||
<rect x="25" y="38" width="570" height="210" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="53" fill="#888" font-size="10">per client connection (raceAny_ -- any thread exit tears down connection)</text>
|
||||
|
||||
<!-- receive -->
|
||||
<rect x="45" y="68" width="90" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="90" y="90" text-anchor="middle">receive</text>
|
||||
|
||||
<!-- rcvQ arrow: receive -> client -->
|
||||
<line x1="135" y1="85" x2="198" y2="85"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="166" y="79" text-anchor="middle" font-size="10" fill="#555">rcvQ</text>
|
||||
|
||||
<!-- client -->
|
||||
<rect x="200" y="68" width="100" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="250" y="90" text-anchor="middle">client</text>
|
||||
|
||||
<!-- sndQ arrow: client -> send -->
|
||||
<line x1="300" y1="78" x2="388" y2="78"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="344" y="72" text-anchor="middle" font-size="10" fill="#555">sndQ</text>
|
||||
|
||||
<!-- send -->
|
||||
<rect x="390" y="62" width="70" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="425" y="84" text-anchor="middle">send</text>
|
||||
|
||||
<!-- msgQ arrow: client -> sendMsg -->
|
||||
<line x1="300" y1="93" x2="388" y2="113"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
<text x="336" y="112" text-anchor="middle" font-size="10" fill="#555">msgQ</text>
|
||||
|
||||
<!-- sendMsg -->
|
||||
<rect x="390" y="102" width="90" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="435" y="124" text-anchor="middle">sendMsg</text>
|
||||
|
||||
<!-- network in label + arrow -->
|
||||
<text x="13" y="81" font-size="9" fill="#999">net</text>
|
||||
<line x1="12" y1="85" x2="45" y2="85"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
|
||||
<!-- network out arrows -->
|
||||
<line x1="460" y1="79" x2="498" y2="79"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
<line x1="480" y1="119" x2="498" y2="119"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
<text x="504" y="100" font-size="9" fill="#999">net</text>
|
||||
|
||||
<!-- client -> QueueStore (down-left) -->
|
||||
<line x1="228" y1="102" x2="130" y2="150"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
|
||||
<!-- client -> MsgStore (down-right) -->
|
||||
<line x1="278" y1="102" x2="395" y2="150"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
|
||||
<!-- QueueStore -->
|
||||
<rect x="45" y="150" width="155" height="48" rx="4"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="122" y="171" text-anchor="middle" font-weight="bold">QueueStore</text>
|
||||
<text x="122" y="189" text-anchor="middle" font-size="10">(STM or Postgres)</text>
|
||||
|
||||
<!-- MsgStore -->
|
||||
<rect x="320" y="150" width="155" height="48" rx="4"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="397" y="171" text-anchor="middle" font-weight="bold">MsgStore</text>
|
||||
<text x="397" y="189" text-anchor="middle" font-size="10">(STM or Postgres)</text>
|
||||
|
||||
<!-- StoreLog under QueueStore only (queues are logged, msgs stored directly) -->
|
||||
<rect x="45" y="208" width="155" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#ccc" />
|
||||
<text x="122" y="221" text-anchor="middle" font-size="9" fill="#888">StoreLog (optional)</text>
|
||||
|
||||
<!-- subQ arrow: from client bottom-center, through gap between stores, to serverThread -->
|
||||
<!-- Gap between stores: x=200..320 -->
|
||||
<line x1="250" y1="102" x2="250" y2="296"
|
||||
stroke="#4285f4" stroke-width="1.5" stroke-dasharray="5,3"
|
||||
marker-end="url(#arr)" />
|
||||
<text x="262" y="238" font-size="10" fill="#4285f4">subQ</text>
|
||||
|
||||
<!-- ===== SINGLETON GROUP ===== -->
|
||||
<rect x="25" y="258" width="810" height="195" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="273" fill="#888" font-size="10">singleton threads (one instance each, all in raceAny_)</text>
|
||||
|
||||
<!-- serverThread (SMP subs) -->
|
||||
<rect x="45" y="288" width="175" height="46" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="132" y="308" text-anchor="middle">serverThread</text>
|
||||
<text x="132" y="324" text-anchor="middle" font-size="10">(SMP subscriptions)</text>
|
||||
|
||||
<!-- serverThread (NTF subs) -->
|
||||
<rect x="240" y="288" width="175" height="46" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="327" y="308" text-anchor="middle">serverThread</text>
|
||||
<text x="327" y="324" text-anchor="middle" font-size="10">(NTF subscriptions)</text>
|
||||
|
||||
<!-- pendingEvents: serverThread -> sendPendingEvts -->
|
||||
<line x1="195" y1="334" x2="230" y2="355"
|
||||
stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
|
||||
<text x="176" y="352" font-size="9" fill="#555">pendingEvents</text>
|
||||
|
||||
<!-- deliverNtfs -->
|
||||
<rect x="45" y="352" width="128" height="30" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="109" y="372" text-anchor="middle" font-size="11">deliverNtfs</text>
|
||||
|
||||
<!-- sendPendingEvts -->
|
||||
<rect x="195" y="352" width="148" height="30" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="269" y="372" text-anchor="middle" font-size="11">sendPendingEvts</text>
|
||||
|
||||
<!-- expireMessages -->
|
||||
<rect x="365" y="352" width="138" height="30" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="434" y="372" text-anchor="middle" font-size="11">expireMessages</text>
|
||||
|
||||
<!-- expireNtfs -->
|
||||
<rect x="45" y="396" width="115" height="30" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="102" y="416" text-anchor="middle" font-size="11">expireNtfs</text>
|
||||
|
||||
<!-- proxyAgent -->
|
||||
<rect x="180" y="396" width="118" height="30" rx="4"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="239" y="416" text-anchor="middle" font-size="11">proxyAgent</text>
|
||||
|
||||
<!-- ===== OPTIONAL (nested inside singleton, right side) ===== -->
|
||||
<rect x="610" y="278" width="215" height="130" rx="6"
|
||||
fill="none" stroke="#bbb" stroke-dasharray="4,3" />
|
||||
<text x="620" y="293" fill="#bbb" font-size="10">optional</text>
|
||||
|
||||
<!-- logServerStats -->
|
||||
<rect x="625" y="302" width="140" height="24" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="695" y="319" text-anchor="middle" font-size="10">logServerStats</text>
|
||||
|
||||
<!-- prometheus -->
|
||||
<rect x="625" y="336" width="140" height="24" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="695" y="353" text-anchor="middle" font-size="10">prometheus</text>
|
||||
|
||||
<!-- controlPort -->
|
||||
<rect x="625" y="370" width="140" height="24" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="695" y="387" text-anchor="middle" font-size="10">controlPort</text>
|
||||
|
||||
<!-- ===== LEGEND ===== -->
|
||||
<rect x="25" y="472" width="810" height="60" rx="4"
|
||||
fill="none" stroke="#ddd" />
|
||||
|
||||
<rect x="40" y="486" width="14" height="11" rx="2"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="60" y="496" font-size="10">per-client thread</text>
|
||||
|
||||
<rect x="195" y="486" width="14" height="11" rx="2"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="215" y="496" font-size="10">singleton thread</text>
|
||||
|
||||
<rect x="350" y="486" width="14" height="11" rx="2"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="370" y="496" font-size="10">storage</text>
|
||||
|
||||
<rect x="445" y="486" width="14" height="11" rx="2"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="465" y="496" font-size="10">optional</text>
|
||||
|
||||
<text x="40" y="522" font-size="10" fill="#666">
|
||||
Solid arrows: TBQueue connections. Dashed blue: subQ linking per-client to singleton threads.
|
||||
</text>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 7.9 KiB |
@@ -0,0 +1,133 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 740 460" font-family="monospace" font-size="12">
|
||||
<defs>
|
||||
<marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
|
||||
</marker>
|
||||
<marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- Title -->
|
||||
<text x="370" y="22" text-anchor="middle" font-size="15" font-weight="bold">XFTP Router -- Component Topology</text>
|
||||
|
||||
<!-- ===== REQUEST HANDLING (no persistent threads) ===== -->
|
||||
<rect x="25" y="38" width="510" height="280" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="53" fill="#888" font-size="10">per request (inline HTTP/2 callback, no spawned threads)</text>
|
||||
|
||||
<!-- network in -->
|
||||
<text x="13" y="78" font-size="9" fill="#999">net</text>
|
||||
<line x1="12" y1="82" x2="45" y2="82"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
|
||||
<!-- HTTP/2 Transport -->
|
||||
<rect x="45" y="66" width="190" height="34" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="140" y="88" text-anchor="middle">HTTP/2 handler</text>
|
||||
|
||||
<!-- Handshake state -->
|
||||
<rect x="280" y="60" width="240" height="48" rx="4"
|
||||
fill="#fce8e6" stroke="#ea4335" />
|
||||
<text x="400" y="78" text-anchor="middle" font-size="11">Handshake State (per session)</text>
|
||||
<text x="400" y="96" text-anchor="middle" font-size="10" fill="#666">None -> Sent -> Accepted</text>
|
||||
|
||||
<!-- sessions TMap link -->
|
||||
<line x1="235" y1="80" x2="278" y2="80"
|
||||
stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
|
||||
<text x="256" y="74" text-anchor="middle" font-size="9" fill="#555">sessions</text>
|
||||
|
||||
<!-- HTTP/2 handler -> Command Processing -->
|
||||
<line x1="140" y1="100" x2="140" y2="118"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
|
||||
<!-- Command Processing -->
|
||||
<rect x="45" y="120" width="475" height="36" rx="4"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="282" y="143" text-anchor="middle">Command Processing (FNEW, FADD, FPUT, FGET, FACK, FDEL)</text>
|
||||
|
||||
<!-- Command Processing -> FileStore -->
|
||||
<line x1="160" y1="156" x2="130" y2="180"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
|
||||
<!-- Command Processing -> Disk Storage -->
|
||||
<line x1="380" y1="156" x2="390" y2="180"
|
||||
stroke="#333" marker-end="url(#arr)" />
|
||||
|
||||
<!-- FileStore -->
|
||||
<rect x="45" y="182" width="180" height="48" rx="4"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="135" y="203" text-anchor="middle" font-weight="bold">FileStore</text>
|
||||
<text x="135" y="221" text-anchor="middle" font-size="10">(TMap in STM)</text>
|
||||
|
||||
<!-- Disk Storage -->
|
||||
<rect x="300" y="182" width="220" height="48" rx="4"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="410" y="203" text-anchor="middle" font-weight="bold">Disk Storage</text>
|
||||
<text x="410" y="221" text-anchor="middle" font-size="10">filesPath / senderId / data</text>
|
||||
|
||||
<!-- Quota note -->
|
||||
<text x="410" y="246" text-anchor="middle" font-size="9" fill="#888">quota-managed via usedStorage TVar</text>
|
||||
|
||||
<!-- StoreLog -->
|
||||
<rect x="45" y="260" width="180" height="18" rx="3"
|
||||
fill="#f5f5f5" stroke="#ccc" />
|
||||
<text x="135" y="273" text-anchor="middle" font-size="9" fill="#888">StoreLog (append-only)</text>
|
||||
|
||||
<!-- FileStore -> StoreLog -->
|
||||
<line x1="135" y1="230" x2="135" y2="258"
|
||||
stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
|
||||
|
||||
<!-- network out (command result) -->
|
||||
<line x1="520" y1="138" x2="548" y2="138"
|
||||
stroke="#999" marker-end="url(#arr-g)" />
|
||||
<text x="555" y="143" font-size="9" fill="#999">net</text>
|
||||
|
||||
<!-- ===== BACKGROUND THREADS ===== -->
|
||||
<rect x="25" y="335" width="690" height="50" rx="6"
|
||||
fill="none" stroke="#888" stroke-dasharray="6,3" />
|
||||
<text x="35" y="350" fill="#888" font-size="10">background threads (singleton, in raceAny_)</text>
|
||||
|
||||
<!-- expireFiles -->
|
||||
<rect x="45" y="358" width="118" height="22" rx="3"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="104" y="374" text-anchor="middle" font-size="10">expireFiles</text>
|
||||
|
||||
<!-- logServerStats -->
|
||||
<rect x="180" y="358" width="128" height="22" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="244" y="374" text-anchor="middle" font-size="10">logServerStats</text>
|
||||
|
||||
<!-- prometheus -->
|
||||
<rect x="325" y="358" width="118" height="22" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="384" y="374" text-anchor="middle" font-size="10">prometheus</text>
|
||||
|
||||
<!-- controlPort -->
|
||||
<rect x="460" y="358" width="118" height="22" rx="3"
|
||||
fill="#f5f5f5" stroke="#bbb" />
|
||||
<text x="519" y="374" text-anchor="middle" font-size="10">controlPort</text>
|
||||
|
||||
<!-- ===== LEGEND ===== -->
|
||||
<rect x="25" y="400" width="690" height="50" rx="4"
|
||||
fill="none" stroke="#ddd" />
|
||||
|
||||
<rect x="40" y="414" width="14" height="11" rx="2"
|
||||
fill="#e8f0fe" stroke="#4285f4" />
|
||||
<text x="60" y="424" font-size="10">request handler (no threads)</text>
|
||||
|
||||
<rect x="250" y="414" width="14" height="11" rx="2"
|
||||
fill="#fef7e0" stroke="#f9ab00" />
|
||||
<text x="270" y="424" font-size="10">storage</text>
|
||||
|
||||
<rect x="350" y="414" width="14" height="11" rx="2"
|
||||
fill="#fce8e6" stroke="#ea4335" />
|
||||
<text x="370" y="424" font-size="10">per-session state</text>
|
||||
|
||||
<rect x="500" y="414" width="14" height="11" rx="2"
|
||||
fill="#e6f4ea" stroke="#34a853" />
|
||||
<text x="520" y="424" font-size="10">background thread</text>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.5 KiB |
@@ -16,6 +16,8 @@ The XFTP router runs several concurrent threads via `raceAny_`:
|
||||
| `savePrometheusMetrics` | Periodic Prometheus metrics dump |
|
||||
| `runCPServer` | Control port for admin commands |
|
||||
|
||||
See [spec/routers.md](../../routers.md) for component and sequence diagrams.
|
||||
|
||||
## Non-obvious behavior
|
||||
|
||||
### 1. Three-state handshake with session caching
|
||||
|
||||
@@ -18,6 +18,8 @@ The NTF router runs several concurrent threads via `raceAny_`:
|
||||
|
||||
Each client connection spawns `receive`, `send`, and `client` threads via `raceAny_`.
|
||||
|
||||
See [spec/routers.md](../../../routers.md) for component and sequence diagrams.
|
||||
|
||||
## Non-obvious behavior
|
||||
|
||||
### 1. Timing attack mitigation on entity lookup
|
||||
|
||||
@@ -10,6 +10,8 @@
|
||||
|
||||
The router runs as `raceAny_` over many threads — any thread exit stops the entire router process. The thread set includes: one `serverThread` per subscription type (SMP, NTF), a notification delivery thread, a pending events thread, a proxy agent receiver, a SIGINT handler, plus per-transport listener threads and optional expiration/stats/prometheus/control-port threads. `E.finally` ensures `stopServer` runs on any exit.
|
||||
|
||||
See [spec/routers.md](../../routers.md) for component and sequence diagrams.
|
||||
|
||||
## serverThread — subscription lifecycle with split STM
|
||||
|
||||
See comment on `serverThread`. It reads the subscription request from `subQ`, then looks up the client **outside** STM (via `getServerClient`), then enters an STM transaction (`updateSubscribers`) to compute which old subscriptions to end, then runs `endPreviousSubscriptions` in IO. If the client disconnects between lookup and transaction, `updateSubscribers` handles `Nothing` by still sending END/DELD to other subscribed clients.
|
||||
|
||||
+164
@@ -0,0 +1,164 @@
|
||||
# Router Architecture
|
||||
|
||||
SimpleX routers are the Layer 1 network infrastructure. This document shows their internal architecture: component topology and command processing flows.
|
||||
|
||||
For deployment and configuration, see [docs/ROUTERS.md](../docs/ROUTERS.md). For protocol specifications, see [SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md), [Push Notifications](../protocol/push-notifications.md).
|
||||
|
||||
---
|
||||
|
||||
## SMP Router
|
||||
|
||||
**Module specs**: [Server](modules/Simplex/Messaging/Server.md) · [Main](modules/Simplex/Messaging/Server/Main.md) · [QueueStore](modules/Simplex/Messaging/Server/QueueStore.md) · [QueueStore Postgres](modules/Simplex/Messaging/Server/QueueStore/Postgres.md) · [MsgStore](modules/Simplex/Messaging/Server/MsgStore.md) · [StoreLog](modules/Simplex/Messaging/Server/StoreLog.md) · [Control](modules/Simplex/Messaging/Server/Control.md) · [Prometheus](modules/Simplex/Messaging/Server/Prometheus.md) · [Stats](modules/Simplex/Messaging/Server/Stats.md)
|
||||
|
||||
### Component topology
|
||||
|
||||
|
||||
|
||||
### Packet delivery flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as Sender
|
||||
|
||||
box SMP Router
|
||||
participant auth as Command<br>Authorization
|
||||
participant QS as QueueStore
|
||||
participant MS as MsgStore
|
||||
participant del as Packet<br>Delivery
|
||||
end
|
||||
|
||||
participant R as Recipient
|
||||
|
||||
S->>auth: SEND (queue ID + packet)
|
||||
auth->>QS: verify sender key (constant-time)
|
||||
auth->>MS: store packet
|
||||
auth->>S: OK (via sndQ)
|
||||
|
||||
auth->>del: tryDeliverMessage
|
||||
|
||||
alt recipient has active SUB
|
||||
del->>R: MSG (via recipient's sndQ)
|
||||
R->>auth: ACK
|
||||
auth->>MS: delete packet
|
||||
else no active subscriber
|
||||
Note over MS: packet waits in MsgStore
|
||||
R->>auth: SUB (subscribe to queue)
|
||||
auth->>MS: fetch pending packets
|
||||
del->>R: MSG
|
||||
end
|
||||
```
|
||||
|
||||
### Proxy forwarding flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant P as Proxy Router
|
||||
participant D as Destination Router
|
||||
|
||||
C->>P: PRXY (destination address)
|
||||
P->>D: connect (if not already connected)
|
||||
P->>C: PKEY (proxy session key)
|
||||
|
||||
C->>P: PFWD (encrypted command for destination)
|
||||
P->>D: RFWD (relay forwarded command)
|
||||
D->>P: command result
|
||||
P->>C: command result
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## XFTP Router
|
||||
|
||||
**Module specs**: [Server](modules/Simplex/FileTransfer/Server.md) · [Main](modules/Simplex/FileTransfer/Server/Main.md) · [Store](modules/Simplex/FileTransfer/Server/Store.md) · [StoreLog](modules/Simplex/FileTransfer/Server/StoreLog.md) · [Stats](modules/Simplex/FileTransfer/Server/Stats.md) · [Transport](modules/Simplex/FileTransfer/Transport.md)
|
||||
|
||||
### Component topology
|
||||
|
||||
|
||||
|
||||
### Data packet delivery flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as Sender
|
||||
|
||||
box XFTP Router
|
||||
participant HS as Handshake
|
||||
participant CP as Command<br>Processing
|
||||
participant FS as FileStore
|
||||
participant D as Disk
|
||||
end
|
||||
|
||||
participant R as Recipient
|
||||
|
||||
S->>HS: HELLO
|
||||
HS->>S: server DH key + version
|
||||
|
||||
S->>CP: FNEW (create data packet)
|
||||
CP->>FS: create FileRec, reserve quota
|
||||
CP->>S: sender ID + recipient IDs
|
||||
|
||||
S->>CP: FPUT (send encrypted data)
|
||||
CP->>D: write to disk
|
||||
CP->>FS: commit filePath
|
||||
CP->>S: OK
|
||||
|
||||
R->>HS: HELLO
|
||||
HS->>R: server DH key + version
|
||||
|
||||
R->>CP: FGET (recipient DH key)
|
||||
CP->>CP: DH key agreement
|
||||
CP->>D: read file
|
||||
CP->>R: encrypted data stream
|
||||
|
||||
R->>CP: FACK
|
||||
CP->>FS: delete recipient entry
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## NTF Router
|
||||
|
||||
**Module specs**: [Server](modules/Simplex/Messaging/Notifications/Server.md) · [Main](modules/Simplex/Messaging/Notifications/Server/Main.md) · [Store Postgres](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) · [APNS](modules/Simplex/Messaging/Notifications/Server/Push/APNS.md) · [Control](modules/Simplex/Messaging/Notifications/Server/Control.md) · [Client](modules/Simplex/Messaging/Notifications/Client.md) · [Protocol](modules/Simplex/Messaging/Notifications/Protocol.md)
|
||||
|
||||
### Component topology
|
||||
|
||||
|
||||
|
||||
### Token registration and notification delivery
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant App
|
||||
|
||||
box NTF Router
|
||||
participant cl as client thread
|
||||
participant Store
|
||||
participant sub as ntfSubscriber
|
||||
participant push as ntfPush
|
||||
end
|
||||
|
||||
participant SMP as SMP Router
|
||||
participant APNS
|
||||
|
||||
App->>cl: TNEW (push token + DH key)
|
||||
cl->>Store: create token (NTRegistered)
|
||||
cl->>push: PNVerification (via pushQ)
|
||||
push->>APNS: verification push
|
||||
APNS-->>App: verification code (encrypted)
|
||||
App->>cl: TVFY (code)
|
||||
cl->>Store: token -> NTActive
|
||||
|
||||
App->>cl: SNEW (subscribe to SMP queue)
|
||||
cl->>Store: create subscription
|
||||
cl->>SMP: NKEY (subscribe for notifications)
|
||||
SMP->>cl: OK (notifier ID)
|
||||
|
||||
Note over SMP: message arrives on queue
|
||||
SMP->>sub: NMSG (via msgQ)
|
||||
sub->>Store: update tokenLastNtfs
|
||||
sub->>push: PNMessage (via pushQ)
|
||||
push->>APNS: push notification
|
||||
APNS-->>App: notification (ID only)
|
||||
App->>SMP: connect and retrieve message
|
||||
```
|
||||
Reference in New Issue
Block a user