mirror of
https://github.com/simplex-chat/simplexmq.git
synced 2026-03-30 14:16:00 +00:00
add plans
This commit is contained in:
304
plans/page-rendering-overview.md
Normal file
304
plans/page-rendering-overview.md
Normal file
@@ -0,0 +1,304 @@
|
||||
# SMP Server Page Generation — Overview
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Architecture](#architecture)
|
||||
2. [Call Graph](#call-graph)
|
||||
3. [Files by Layer](#files-by-layer)
|
||||
4. [Data Flow: INI → Types → Template → HTML](#data-flow-ini--types--template--html)
|
||||
5. [INI Configuration](#ini-configuration)
|
||||
6. [ServerInformation Construction](#serverinformation-construction)
|
||||
7. [Template Engine](#template-engine)
|
||||
8. [Template Variables](#template-variables-indexhtml)
|
||||
9. [Serving Modes and Routing](#serving-modes-and-routing)
|
||||
10. [Link Pages](#link-pages)
|
||||
11. [Static Assets](#static-assets)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
The SMP server generates a static mini-site at startup and serves it via three possible mechanisms: standalone HTTP, standalone HTTPS, or ALPN-multiplexed on the SMP TLS port.
|
||||
|
||||
## Call Graph
|
||||
|
||||
```
|
||||
Main.main
|
||||
└─ smpServerCLI_(Static.generateSite, Static.serveStaticFiles, Static.attachStaticFiles, ...)
|
||||
└─ runServer
|
||||
├─ builds ServerInformation { ServerPublicConfig, Maybe ServerPublicInfo }
|
||||
├─ runWebServer(path, httpsParams, serverInfo)
|
||||
│ ├─ generateSite(si, onionHost, path) ← writes files to disk
|
||||
│ │ ├─ serverInformation(si, onionHost) ← renders index.html
|
||||
│ │ │ └─ render(E.indexHtml, substs)
|
||||
│ │ │ └─ section_ / item_ ← template engine
|
||||
│ │ ├─ copyDir "media" E.mediaContent
|
||||
│ │ ├─ copyDir "well-known" E.wellKnown
|
||||
│ │ └─ createLinkPage × 7 (contact, invitation, a, c, g, r, i)
|
||||
│ │ └─ writes E.linkHtml
|
||||
│ └─ serveStaticFiles(EmbeddedWebParams) ← starts HTTP/HTTPS Warp
|
||||
│ └─ staticFiles(path) :: Application
|
||||
│ └─ wai-app-static + .well-known rewrite
|
||||
└─ [if sharedHTTP] attachStaticFiles(path, action)
|
||||
└─ Warp's serveConnection on ALPN-routed HTTP connections
|
||||
```
|
||||
|
||||
## Files by Layer
|
||||
|
||||
| Layer | File | Role |
|
||||
|---|---|---|
|
||||
| **Entry** | `apps/smp-server/Main.hs:21` | Wires `Static.*` into `smpServerCLI_` |
|
||||
| **Orchestration** | `src/.../Server/Main.hs:466-603` | `runServer` — builds `ServerInformation`, calls `runWebServer`, decides `attachStaticFiles` vs standalone |
|
||||
| **INI parsing** | `src/.../Server/Main.hs:748-779` | `serverPublicInfo` reads `[INFORMATION]` section into `ServerPublicInfo` |
|
||||
| **INI generation** | `src/.../Server/Main/Init.hs:65-171` | `iniFileContent` generates `[WEB]` section; `informationIniContent` generates `[INFORMATION]` section |
|
||||
| **Types** | `src/.../Server/Information.hs` | `ServerInformation`, `ServerPublicConfig`, `ServerPublicInfo`, `Entity`, `ServerContactAddress`, `PGPKey`, `HostingType`, etc. |
|
||||
| **Generation** | `apps/smp-server/web/Static.hs:95-117` | `generateSite` — writes all files to disk |
|
||||
| **Rendering** | `apps/smp-server/web/Static.hs:119-253` | `serverInformation` — builds substitution pairs; `render`/`section_`/`item_` — template engine |
|
||||
| **Serving** | `apps/smp-server/web/Static.hs:39-93` | `serveStaticFiles` (standalone Warp), `attachStaticFiles` (shared TLS port), `staticFiles` (WAI app) |
|
||||
| **Embedding** | `apps/smp-server/web/Static/Embedded.hs` | TH `embedFile`/`embedDir` for `index.html`, `link.html`, `media/`, `.well-known/` |
|
||||
| **Transport routing** | `src/.../Server.hs:202-219` | `runServer` per-port — routes `sniUsed` TLS connections to `attachHTTP` |
|
||||
| **Transport type** | `src/.../Server.hs:163` | `type AttachHTTP = Socket -> TLS.Context -> IO ()` |
|
||||
| **Shared port detection** | `src/.../Server/CLI.hs:374-387` | `iniTransports` — sets `addHTTP=True` when a transport port matches `[WEB] https` |
|
||||
|
||||
## Data Flow: INI → Types → Template → HTML
|
||||
|
||||
```
|
||||
smp-server.ini
|
||||
│
|
||||
├─ [INFORMATION] section
|
||||
│ └─ serverPublicInfo (Main.hs:748)
|
||||
│ └─ Maybe ServerPublicInfo
|
||||
│
|
||||
├─ [WEB] section
|
||||
│ ├─ static_path → webStaticPath'
|
||||
│ ├─ http → webHttpPort
|
||||
│ └─ https + cert + key → webHttpsParams'
|
||||
│
|
||||
├─ [TRANSPORT] section
|
||||
│ ├─ host → onionHost detection (find THOnionHost in parsed hosts)
|
||||
│ └─ port → iniTransports (sets addHTTP when port == [WEB] https)
|
||||
│
|
||||
└─ Runtime config (ServerConfig fields)
|
||||
└─ ServerPublicConfig { persistence, messageExpiration, statsEnabled, newQueuesAllowed, basicAuthEnabled }
|
||||
│
|
||||
└─ ServerInformation { config, information }
|
||||
│
|
||||
└─ serverInformation (Static.hs:119)
|
||||
│
|
||||
├─ substConfig: 5 always-present substitution pairs
|
||||
├─ substInfo: conditional substitution pairs from ServerPublicInfo
|
||||
└─ onionHost: optional meta tag
|
||||
│
|
||||
└─ render(E.indexHtml, substs)
|
||||
│
|
||||
└─ section_ / item_ engine
|
||||
│
|
||||
└─ ByteString → written to sitePath/index.html
|
||||
```
|
||||
|
||||
## INI Configuration
|
||||
|
||||
### `[INFORMATION]` Section — parsed by `serverPublicInfo` (Main.hs:748-779)
|
||||
|
||||
| INI key | Type | Maps to |
|
||||
|---|---|---|
|
||||
| `source_code` | Required (gates entire section) | `ServerPublicInfo.sourceCode` |
|
||||
| `usage_conditions` | Optional | `ServerConditions.conditions` |
|
||||
| `condition_amendments` | Optional | `ServerConditions.amendments` |
|
||||
| `server_country` | Optional, ISO-3166 2-letter | `ServerPublicInfo.serverCountry` |
|
||||
| `operator` | Optional | `Entity.name` |
|
||||
| `operator_country` | Optional, ISO-3166 | `Entity.country` |
|
||||
| `website` | Optional | `ServerPublicInfo.website` |
|
||||
| `admin_simplex` | Optional, SimpleX address | `ServerContactAddress.simplex` |
|
||||
| `admin_email` | Optional | `ServerContactAddress.email` |
|
||||
| `admin_pgp` + `admin_pgp_fingerprint` | Optional, both required | `PGPKey` |
|
||||
| `complaints_simplex`, `complaints_email`, `complaints_pgp`, `complaints_pgp_fingerprint` | Same structure as admin | `ServerPublicInfo.complaintsContacts` |
|
||||
| `hosting` | Optional | `Entity.name` |
|
||||
| `hosting_country` | Optional, ISO-3166 | `Entity.country` |
|
||||
| `hosting_type` | Optional | `HostingType` (virtual/dedicated/colocation/owned) |
|
||||
|
||||
If `source_code` is absent, `serverPublicInfo` returns `Nothing` and the entire information section is omitted.
|
||||
|
||||
### `[WEB]` Section — parsed in `runServer` (Main.hs:597-603)
|
||||
|
||||
| INI key | Parser | Variable |
|
||||
|---|---|---|
|
||||
| `static_path` | `lookupValue` | `webStaticPath'` — if absent, no site generated |
|
||||
| `http` | `read . T.unpack` | `webHttpPort` — standalone HTTP Warp |
|
||||
| `https` | `read . T.unpack` | `webHttpsParams'.port` — standalone HTTPS Warp OR shared port |
|
||||
| `cert` | `T.unpack` | `webHttpsParams'.cert` |
|
||||
| `key` | `T.unpack` | `webHttpsParams'.key` |
|
||||
|
||||
### `[TRANSPORT]` Section — affects serving mode
|
||||
|
||||
| INI key | Effect on page serving |
|
||||
|---|---|
|
||||
| `host` | Parsed for `.onion` hostnames → `onionHost` → `<x-onionHost>` meta tag |
|
||||
| `port` | Comma-separated ports. If any matches `[WEB] https`, that port gets `addHTTP=True` |
|
||||
|
||||
## ServerInformation Construction
|
||||
|
||||
Built in `runServer` (Main.hs:454-465) from runtime `ServerConfig` fields:
|
||||
|
||||
```haskell
|
||||
ServerPublicConfig
|
||||
{ persistence -- derived from serverStoreCfg:
|
||||
-- SSCMemory Nothing → SPMMemoryOnly
|
||||
-- SSCMemory (Just {storeMsgsFile=Nothing}) → SPMQueues
|
||||
-- otherwise → SPMMessages
|
||||
, messageExpiration -- ttl <$> cfg.messageExpiration
|
||||
, statsEnabled -- isJust logStats
|
||||
, newQueuesAllowed -- cfg.allowNewQueues
|
||||
, basicAuthEnabled -- isJust cfg.newQueueBasicAuth
|
||||
}
|
||||
|
||||
ServerInformation { config, information }
|
||||
-- information = cfg.information :: Maybe ServerPublicInfo (from INI [INFORMATION])
|
||||
```
|
||||
|
||||
## Template Engine
|
||||
|
||||
Custom two-pass substitution in `Static.hs:219-253`:
|
||||
|
||||
1. **`render`**: Iterates over `[(label, Maybe content)]` pairs, calling `section_` for each.
|
||||
2. **`section_`**: Finds `<x-label>...</x-label>` markers. If the substitution value is `Just non-empty`, keeps the section and processes inner `${label}` items via `item_`. If `Nothing` or empty, collapses the entire section. If no section markers found, delegates to `item_` on the whole source.
|
||||
3. **`item_`**: Replaces all `${label}` occurrences with the value.
|
||||
|
||||
## Template Variables (index.html)
|
||||
|
||||
### Substitution Pairs — built by `serverInformation` (Static.hs:119-190)
|
||||
|
||||
**`substConfig`** (always present, derived from `ServerPublicConfig`):
|
||||
|
||||
| Label | Value |
|
||||
|---|---|
|
||||
| `persistence` | `"In-memory only"` / `"Queues"` / `"Queues and messages"` |
|
||||
| `messageExpiration` | `timedTTLText ttl` or `"Never"` |
|
||||
| `statsEnabled` | `"Yes"` / `"No"` |
|
||||
| `newQueuesAllowed` | `"Yes"` / `"No"` |
|
||||
| `basicAuthEnabled` | `"Yes"` / `"No"` |
|
||||
|
||||
**`substInfo`** (from `Maybe ServerPublicInfo`, with `emptyServerInfo ""` as fallback):
|
||||
|
||||
| Label | Source | Conditional |
|
||||
|---|---|---|
|
||||
| `sourceCode` | `spi.sourceCode` | Section present if non-empty |
|
||||
| `noSourceCode` | `Just "none"` if sourceCode empty | Inverse of above |
|
||||
| `version` | `simplexMQVersion` | Always |
|
||||
| `commitSourceCode` | `spi.sourceCode` or `simplexmqSource` | Always |
|
||||
| `shortCommit` | `take 7 simplexmqCommit` | Always |
|
||||
| `commit` | `simplexmqCommit` | Always |
|
||||
| `website` | `spi.website` | Section collapsed if Nothing |
|
||||
| `usageConditions` | `conditions` | Section collapsed if Nothing |
|
||||
| `usageAmendments` | `amendments` | Section collapsed if Nothing |
|
||||
| `operator` | `Just ""` (section marker) | Section collapsed if no operator |
|
||||
| `operatorEntity` | `entity.name` | Inside operator section |
|
||||
| `operatorCountry` | `entity.country` | Inside operator section |
|
||||
| `admin` | `Just ""` (section marker) | Section collapsed if no adminContacts |
|
||||
| `adminSimplex` | `strEncode simplex` | Inside admin section |
|
||||
| `adminEmail` | `email` | Inside admin section |
|
||||
| `adminPGP` | `pkURI` | Inside admin section |
|
||||
| `adminPGPFingerprint` | `pkFingerprint` | Inside admin section |
|
||||
| `complaints` | Same structure as admin | Section collapsed if no complaintsContacts |
|
||||
| `hosting` | `Just ""` (section marker) | Section collapsed if no hosting |
|
||||
| `hostingEntity` | `entity.name` | Inside hosting section |
|
||||
| `hostingCountry` | `entity.country` | Inside hosting section |
|
||||
| `serverCountry` | `spi.serverCountry` | Section collapsed if Nothing |
|
||||
| `hostingType` | `strEncode`, capitalized first letter | Section collapsed if Nothing |
|
||||
|
||||
**`onionHost`** (separate):
|
||||
|
||||
| Label | Source |
|
||||
|---|---|
|
||||
| `onionHost` | `strEncode <$> onionHost` — from `[TRANSPORT] host`, first `.onion` entry |
|
||||
|
||||
## Serving Modes and Routing
|
||||
|
||||
### Mode Decision (Main.hs:466-477)
|
||||
|
||||
```
|
||||
webStaticPath' = [WEB] static_path from INI
|
||||
sharedHTTP = any transport port matches [WEB] https port
|
||||
|
||||
case webStaticPath' of
|
||||
Just path | sharedHTTP →
|
||||
runWebServer path Nothing si -- generate site, NO standalone HTTPS (shared instead)
|
||||
attachStaticFiles path $ \attachHTTP →
|
||||
runSMPServer cfg (Just attachHTTP) -- SMP server with HTTP routing callback
|
||||
Just path →
|
||||
runWebServer path webHttpsParams' si -- generate site, maybe start standalone HTTP/HTTPS
|
||||
runSMPServer cfg Nothing -- SMP server without HTTP routing
|
||||
Nothing →
|
||||
logWarn "No server static path set"
|
||||
runSMPServer cfg Nothing
|
||||
```
|
||||
|
||||
### `runWebServer` (Main.hs:587-596)
|
||||
|
||||
1. Extracts `onionHost` from `[TRANSPORT] host` (finds first `THOnionHost`)
|
||||
2. Extracts `webHttpPort` from `[WEB] http`
|
||||
3. Calls `generateSite si onionHost webStaticPath` — writes all files
|
||||
4. If `webHttpPort` or `webHttpsParams` set → calls `serveStaticFiles` (starts standalone Warp)
|
||||
|
||||
### Shared Port — ALPN Routing (Server.hs:202-219)
|
||||
|
||||
`iniTransports` (CLI.hs:374-387) builds `[(ServiceName, ASrvTransport, AddHTTP)]`:
|
||||
- For each comma-separated port in `[TRANSPORT] port`, creates a `(port, TLS, addHTTP)` entry
|
||||
- `addHTTP = True` when `port == [WEB] https`
|
||||
|
||||
Per-port `runServer` (Server.hs:202):
|
||||
- If `httpCreds` + `attachHTTP_` + `addHTTP` all present:
|
||||
- Uses `combinedCreds = TLSServerCredential { credential = smpCreds, sniCredential = Just httpCreds }`
|
||||
- `runTransportServerState_` with HTTPS TLS params
|
||||
- On each connection: if `sniUsed` (client connected using the HTTP SNI credential) → calls `attachHTTP socket tlsContext`
|
||||
- Otherwise → normal SMP client handling
|
||||
- If not: standard SMP transport, no HTTP routing
|
||||
|
||||
### `attachStaticFiles` (Static.hs:52-73)
|
||||
|
||||
Initializes Warp internal state (`WI.withII`) once, then provides a callback that:
|
||||
1. Gets peer address from socket
|
||||
2. Attaches the TLS context as a Warp connection (`WT.attachConn`)
|
||||
3. Registers a timeout handler
|
||||
4. Calls `WI.serveConnection` — Warp processes HTTP requests using `staticFiles` WAI app
|
||||
|
||||
### `staticFiles` WAI Application (Static.hs:78-93)
|
||||
|
||||
- Uses `wai-app-static` (`S.staticApp`) rooted at the generated site directory
|
||||
- Directory listing disabled (`ssListing = Nothing`)
|
||||
- Custom MIME type: `apple-app-site-association` → `application/json`
|
||||
- Path rewrite: `/.well-known/...` → `/well-known/...` (because `staticApp` doesn't allow hidden folders)
|
||||
|
||||
## Link Pages
|
||||
|
||||
`link.html` is used unchanged for `/contact/`, `/invitation/`, `/a/`, `/c/`, `/g/`, `/r/`, `/i/`. Each path gets a directory with `index.html` = `E.linkHtml`.
|
||||
|
||||
Client-side `contact.js`:
|
||||
1. Reads `document.location` URL
|
||||
2. Extracts path action (`contact`, `a`, etc.)
|
||||
3. Rewrites protocol to `https://`
|
||||
4. Constructs `simplex:` app URI with hostname injection into hash params
|
||||
5. Sets `mobileConnURIanchor.href` to app URI
|
||||
6. Renders QR code of the HTTPS URL via `qrcode.js`
|
||||
|
||||
## Static Assets
|
||||
|
||||
All under `apps/smp-server/static/`, embedded at compile time via `file-embed` TH:
|
||||
|
||||
| File | Embedded via | Purpose |
|
||||
|---|---|---|
|
||||
| `index.html` | `embedFile` → `E.indexHtml` | Server information page template |
|
||||
| `link.html` | `embedFile` → `E.linkHtml` | Contact/invitation link page |
|
||||
| `media/*` | `embedDir` → `E.mediaContent` | CSS, JS, fonts, icons (23 files) |
|
||||
| `.well-known/*` | `embedDir` → `E.wellKnown` | `apple-app-site-association`, `assetlinks.json` |
|
||||
|
||||
Media files include: `style.css`, `tailwind.css`, `script.js`, `contact.js`, `qrcode.js`, `swiper-bundle.min.{css,js}`, `favicon.ico`, `logo-{light,dark}.png`, `logo-symbol-{light,dark}.svg`, `sun.svg`, `moon.svg`, `apple_store.svg`, `google_play.svg`, `f_droid.svg`, `testflight.png`, `apk_icon.png`, `contact_page_mobile.png`, `Gilroy{Bold,Light,Medium,Regular,RegularItalic}.woff2`.
|
||||
|
||||
### Cabal Dependencies (smp-server executable, simplexmq.cabal:398-429)
|
||||
|
||||
```
|
||||
other-modules: Static, Static.Embedded
|
||||
hs-source-dirs: apps/smp-server, apps/smp-server/web
|
||||
build-depends: file-embed, wai, wai-app-static, warp ==3.3.30, warp-tls ==3.4.7,
|
||||
network, directory, filepath, text, bytestring, unliftio, simple-logger
|
||||
```
|
||||
491
plans/xftp-server-pages-implementation.md
Normal file
491
plans/xftp-server-pages-implementation.md
Normal file
@@ -0,0 +1,491 @@
|
||||
# XFTP Server Pages Implementation Plan
|
||||
|
||||
## Table of Contents
|
||||
1. [Context](#context)
|
||||
2. [Executive Summary](#executive-summary)
|
||||
3. [High-Level Design](#high-level-design)
|
||||
4. [Detailed Implementation Plan](#detailed-implementation-plan)
|
||||
5. [Verification](#verification)
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
The SMP server has a full web infrastructure: server info page, link pages, static site generation, and serving (standalone HTTP/HTTPS or shared TLS port). The XFTP server has none of this — only `httpCredentials` for CORS/browser access to the XFTP protocol.
|
||||
|
||||
**Goal:** Add XFTP server pages identical in structure to SMP, with XFTP-specific configuration display and a `/file` page embedding the xftp-web upload/download app.
|
||||
|
||||
---
|
||||
|
||||
## 2. Executive Summary
|
||||
|
||||
**Share SMP's `Static.hs` via `hs-source-dirs`** — XFTP's cabal section references `apps/smp-server/web` for `Static.hs`, while providing its own `Static/Embedded.hs` with XFTP-specific templates. Zero code duplication for serving/rendering logic.
|
||||
|
||||
**Key changes (Haskell):**
|
||||
- Parameterize `Static.hs`: use `E.linkPages` + `E.extraDirs` instead of hardcoded link page list
|
||||
- Create `apps/xftp-server/web/Static/Embedded.hs` (XFTP templates + xftp-web dist)
|
||||
- Create `apps/xftp-server/static/index.html` (XFTP server info template)
|
||||
- Add `xftpServerCLI_` callback pattern (mirrors `smpServerCLI_`)
|
||||
- Add `[INFORMATION]` section, `[WEB] static_path/http/relay_servers` to XFTP INI parsing/generation
|
||||
- Update `simplexmq.cabal`: XFTP exe gets `Static`, `Static.Embedded` modules + web deps
|
||||
|
||||
**Key changes (TypeScript):**
|
||||
- `servers.ts`: add `loadServers()` — fetches `./servers.json` at runtime, falls back to baked-in defaults
|
||||
- `main.ts`: call `await loadServers()` before `initApp()`
|
||||
- `vite.config.ts`: add `server` mode (empty baked-in servers, CSP placeholder preserved, `base: './'`)
|
||||
|
||||
**Reused without duplication:**
|
||||
- `ServerInformation`, `ServerPublicConfig`, `ServerPublicInfo` types (`src/Simplex/Messaging/Server/Information.hs`)
|
||||
- `serverPublicInfo` INI parser (`src/Simplex/Messaging/Server/Main.hs`)
|
||||
- `EmbeddedWebParams`, `WebHttpsParams` types (`src/Simplex/Messaging/Server/Main.hs`)
|
||||
- All of `Static.hs`: `generateSite`, `serverInformation`, `serveStaticFiles`, `attachStaticFiles`, `staticFiles`, `render`, `section_`, `item_`, `timedTTLText`
|
||||
- All media assets (CSS, JS, fonts, icons) via `$(embedDir "apps/smp-server/static/media/")`
|
||||
|
||||
---
|
||||
|
||||
## 3. High-Level Design
|
||||
|
||||
### Module Sharing Strategy
|
||||
|
||||
```
|
||||
apps/smp-server/web/Static.hs ← SHARED (both exes use this)
|
||||
apps/smp-server/web/Static/Embedded.hs ← SMP-specific embedded content
|
||||
apps/xftp-server/web/Static/Embedded.hs ← XFTP-specific embedded content
|
||||
|
||||
XFTP cabal hs-source-dirs order:
|
||||
1. apps/xftp-server/web → finds Static/Embedded.hs (XFTP version)
|
||||
2. apps/smp-server/web → finds Static.hs (shared)
|
||||
|
||||
GHC searches source dirs in order, first match wins:
|
||||
Static.hs → NOT in xftp-server/web → found in smp-server/web ✓
|
||||
Static/Embedded → found in xftp-server/web first ✓
|
||||
```
|
||||
|
||||
**Note:** `Static.hs` imports `Simplex.Messaging.Server (AttachHTTP)` — this SMP module IS exposed in the library, so the XFTP executable can import it. `AttachHTTP` is just a type alias `Socket -> TLS.Context -> IO ()`.
|
||||
|
||||
### ServerPublicConfig Mapping (XFTP → existing fields)
|
||||
|
||||
| XFTP concept | `ServerPublicConfig` field | Value |
|
||||
|---|---|---|
|
||||
| *(not applicable)* | `persistence` | `SPMMemoryOnly` |
|
||||
| File expiration | `messageExpiration` | `ttl <$> fileExpiration` |
|
||||
| Stats enabled | `statsEnabled` | `isJust logStats` |
|
||||
| File upload allowed | `newQueuesAllowed` | `allowNewFiles` |
|
||||
| Basic auth enabled | `basicAuthEnabled` | `isJust newFileBasicAuth` |
|
||||
|
||||
The XFTP `index.html` template uses the same `${...}` variable names but has different label text. The `persistence` row is omitted from the XFTP template entirely.
|
||||
|
||||
### `/file` Page Architecture
|
||||
|
||||
```
|
||||
sitePath/
|
||||
index.html ← XFTP server info page (from template)
|
||||
media/ ← shared CSS, JS, fonts, icons
|
||||
well-known/ ← AASA, assetlinks
|
||||
file/ ← xftp-web dist (from extraDirs)
|
||||
index.html ← CSP patched at site generation time
|
||||
servers.json ← generated from [WEB] relay_servers
|
||||
assets/
|
||||
index-xxx.js ← xftp-web compiled JS bundle
|
||||
index-xxx.css ← styles
|
||||
crypto.worker-xxx.js ← encryption Web Worker
|
||||
```
|
||||
|
||||
### Data Flow at Runtime
|
||||
|
||||
```
|
||||
file-server.ini
|
||||
├─ [INFORMATION] → serverPublicInfo → Maybe ServerPublicInfo
|
||||
├─ [WEB] relay_servers → relayServers :: [Text]
|
||||
├─ [WEB] static_path → sitePath
|
||||
└─ XFTPServerConfig fields → ServerPublicConfig
|
||||
↓
|
||||
ServerInformation {config, information}
|
||||
↓
|
||||
generateSite si onionHost sitePath ← writes index.html + media + file/
|
||||
↓
|
||||
writeRelayConfig sitePath relayServers ← writes file/servers.json, patches CSP
|
||||
↓
|
||||
serveStaticFiles EmbeddedWebParams ← standalone HTTP/HTTPS warp
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Detailed Implementation Plan
|
||||
|
||||
### Step 1: Parameterize `Static.hs`
|
||||
|
||||
**File:** `apps/smp-server/web/Static.hs`
|
||||
|
||||
1. Add import: `System.FilePath (takeDirectory)`
|
||||
2. Replace hardcoded link pages with `E.linkPages`; add `E.extraDirs` copying:
|
||||
|
||||
```haskell
|
||||
-- BEFORE:
|
||||
createLinkPage "contact"
|
||||
createLinkPage "invitation"
|
||||
createLinkPage "a"
|
||||
createLinkPage "c"
|
||||
createLinkPage "g"
|
||||
createLinkPage "r"
|
||||
createLinkPage "i"
|
||||
|
||||
-- AFTER:
|
||||
mapM_ createLinkPage E.linkPages
|
||||
forM_ E.extraDirs $ \(dir, content) -> do
|
||||
createDirectoryIfMissing True $ sitePath </> dir
|
||||
forM_ content $ \(path, s) -> do
|
||||
createDirectoryIfMissing True $ sitePath </> dir </> takeDirectory path
|
||||
B.writeFile (sitePath </> dir </> path) s
|
||||
```
|
||||
|
||||
No change to `serverInformation` — it already uses `E.indexHtml` which resolves per-app via the `Embedded` module.
|
||||
|
||||
### Step 2: Update SMP's `Static/Embedded.hs`
|
||||
|
||||
**File:** `apps/smp-server/web/Static/Embedded.hs`
|
||||
|
||||
Add two new exports to maintain the shared interface:
|
||||
```haskell
|
||||
linkPages :: [FilePath]
|
||||
linkPages = ["contact", "invitation", "a", "c", "g", "r", "i"]
|
||||
|
||||
extraDirs :: [(FilePath, [(FilePath, ByteString)])]
|
||||
extraDirs = []
|
||||
```
|
||||
|
||||
### Step 3: Create XFTP's `Static/Embedded.hs`
|
||||
|
||||
**New file:** `apps/xftp-server/web/Static/Embedded.hs`
|
||||
|
||||
```haskell
|
||||
module Static.Embedded where
|
||||
|
||||
import Data.FileEmbed (embedDir, embedFile)
|
||||
import Data.ByteString (ByteString)
|
||||
|
||||
indexHtml :: ByteString
|
||||
indexHtml = $(embedFile "apps/xftp-server/static/index.html")
|
||||
|
||||
linkHtml :: ByteString
|
||||
linkHtml = "" -- unused: XFTP has no simple link pages
|
||||
|
||||
mediaContent :: [(FilePath, ByteString)]
|
||||
mediaContent = $(embedDir "apps/smp-server/static/media/") -- reuse SMP media
|
||||
|
||||
wellKnown :: [(FilePath, ByteString)]
|
||||
wellKnown = $(embedDir "apps/smp-server/static/.well-known/")
|
||||
|
||||
linkPages :: [FilePath]
|
||||
linkPages = []
|
||||
|
||||
extraDirs :: [(FilePath, [(FilePath, ByteString)])]
|
||||
extraDirs = [("file", $(embedDir "xftp-web/dist-web/"))]
|
||||
```
|
||||
|
||||
**Build dependency:** `xftp-web/dist-web/` must exist at compile time. Build with `cd xftp-web && npm run build -- --mode server` first.
|
||||
|
||||
### Step 4: Create XFTP `index.html` Template
|
||||
|
||||
**New file:** `apps/xftp-server/static/index.html`
|
||||
|
||||
Copy SMP's `apps/smp-server/static/index.html` with these differences:
|
||||
- Title: "SimpleX XFTP - Server Information"
|
||||
- Nav link list: add `<li>` for "/file" ("File transfer")
|
||||
- Configuration section:
|
||||
- **Remove** "Persistence" row
|
||||
- "File expiration:" → `${messageExpiration}`
|
||||
- "Stats enabled:" → `${statsEnabled}`
|
||||
- "File upload allowed:" → `${newQueuesAllowed}`
|
||||
- "Basic auth enabled:" → `${basicAuthEnabled}`
|
||||
- Public information section: identical (same template variables, same structure)
|
||||
- Footer: identical
|
||||
|
||||
### Step 5: Add `xftpServerCLI_` with Callbacks
|
||||
|
||||
**File:** `src/Simplex/FileTransfer/Server/Main.hs`
|
||||
|
||||
**New imports:**
|
||||
```haskell
|
||||
import Simplex.Messaging.Server.Information
|
||||
import Simplex.Messaging.Server.Main (EmbeddedWebParams (..), WebHttpsParams (..), serverPublicInfo, simplexmqSource)
|
||||
import Simplex.Messaging.Transport.Client (TransportHost (..))
|
||||
```
|
||||
|
||||
**New function** (mirrors `smpServerCLI_`):
|
||||
```haskell
|
||||
xftpServerCLI_ ::
|
||||
(ServerInformation -> Maybe TransportHost -> FilePath -> IO ()) ->
|
||||
(EmbeddedWebParams -> IO ()) ->
|
||||
FilePath -> FilePath -> IO ()
|
||||
```
|
||||
|
||||
**Refactor existing:**
|
||||
```haskell
|
||||
xftpServerCLI :: FilePath -> FilePath -> IO ()
|
||||
xftpServerCLI = xftpServerCLI_ (\_ _ _ -> pure ()) (\_ -> pure ())
|
||||
```
|
||||
|
||||
**In `runServer`, add after `printXFTPConfig`:**
|
||||
|
||||
1. Build `ServerPublicConfig` (see mapping table in Section 3)
|
||||
2. Build `ServerInformation {config, information = serverPublicInfo ini}`
|
||||
3. Parse web config:
|
||||
- `webStaticPath' = eitherToMaybe $ T.unpack <$> lookupValue "WEB" "static_path" ini`
|
||||
- `webHttpPort = eitherToMaybe $ read . T.unpack <$> lookupValue "WEB" "http" ini`
|
||||
- `webHttpsParams'` = `{port, cert, key}` from `[WEB]` (same pattern as SMP)
|
||||
- `relayServers = eitherToMaybe $ T.splitOn "," <$> lookupValue "WEB" "relay_servers" ini`
|
||||
4. Extract `onionHost` from `[TRANSPORT] host` (same as SMP)
|
||||
5. Web server logic:
|
||||
```haskell
|
||||
case webStaticPath' of
|
||||
Just path -> do
|
||||
generateSite si onionHost path
|
||||
-- Post-process: inject relay server config into /file page
|
||||
forM_ relayServers $ \servers -> do
|
||||
let fileDir = path </> "file"
|
||||
hosts = map (encodeUtf8 . T.strip) $ filter (not . T.null) servers
|
||||
-- Write servers.json for xftp-web runtime loading
|
||||
B.writeFile (fileDir </> "servers.json") $ "[" <> B.intercalate "," (map (\h -> "\"" <> h <> "\"") hosts) <> "]"
|
||||
-- Patch CSP connect-src in file/index.html (inline ByteString replacement)
|
||||
let cspHosts = B.intercalate " " $ map (parseXFTPHost . T.strip) $ filter (not . T.null) servers
|
||||
marker = "__CSP_CONNECT_SRC__"
|
||||
fileIndex <- B.readFile (fileDir </> "index.html")
|
||||
let (before, after) = B.breakSubstring marker fileIndex
|
||||
patched = if B.null after then fileIndex
|
||||
else before <> cspHosts <> B.drop (B.length marker) after
|
||||
B.writeFile (fileDir </> "index.html") patched
|
||||
when (isJust webHttpPort || isJust webHttpsParams') $
|
||||
serveStaticFiles EmbeddedWebParams {webStaticPath = path, webHttpPort, webHttpsParams = webHttpsParams'}
|
||||
runXFTPServer serverConfig
|
||||
Nothing -> runXFTPServer serverConfig
|
||||
```
|
||||
|
||||
Where `parseXFTPHost` extracts `https://host:port` from an `xftp://fingerprint@host:port` address.
|
||||
|
||||
**Note:** `B.replace` is not in `Data.ByteString.Char8`. Use a simple find-and-replace helper (similar pattern to `item_` in `Static.hs`), or use `Data.ByteString.Search` from `stringsearch` package, or inline a ByteString replacement.
|
||||
|
||||
### Step 6: Update XFTP INI Generation
|
||||
|
||||
**File:** `src/Simplex/FileTransfer/Server/Main.hs` (in `iniFileContent`)
|
||||
|
||||
Add to the generated INI string:
|
||||
|
||||
After existing `[WEB]` section:
|
||||
```ini
|
||||
[WEB]
|
||||
# cert: /etc/opt/simplex-xftp/web.crt
|
||||
# key: /etc/opt/simplex-xftp/web.key
|
||||
# static_path: /var/opt/simplex-xftp/www
|
||||
# http: 8080
|
||||
# relay_servers: xftp://fingerprint@host1,xftp://fingerprint@host2
|
||||
```
|
||||
|
||||
Add new `[INFORMATION]` section (same format as SMP):
|
||||
```ini
|
||||
[INFORMATION]
|
||||
# source_code: https://github.com/simplex-chat/simplexmq
|
||||
# usage_conditions:
|
||||
# condition_amendments:
|
||||
# server_country:
|
||||
# operator:
|
||||
# operator_country:
|
||||
# website:
|
||||
# admin_simplex:
|
||||
# admin_email:
|
||||
# admin_pgp:
|
||||
# admin_pgp_fingerprint:
|
||||
# complaints_simplex:
|
||||
# complaints_email:
|
||||
# complaints_pgp:
|
||||
# complaints_pgp_fingerprint:
|
||||
# hosting:
|
||||
# hosting_country:
|
||||
# hosting_type: virtual
|
||||
```
|
||||
|
||||
### Step 7: Update XFTP Entry Point
|
||||
|
||||
**File:** `apps/xftp-server/Main.hs`
|
||||
|
||||
```haskell
|
||||
import qualified Static
|
||||
import Simplex.FileTransfer.Server.Main (xftpServerCLI_)
|
||||
|
||||
main = do
|
||||
...
|
||||
withGlobalLogging logCfg $ xftpServerCLI_ Static.generateSite Static.serveStaticFiles cfgPath logPath
|
||||
```
|
||||
|
||||
### Step 8: Update `simplexmq.cabal`
|
||||
|
||||
**xftp-server executable:**
|
||||
```cabal
|
||||
executable xftp-server
|
||||
main-is: Main.hs
|
||||
other-modules:
|
||||
Static
|
||||
Static.Embedded
|
||||
Paths_simplexmq
|
||||
hs-source-dirs:
|
||||
apps/xftp-server
|
||||
apps/xftp-server/web
|
||||
apps/smp-server/web
|
||||
build-depends:
|
||||
base
|
||||
, bytestring
|
||||
, directory
|
||||
, file-embed
|
||||
, filepath
|
||||
, network
|
||||
, simple-logger
|
||||
, simplexmq
|
||||
, text
|
||||
, unliftio
|
||||
, wai
|
||||
, wai-app-static
|
||||
, warp ==3.3.30
|
||||
, warp-tls ==3.4.7
|
||||
```
|
||||
|
||||
**extra-source-files:** Add:
|
||||
```cabal
|
||||
apps/xftp-server/static/index.html
|
||||
```
|
||||
|
||||
### Step 9: Modify xftp-web — Runtime Server Loading
|
||||
|
||||
**File:** `xftp-web/web/servers.ts`
|
||||
|
||||
```typescript
|
||||
import {parseXFTPServer, type XFTPServer} from '../src/protocol/address.js'
|
||||
|
||||
declare const __XFTP_SERVERS__: string[]
|
||||
const defaultServers: string[] = __XFTP_SERVERS__
|
||||
|
||||
let runtimeServers: string[] | null = null
|
||||
|
||||
export async function loadServers(): Promise<void> {
|
||||
try {
|
||||
const resp = await fetch('./servers.json')
|
||||
if (resp.ok) {
|
||||
const data: string[] = await resp.json()
|
||||
if (Array.isArray(data) && data.length > 0) {
|
||||
runtimeServers = data
|
||||
}
|
||||
}
|
||||
} catch { /* fall back to defaults */ }
|
||||
}
|
||||
|
||||
export function getServers(): XFTPServer[] {
|
||||
return (runtimeServers ?? defaultServers).map(parseXFTPServer)
|
||||
}
|
||||
|
||||
export function pickRandomServer(servers: XFTPServer[]): XFTPServer {
|
||||
return servers[Math.floor(Math.random() * servers.length)]
|
||||
}
|
||||
```
|
||||
|
||||
**File:** `xftp-web/web/main.ts` — add `loadServers` import and call:
|
||||
|
||||
```typescript
|
||||
import {loadServers} from './servers.js'
|
||||
|
||||
async function main() {
|
||||
await sodium.ready
|
||||
await loadServers()
|
||||
initApp()
|
||||
window.addEventListener('hashchange', initApp)
|
||||
}
|
||||
```
|
||||
|
||||
**File:** `xftp-web/web/download.ts` — NO changes needed (uses server addresses from file description in URL hash, not `getServers()`).
|
||||
|
||||
### Step 10: Modify xftp-web — Vite `server` Build Mode
|
||||
|
||||
**File:** `xftp-web/vite.config.ts`
|
||||
|
||||
Add `server` mode handling:
|
||||
```typescript
|
||||
if (mode === 'server') {
|
||||
define['__XFTP_SERVERS__'] = JSON.stringify([])
|
||||
servers = []
|
||||
} else if (mode === 'development') {
|
||||
// ... existing dev logic
|
||||
} else {
|
||||
// ... existing production logic
|
||||
}
|
||||
```
|
||||
|
||||
CSP plugin: skip replacement in server mode:
|
||||
```typescript
|
||||
handler(html) {
|
||||
if (isDev) return html.replace(/<meta\s[^>]*?Content-Security-Policy[\s\S]*?>/i, '')
|
||||
if (mode === 'server') return html // leave __CSP_CONNECT_SRC__ placeholder
|
||||
return html.replace('__CSP_CONNECT_SRC__', origins)
|
||||
}
|
||||
```
|
||||
|
||||
Add `base: './'` for server mode (relative asset paths, needed for `/file/` subpath):
|
||||
```typescript
|
||||
base: mode === 'server' ? './' : '/',
|
||||
```
|
||||
|
||||
### `servers.json` Format
|
||||
|
||||
Generated at site-generation time by the Haskell server:
|
||||
```json
|
||||
["xftp://fingerprint1@host1:443", "xftp://fingerprint2@host2:443"]
|
||||
```
|
||||
|
||||
Simple JSON array of XFTP server address strings.
|
||||
|
||||
### Fallback Behavior
|
||||
|
||||
| Scenario | Upload servers | Download servers |
|
||||
|---|---|---|
|
||||
| `relay_servers` configured | From `servers.json` | From file description (URL hash) |
|
||||
| `relay_servers` not configured | Build-time defaults (empty in server mode) | From file description (URL hash) |
|
||||
| No `static_path` configured | No `/file` page served | N/A |
|
||||
|
||||
---
|
||||
|
||||
## 5. Verification
|
||||
|
||||
### Build
|
||||
```bash
|
||||
# 1. Build xftp-web for server embedding
|
||||
cd xftp-web && npm run build -- --mode server && cd ..
|
||||
|
||||
# 2. Build both servers (fast, no optimization)
|
||||
cabal build smp-server --ghc-options=-O0
|
||||
cabal build xftp-server --ghc-options=-O0
|
||||
```
|
||||
|
||||
### Test
|
||||
```bash
|
||||
cabal test simplexmq-test --ghc-options=-O0
|
||||
```
|
||||
|
||||
### Manual Smoke Test
|
||||
1. `cabal run xftp-server -- init -p /tmp/xftp-files -q 10gb`
|
||||
2. Edit `file-server.ini`: uncomment `static_path`, `http: 8080`, add `relay_servers`
|
||||
3. `cabal run xftp-server -- start`
|
||||
4. `curl http://localhost:8080/` → server info HTML
|
||||
5. `curl http://localhost:8080/file/` → xftp-web HTML
|
||||
6. `curl http://localhost:8080/file/servers.json` → relay servers JSON
|
||||
|
||||
### Files Modified (Summary)
|
||||
|
||||
| File | Type | Change |
|
||||
|---|---|---|
|
||||
| `apps/smp-server/web/Static.hs` | Modify | `E.linkPages`, `E.extraDirs`, `takeDirectory` import |
|
||||
| `apps/smp-server/web/Static/Embedded.hs` | Modify | Add `linkPages`, `extraDirs` exports |
|
||||
| `apps/xftp-server/Main.hs` | Modify | Import Static, use `xftpServerCLI_` |
|
||||
| `apps/xftp-server/web/Static/Embedded.hs` | **NEW** | XFTP templates + xftp-web dist embedding |
|
||||
| `apps/xftp-server/static/index.html` | **NEW** | XFTP server info HTML template |
|
||||
| `src/Simplex/FileTransfer/Server/Main.hs` | Modify | `xftpServerCLI_`, web logic, INI parsing/generation |
|
||||
| `simplexmq.cabal` | Modify | XFTP exe: modules, deps, source dirs, extra-source-files |
|
||||
| `xftp-web/web/servers.ts` | Modify | Add `loadServers()` for runtime config |
|
||||
| `xftp-web/web/main.ts` | Modify | Call `loadServers()` at startup |
|
||||
| `xftp-web/vite.config.ts` | Modify | `server` mode, conditional `base` |
|
||||
Reference in New Issue
Block a user