diff --git a/rfcs/2026-02-03-browser-transport.md b/rfcs/2026-02-03-browser-transport.md new file mode 100644 index 000000000..41915bf64 --- /dev/null +++ b/rfcs/2026-02-03-browser-transport.md @@ -0,0 +1,920 @@ +# Browser Transport & Web Worker Architecture + +## TOC + +1. Executive Summary +2. Transport: fetch() API +3. Architecture: Environment Abstraction +4. Web Worker Implementation +5. OPFS Implementation +6. Implementation Plan +7. Testing Strategy + +## 1. Executive Summary + +Adapt `client.ts` from `node:http2` to `fetch()` API for isomorphic Node.js/browser support. Add environment abstraction layer so the same upload/download pipeline works with or without Web Workers and with or without OPFS. In browsers, crypto runs in a Web Worker to keep UI responsive; in Node.js tests, crypto runs directly. + +**Key architectural constraint:** Existing crypto functions (`encryptFile`, `decryptChunks`, etc.) remain unchanged. The abstraction layer wraps them, choosing execution context (direct vs Worker) and storage (memory vs OPFS) based on environment. + +**Scope:** +- Replace `node:http2` with `fetch()` in `client.ts` +- Add `CryptoBackend` abstraction with three implementations +- Create Web Worker that calls existing crypto functions +- Add OPFS storage for large files in browser + +**Out of scope:** Web page UI (Phase 5 in main RFC). + +## 2. Transport: fetch() API + +### 2.1 Current State + +`client.ts` uses `node:http2`: +```typescript +import http2 from "node:http2" +const session = http2.connect(url) +const stream = session.request({':method': 'POST', ':path': '/'}) +stream.write(commandBlock) +stream.end(chunkData) +``` + +### 2.2 Target State + +Isomorphic `fetch()` (Node.js 18+ and browsers): +```typescript +const response = await fetch(url, { + method: 'POST', + body: concatStreams(commandBlock, chunkData), + duplex: 'half', // Required for streaming request body +}) +const reader = response.body!.getReader() +``` + +### 2.3 Key Differences + +| Aspect | node:http2 | fetch() | +|--------|-----------|---------| +| Session management | Explicit `session.connect()` / `session.close()` | Per-request (HTTP/2 connection reuse is automatic) | +| Streaming upload | `stream.write()` chunks | `ReadableStream` body + `duplex: 'half'` | +| Streaming download | `stream.on('data')` | `response.body.getReader()` | +| Connection pooling | Manual | Automatic per origin | + +### 2.4 API Changes + +```typescript +// Before (node:http2) +export interface XFTPClient { + session: http2.ClientHttp2Session + thParams: THParams + server: XFTPServer +} + +// After (fetch) +export interface XFTPClient { + baseUrl: string // "https://host:port" + thParams: THParams + server: XFTPServer +} +``` + +`connectXFTP()` performs handshake via fetch, returns `XFTPClient` with `baseUrl`. +Subsequent commands use `fetch(client.baseUrl, ...)`. + +### 2.5 Handshake via fetch() + +**TLS session binding:** Multiple fetch() requests to the same origin reuse the HTTP/2 connection, which means they share the same TLS session. The server's `sessionId` (derived from TLS channel binding) remains consistent across the handshake round-trips and subsequent commands. + +```typescript +async function connectXFTP(server: XFTPServer): Promise { + const baseUrl = `https://${server.host}:${server.port}` + + // Round-trip 1: challenge → server handshake + identity proof + const challenge = crypto.getRandomValues(new Uint8Array(32)) + const req1 = pad(encodeWebClientHello(challenge), xftpBlockSize) + const resp1 = await fetch(baseUrl, {method: 'POST', body: req1}) + + const reader = resp1.body!.getReader() + const serverBlock = await readExactly(reader, xftpBlockSize) + const serverHs = decodeServerHandshake(unPad(serverBlock)) + const proofBody = await readRemaining(reader) + verifyIdentityProof(server.keyHash, challenge, serverHs.sessionId, proofBody) + + // Round-trip 2: client handshake → server ack + const clientHs = encodeClientHandshake({xftpVersion: 3, keyHash: server.keyHash}) + const req2 = pad(clientHs, xftpBlockSize) + await fetch(baseUrl, {method: 'POST', body: req2}) + + return {baseUrl, thParams: {sessionId: serverHs.sessionId, ...}, server} +} +``` + +### 2.6 Command Execution + +```typescript +async function sendXFTPCommand( + client: XFTPClient, + key: Uint8Array, + entityId: Uint8Array, + cmd: Uint8Array, + chunkData?: Uint8Array +): Promise<{response: Uint8Array, body?: ReadableStream}> { + const block = xftpEncodeAuthTransmission(client.thParams, key, entityId, cmd) + + const reqBody = chunkData + ? concatBytes(block, chunkData) + : block + + const resp = await fetch(client.baseUrl, { + method: 'POST', + body: reqBody, + duplex: 'half', + }) + + const reader = resp.body!.getReader() + const responseBlock = await readExactly(reader, xftpBlockSize) + const parsed = xftpDecodeTransmission(responseBlock) + + // For FGET: remaining body is encrypted chunk + const hasMore = await peekReader(reader) + return { + response: parsed, + body: hasMore ? wrapAsStream(reader) : undefined + } +} +``` + +## 3. Architecture: Environment Abstraction + +### 3.1 Core Principle + +**Existing crypto functions remain unchanged.** The functions `encryptFile()`, `decryptChunks()`, `sha512()`, etc. in `crypto/file.ts` and `crypto/digest.ts` are pure computation — they take input bytes and produce output bytes. They have no knowledge of Workers, OPFS, or execution context. + +The abstraction layer sits between `agent.ts` (upload/download orchestration) and these crypto functions: + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ agent.ts (upload/download orchestration) │ +│ - Unchanged logic: encrypt → chunk → upload → build description │ +│ - Calls CryptoBackend interface, not crypto functions directly │ +├─────────────────────────────────────────────────────────────────────┤ +│ CryptoBackend interface (env.ts) │ +│ - Abstract interface for encrypt/decrypt/readChunk/writeChunk │ +│ - Factory function selects implementation based on environment │ +├──────────────┬──────────────────────┬───────────────────────────────┤ +│ DirectMemory │ WorkerMemory │ WorkerOPFS │ +│ Backend │ Backend │ Backend │ +│ (Node.js) │ (Browser, ≤50MB) │ (Browser, >50MB) │ +├──────────────┼──────────────────────┼───────────────────────────────┤ +│ Calls crypto │ Posts to Worker, │ Posts to Worker, │ +│ functions │ Worker calls crypto │ Worker calls crypto, │ +│ directly │ functions, returns │ streams through OPFS │ +│ │ via postMessage │ │ +├──────────────┴──────────────────────┴───────────────────────────────┤ +│ crypto/file.ts, crypto/digest.ts (unchanged) │ +│ - encryptFile(), decryptChunks(), sha512(), etc. │ +│ - Pure functions, no environment dependencies │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +### 3.2 CryptoBackend Interface + +```typescript +// env.ts +export interface CryptoBackend { + // Encrypt file, store result (in memory or OPFS depending on backend) + encrypt( + data: Uint8Array, + fileName: string, + onProgress?: (done: number, total: number) => void + ): Promise + + // Decrypt from stored encrypted data + decrypt( + key: Uint8Array, + nonce: Uint8Array, + size: number, + onProgress?: (done: number, total: number) => void + ): Promise + + // Read chunk from stored encrypted data (for upload) + readChunk(offset: number, size: number): Promise + + // Write chunk to storage (for download, before decrypt) + writeChunk(data: Uint8Array, offset: number): Promise + + // Clean up temporary storage + cleanup(): Promise +} + +export interface EncryptResult { + digest: Uint8Array // SHA-512 of encrypted data + key: Uint8Array // Generated encryption key + nonce: Uint8Array // Generated nonce + chunkSizes: number[] // Chunk sizes for upload + totalSize: number // Total encrypted size +} + +export interface DecryptResult { + header: FileHeader // Extracted file header (fileName, etc.) + content: Uint8Array // Decrypted file content +} +``` + +### 3.3 Backend Implementations + +**DirectMemoryBackend** (Node.js): +```typescript +class DirectMemoryBackend implements CryptoBackend { + private encryptedData: Uint8Array | null = null + + async encrypt(data: Uint8Array, fileName: string, onProgress?): Promise { + const key = randomBytes(32) + const nonce = randomBytes(24) + // Call existing crypto function directly + this.encryptedData = encryptFile(data, fileName, key, nonce, onProgress) + const digest = sha512(this.encryptedData) + const chunkSizes = prepareChunkSizes(this.encryptedData.length) + return { digest, key, nonce, chunkSizes, totalSize: this.encryptedData.length } + } + + async decrypt(key, nonce, size, onProgress): Promise { + // Call existing crypto function directly + return decryptChunks([this.encryptedData!], key, nonce, size, onProgress) + } + + async readChunk(offset: number, size: number): Promise { + return this.encryptedData!.slice(offset, offset + size) + } + + async writeChunk(data: Uint8Array, offset: number): Promise { + if (!this.encryptedData) this.encryptedData = new Uint8Array(offset + data.length) + this.encryptedData.set(data, offset) + } + + async cleanup(): Promise { + this.encryptedData = null + } +} +``` + +**WorkerMemoryBackend** and **WorkerOPFSBackend** are similar but post messages to a Web Worker instead of calling crypto directly. The Worker then calls the same `encryptFile()`, `decryptChunks()` functions. See §4 for Worker implementation details. + +### 3.4 Factory Function + +```typescript +// env.ts +export function createCryptoBackend(fileSize: number): CryptoBackend { + const hasWorker = typeof Worker !== 'undefined' + const hasOPFS = typeof navigator?.storage?.getDirectory !== 'undefined' + const isLargeFile = fileSize > 50 * 1024 * 1024 + + if (hasWorker && hasOPFS && isLargeFile) { + return new WorkerOPFSBackend() // Browser + large file + } else if (hasWorker) { + return new WorkerMemoryBackend() // Browser + small file + } else { + return new DirectMemoryBackend() // Node.js + } +} +``` + +### 3.5 Usage in agent.ts + +```typescript +// agent.ts - upload orchestration (simplified) +export async function uploadFile( + server: XFTPServer, + fileData: Uint8Array, + fileName: string, + onProgress?: ProgressCallback +): Promise { + // Create backend based on environment + const backend = createCryptoBackend(fileData.length) + + try { + // Encrypt (runs in Worker in browser, directly in Node) + const enc = await backend.encrypt(fileData, fileName, onProgress) + + // Upload chunks (same code regardless of backend) + const client = await connectXFTP(server) + const sentChunks = [] + let offset = 0 + for (const size of enc.chunkSizes) { + const chunk = await backend.readChunk(offset, size) + const sent = await uploadChunk(client, chunk, enc.digest) + sentChunks.push(sent) + offset += size + } + + // Build description and URI + const fd = buildFileDescription(enc, sentChunks) + return encodeFileDescriptionURI(fd) + } finally { + await backend.cleanup() + } +} +``` + +The key point: `uploadFile()` logic is identical regardless of whether crypto runs in a Worker or directly. The `CryptoBackend` abstraction hides that detail. + +### 3.6 Why This Matters for Testing + +- **Layer 1 tests** (per-function): Call `encryptFile()`, `decryptChunks()` directly via Node — unchanged +- **Layer 2 tests** (full flow): Call `uploadFile()`, `downloadFile()` in Node — uses `DirectMemoryBackend`, same code path as browser except for Worker +- **Layer 3 tests** (browser): Call `uploadFile()`, `downloadFile()` in Playwright — uses `WorkerMemoryBackend` or `WorkerOPFSBackend` + +All three layers exercise the same crypto functions. The only difference is execution context. + +## 4. Web Worker Implementation + +### 4.1 Why Web Worker + +File encryption (XSalsa20-Poly1305) is sequential and CPU-bound: +- 100 MB file ≈ 1-2 seconds of continuous computation +- Running on main thread blocks UI (no progress updates, frozen page) +- Chunking into async microtasks adds complexity and still causes jank + +Web Worker runs crypto in parallel thread. Main thread stays responsive. + +### 4.2 Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Main Thread │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ +│ │ UI (upload/ │ │ Progress │ │ Network (fetch) │ │ +│ │ download) │ │ display │ │ │ │ +│ └──────┬──────┘ └──────▲──────┘ └──────────▲──────────┘ │ +│ │ │ │ │ +│ │ postMessage │ progress │ encrypted │ +│ ▼ │ events │ chunks │ +├─────────────────────────────────────────────────────────────┤ +│ Web Worker │ +│ ┌─────────────────────────────────────────────────────────┐│ +│ │ Crypto Pipeline ││ +│ │ - encryptFile() with progress callbacks ││ +│ │ - decryptChunks() with progress callbacks ││ +│ │ - OPFS read/write for temp storage ││ +│ └─────────────────────────────────────────────────────────┘│ +└─────────────────────────────────────────────────────────────┘ +``` + +### 4.3 Message Protocol + +**Main → Worker:** + +```typescript +type WorkerRequest = + // Encrypt file, store result in OPFS (large) or memory (small) + | {type: 'encrypt', file: File, fileName: string, useOPFS: boolean} + // Read encrypted chunk from OPFS for upload + | {type: 'readChunk', offset: number, size: number} + // Write downloaded chunk to OPFS for later decryption + | {type: 'writeChunk', data: ArrayBuffer, offset: number} + // Decrypt from OPFS or provided chunks + | {type: 'decrypt', key: Uint8Array, nonce: Uint8Array, size: number, chunks?: ArrayBuffer[]} + // Delete OPFS temp files + | {type: 'cleanup'} + | {type: 'cancel'} +``` + +**Worker → Main:** + +```typescript +type WorkerResponse = + | {type: 'progress', phase: 'encrypt' | 'decrypt', done: number, total: number} + // For OPFS: encData is empty, data lives in OPFS temp file + | {type: 'encrypted', encData: ArrayBuffer | null, digest: Uint8Array, key: Uint8Array, nonce: Uint8Array, chunkSizes: number[]} + | {type: 'chunk', data: ArrayBuffer} // Response to readChunk + | {type: 'chunkWritten'} // Response to writeChunk + | {type: 'decrypted', header: FileHeader, content: ArrayBuffer} + | {type: 'cleaned'} // Response to cleanup + | {type: 'error', message: string} +``` + +### 4.4 Worker Implementation + +```typescript +// crypto.worker.ts +import {encryptFile, encryptFileStreaming, decryptChunks, decryptFromOPFS} from './crypto/file.js' +import {sha512} from './crypto/digest.js' +import {prepareChunkSizes} from './protocol/chunks.js' + +let opfsHandle: FileSystemSyncAccessHandle | null = null + +self.onmessage = async (e: MessageEvent) => { + const req = e.data + + if (req.type === 'encrypt') { + const key = crypto.getRandomValues(new Uint8Array(32)) + const nonce = crypto.getRandomValues(new Uint8Array(24)) + + if (req.useOPFS) { + // Large file: stream through OPFS to avoid memory pressure + const root = await navigator.storage.getDirectory() + const fileHandle = await root.getFileHandle('encrypted-temp', {create: true}) + opfsHandle = await fileHandle.createSyncAccessHandle() + + // Stream encrypt: read 64KB from File, encrypt, write to OPFS + const digest = await encryptFileStreaming( + req.file, + req.fileName, + key, + nonce, + opfsHandle, + (done, total) => self.postMessage({type: 'progress', phase: 'encrypt', done, total}) + ) + + const encSize = opfsHandle.getSize() + const chunkSizes = prepareChunkSizes(encSize) + + self.postMessage({ + type: 'encrypted', + encData: null, // Data in OPFS, not memory + digest, key, nonce, chunkSizes + }) + } else { + // Small file: in-memory is fine + const source = new Uint8Array(await req.file.arrayBuffer()) + const encData = encryptFile(source, req.fileName, key, nonce, (done, total) => { + self.postMessage({type: 'progress', phase: 'encrypt', done, total}) + }) + + const digest = sha512(encData) + const chunkSizes = prepareChunkSizes(encData.length) + + self.postMessage({ + type: 'encrypted', + encData: encData.buffer, + digest, key, nonce, chunkSizes + }, [encData.buffer]) + } + } + + if (req.type === 'readChunk') { + // Read chunk from OPFS for upload + const chunk = new Uint8Array(req.size) + opfsHandle!.read(chunk, {at: req.offset}) + self.postMessage({type: 'chunk', data: chunk.buffer}, [chunk.buffer]) + } + + if (req.type === 'writeChunk') { + // Write downloaded chunk to OPFS + if (!opfsHandle) { + const root = await navigator.storage.getDirectory() + const fileHandle = await root.getFileHandle('download-temp', {create: true}) + opfsHandle = await fileHandle.createSyncAccessHandle() + } + opfsHandle.write(new Uint8Array(req.data), {at: req.offset}) + self.postMessage({type: 'chunkWritten'}) + } + + if (req.type === 'decrypt') { + let result + if (req.chunks) { + // Small file: chunks provided in memory + const chunks = req.chunks.map(b => new Uint8Array(b)) + result = decryptChunks(chunks, req.key, req.nonce, req.size, (done, total) => { + self.postMessage({type: 'progress', phase: 'decrypt', done, total}) + }) + } else { + // Large file: read from OPFS + result = decryptFromOPFS(opfsHandle!, req.key, req.nonce, req.size, (done, total) => { + self.postMessage({type: 'progress', phase: 'decrypt', done, total}) + }) + } + + self.postMessage({ + type: 'decrypted', + header: result.header, + content: result.content.buffer + }, [result.content.buffer]) + } + + if (req.type === 'cleanup') { + if (opfsHandle) { + opfsHandle.close() + opfsHandle = null + } + const root = await navigator.storage.getDirectory() + try { await root.removeEntry('encrypted-temp') } catch {} + try { await root.removeEntry('download-temp') } catch {} + self.postMessage({type: 'cleaned'}) + } +} +``` + +### 4.5 Main Thread Wrapper + +```typescript +// crypto-worker.ts (main thread) +export class CryptoWorker { + private worker: Worker + private pending: Map = new Map() + private onProgress?: (done: number, total: number) => void + + constructor() { + this.worker = new Worker(new URL('./crypto.worker.js', import.meta.url), {type: 'module'}) + this.worker.onmessage = (e) => this.handleMessage(e.data) + } + + async encrypt(file: File, onProgress?: (done: number, total: number) => void): Promise { + const useOPFS = file.size > 50 * 1024 * 1024 // 50 MB threshold + return new Promise((resolve, reject) => { + this.pending.set('encrypt', {resolve, reject}) + this.onProgress = onProgress + this.worker.postMessage({type: 'encrypt', file, fileName: file.name, useOPFS}) + }) + } + + async decrypt( + chunks: Uint8Array[], + key: Uint8Array, + nonce: Uint8Array, + size: number, + onProgress?: (done: number, total: number) => void + ): Promise { + return new Promise((resolve, reject) => { + this.pending.set('decrypt', {resolve, reject}) + this.onProgress = onProgress + this.worker.postMessage({ + type: 'decrypt', + chunks: chunks.map(c => c.buffer), + key, nonce, size + }, chunks.map(c => c.buffer)) + }) + } + + private handleMessage(msg: WorkerResponse) { + if (msg.type === 'progress') { + this.onProgress?.(msg.done, msg.total) + } else if (msg.type === 'encrypted') { + this.pending.get('encrypt')?.resolve({ + encData: msg.encData ? new Uint8Array(msg.encData) : null, // null when using OPFS + digest: msg.digest, + key: msg.key, + nonce: msg.nonce, + chunkSizes: msg.chunkSizes + }) + } else if (msg.type === 'decrypted') { + this.pending.get('decrypt')?.resolve({ + header: msg.header, + content: new Uint8Array(msg.content) + }) + } else if (msg.type === 'error') { + // Reject all pending + for (const p of this.pending.values()) p.reject(new Error(msg.message)) + } + } +} +``` + +## 5. OPFS Implementation + +### 5.1 Purpose + +For files approaching 100 MB, holding encrypted data in memory while uploading creates memory pressure. OPFS provides temporary file storage: +- Write encrypted data to OPFS as it's generated +- Read chunks from OPFS for upload +- Delete after upload completes + +### 5.2 When to Use + +- Files > 50 MB: Use OPFS +- Files ≤ 50 MB: In-memory (simpler, no OPFS overhead) + +Threshold is configurable. + +### 5.3 OPFS API + +```typescript +// In Web Worker (synchronous API for performance) +const root = await navigator.storage.getDirectory() +const fileHandle = await root.getFileHandle('encrypted-temp', {create: true}) +const accessHandle = await fileHandle.createSyncAccessHandle() + +// Write encrypted chunks as they're generated +accessHandle.write(encryptedChunk, {at: offset}) + +// Read chunk for upload +const chunk = new Uint8Array(chunkSize) +accessHandle.read(chunk, {at: chunkOffset}) + +// Cleanup +accessHandle.close() +await root.removeEntry('encrypted-temp') +``` + +### 5.4 Upload Flow with OPFS + +``` +1. Main: user drops file +2. Main → Worker: {type: 'encrypt', file} +3. Worker: + - Create OPFS temp file + - Encrypt 64KB at a time, write to OPFS + - Post progress every 64KB + - Compute digest + - Return {digest, key, nonce, chunkSizes} (data stays in OPFS) +4. Main: for each chunk: + - Main → Worker: {type: 'readChunk', offset, size} + - Worker: read from OPFS, return chunk + - Main: upload chunk via fetch() +5. Main → Worker: {type: 'cleanup'} +6. Worker: delete OPFS temp file +``` + +### 5.5 Download Flow with OPFS + +``` +1. Main: parse URL, get FileDescription +2. Main: for each chunk: + - Download via fetch() + - Main → Worker: {type: 'writeChunk', data, offset} + - Worker: write to OPFS temp file +3. Main → Worker: {type: 'decrypt', key, nonce, size} +4. Worker: + - Read from OPFS + - Decrypt, verify auth tag + - Return {header, content} +5. Main: trigger browser download +6. Main → Worker: {type: 'cleanup'} +``` + +## 6. Implementation Plan + +### 6.1 Phase A: fetch() Transport + +**Goal:** Replace `node:http2` with `fetch()` in `client.ts`. All existing Node.js tests pass. + +1. Rewrite `connectXFTP()` to use fetch() for handshake +2. Rewrite `sendXFTPCommand()` to use fetch() +3. Update `createXFTPChunk`, `uploadXFTPChunk`, `downloadXFTPChunk`, etc. +4. Remove `node:http2` import +5. Run existing Haskell integration tests — must pass + +**Files:** `client.ts` + +### 6.2 Phase B: Environment Abstraction + Web Worker + +**Goal:** Add `CryptoBackend` abstraction (§3) so the same code works in Node (direct) and browser (Worker). + +1. Create `env.ts` with `CryptoBackend` interface and `createCryptoBackend()` factory (as specified in §3) +2. Implement `DirectMemoryBackend` for Node.js +3. Create `crypto.worker.ts` that imports and calls existing crypto functions +4. Implement `WorkerMemoryBackend` for browser +5. Update `agent.ts` to use `createCryptoBackend()` instead of direct crypto calls +6. Existing tests pass (now using `DirectMemoryBackend`) + +**Files:** `env.ts`, `crypto.worker.ts`, `agent.ts` + +### 6.3 Phase C: OPFS Backend + +**Goal:** Large files (>50 MB) use OPFS for temp storage in browser. + +1. Implement `WorkerOPFSBackend` — uses OPFS sync API in worker +2. Add OPFS helpers in worker: read/write to temp file +3. Factory function now returns `WorkerOPFSBackend` for large files +4. Same `agent.ts` code works — only backend implementation differs + +**Files:** `env.ts`, `crypto.worker.ts` + +### 6.4 Phase D: Browser Testing + +**Goal:** Verify everything works in real browsers. + +1. Create minimal test HTML page +2. Test upload flow in Chrome, Firefox, Safari +3. Test download flow +4. Test progress reporting +5. Test cancellation +6. Test error handling (network failure, invalid file) + +## 7. Testing Strategy + +### 7.1 Test Layers + +The `CryptoBackend` abstraction (§3) enables testing at multiple levels without code duplication: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Layer 3: Browser Integration (Playwright) │ +│ - Web Worker message passing │ +│ - OPFS read/write │ +│ - Progress UI updates │ +│ - Real browser fetch() with CORS │ +├─────────────────────────────────────────────────────────────────┤ +│ Layer 2: Full Flow (Haskell-driven, Node.js) │ +│ - fetch() transport against real xftp-server │ +│ - Upload: encrypt → chunk → upload → build description │ +│ - Download: parse → download → verify → decrypt │ +│ - Cross-language: TS upload ↔ Haskell download (and vice versa) │ +├─────────────────────────────────────────────────────────────────┤ +│ Layer 1: Per-Function (Haskell-driven, Node.js) │ +│ - 172 existing tests │ +│ - Byte-identical output vs Haskell functions │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### 7.2 Layer 1: Per-Function Tests (Existing) + +Existing Haskell-driven tests in `XFTPWebTests.hs`. Each test calls a TypeScript function via Node and compares output with Haskell. + +```bash +cabal test --ghc-options -O0 --test-option='--match=/XFTP Web Client/' +``` + +All 172 tests must pass. No changes needed for browser transport work. + +### 7.3 Layer 2: Full Flow Tests (Node.js + fetch) + +Haskell-driven integration tests using Node.js native fetch(). These test the complete upload/download flow without Worker/OPFS. + +```haskell +-- XFTPWebTests.hs (extends existing test file) +it "fetch transport: upload and download round-trip" $ do + withXFTPServer testXFTPServerConfigSNI $ \server -> do + -- TypeScript uploads via fetch(), returns URI + uri <- jsOut $ callTS "src/agent" "uploadFileTest" serverAddrHex <> testFileHex + -- TypeScript downloads via fetch() + content <- jsOut $ callTS "src/agent" "downloadFileTest" uriHex + content `shouldBe` testFileContent + +it "fetch transport: TS upload, Haskell download" $ do + withXFTPServer testXFTPServerConfigSNI $ \server -> do + uri <- jsOut $ callTS "src/agent" "uploadFileTest" serverAddrHex <> testFileHex + -- Haskell agent downloads using existing xftp CLI pattern + outPath <- withAgent 1 agentCfg initAgentServers testDB $ \a -> do + rfId <- xftpReceiveFile' a 1 uri Nothing + waitRfDone a + content <- B.readFile outPath + content `shouldBe` testFileContent +``` + +**What this tests:** +- fetch() handshake (challenge-response, TLS session binding) +- fetch() command execution (FNEW, FPUT, FGET, FACK) +- Streaming request/response bodies +- Full encrypt → upload → download → decrypt flow + +**What this doesn't test:** +- Web Worker message passing +- OPFS storage +- Browser-specific fetch() behavior (CORS preflight, etc.) + +### 7.4 Layer 3: Browser Integration Tests (Playwright) + +Playwright tests run in real browsers, testing browser-specific functionality. + +**Test infrastructure:** + +``` +xftp-web/ +├── test/ +│ ├── browser.test.ts # Playwright test file +│ └── test-server.ts # Spawns xftp-server for tests +└── test-page/ + ├── index.html # Minimal test UI + └── test-harness.ts # Exposes test functions to window +``` + +**Running browser tests:** + +```bash +cd xftp-web +npm run test:browser # Spawns xftp-server, runs Playwright +``` + +**Test cases:** + +```typescript +// test/browser.test.ts +import { test, expect } from '@playwright/test' +import { spawn } from 'child_process' + +let serverProcess: ChildProcess + +test.beforeAll(async () => { + // Spawn xftp-server with SNI cert for browser TLS + serverProcess = spawn('xftp-server', ['start', '-c', 'test-config.ini']) + await waitForServer() +}) + +test.afterAll(async () => { + serverProcess.kill() +}) + +test('small file upload/download (in-memory)', async ({ page }) => { + await page.goto('/test-page/') + + const result = await page.evaluate(async () => { + const data = new Uint8Array(1024 * 1024) // 1 MB + crypto.getRandomValues(data) + const file = new File([data], 'small.bin') + + const uri = await window.xftp.uploadFile(file) + const downloaded = await window.xftp.downloadFile(uri) + + return { + uploadedSize: data.length, + downloadedSize: downloaded.length, + match: arraysEqual(data, downloaded), + usedOPFS: window.xftp.lastUploadUsedOPFS + } + }) + + expect(result.match).toBe(true) + expect(result.usedOPFS).toBe(false) // Small file, no OPFS +}) + +test('large file upload/download (OPFS)', async ({ page }) => { + await page.goto('/test-page/') + + const result = await page.evaluate(async () => { + const data = new Uint8Array(60 * 1024 * 1024) // 60 MB + crypto.getRandomValues(data) + const file = new File([data], 'large.bin') + + const uri = await window.xftp.uploadFile(file) + const downloaded = await window.xftp.downloadFile(uri) + + return { + match: arraysEqual(data, downloaded), + usedOPFS: window.xftp.lastUploadUsedOPFS + } + }) + + expect(result.match).toBe(true) + expect(result.usedOPFS).toBe(true) // Large file, used OPFS +}) + +test('progress events fire during upload', async ({ page }) => { + await page.goto('/test-page/') + + const progressEvents = await page.evaluate(async () => { + const events: number[] = [] + const data = new Uint8Array(10 * 1024 * 1024) // 10 MB + const file = new File([data], 'progress.bin') + + await window.xftp.uploadFile(file, (done, total) => { + events.push(done / total) + }) + + return events + }) + + expect(progressEvents.length).toBeGreaterThan(1) + expect(progressEvents[progressEvents.length - 1]).toBe(1) // 100% at end +}) + +test('Web Worker keeps UI responsive', async ({ page }) => { + await page.goto('/test-page/') + + // Start upload and measure main thread responsiveness + const result = await page.evaluate(async () => { + const data = new Uint8Array(50 * 1024 * 1024) // 50 MB + const file = new File([data], 'responsive.bin') + + let frameCount = 0 + let uploadDone = false + + // Count animation frames during upload + function countFrames() { + frameCount++ + if (!uploadDone) requestAnimationFrame(countFrames) + } + requestAnimationFrame(countFrames) + + const start = performance.now() + await window.xftp.uploadFile(file) + uploadDone = true + const elapsed = performance.now() - start + + // If main thread was blocked, frameCount would be very low + const expectedFrames = (elapsed / 1000) * 30 // ~30 fps minimum + return { frameCount, expectedFrames, elapsed } + }) + + // Should maintain reasonable frame rate (Worker offloaded crypto) + expect(result.frameCount).toBeGreaterThan(result.expectedFrames * 0.5) +}) +``` + +### 7.5 Cross-Browser Matrix + +| Browser | fetch streaming | Web Worker | OPFS sync | Status | +|---------|----------------|------------|-----------|--------| +| Chrome 105+ | ✓ | ✓ | ✓ | Primary target | +| Firefox 111+ | ✓ | ✓ | ✓ | Supported | +| Safari 16.4+ | ✓ | ✓ | ✓ | Supported | +| Edge 105+ | ✓ | ✓ | ✓ | Supported (Chromium) | + +Playwright tests run against Chrome by default. CI can run against all browsers. + +### 7.6 Test Execution Summary + +| Phase | Test Layer | Command | What's Verified | +|-------|-----------|---------|-----------------| +| A | Layer 1 + 2 | `cabal test --test-option='--match=/XFTP Web Client/'` | fetch() transport, full flow | +| B | Layer 3 | `npm run test:browser` | Worker message passing, progress | +| C | Layer 3 | `npm run test:browser` | OPFS storage for large files | +| D | Layer 3 | `npm run test:browser -- --project=firefox,webkit` | Cross-browser |