Streaming bytes, messages and recoverable progress

Prerequisites and outcome

Understand async iterators, cancellation and state reducers. Build a stream that survives arbitrary chunk boundaries and reconnects without losing authoritative state.

Mental model: three independent contracts

Transport moves bytes. A framing protocol identifies complete messages. A domain protocol defines what those messages mean. TCP, HTTP and a ReadableStream do not promise that one received chunk equals one event. SSE defines line-based framing; the application still needs validated event payloads, ordering and recovery rules.

Our event envelope contains version, run ID, event ID, sequence, timestamp, kind and payload. Sequence is local to a run. The server persists before emitting. A client cursor means “I processed through sequence N,” not “the business action was successful.” UI state comes from a snapshot and subsequent events.

Implementation and failure cases

Use TextDecoder with streaming enabled. Accumulate until the blank-line frame delimiter; support CRLF and LF, comments and multiple data lines. Validate the complete JSON after framing. Reject malformed UTF-8, oversized partial frames and a truncated nonempty final frame. This is an intentionally bounded SSE subset; it does not implement every EventSource reconnection field.

The tests split a multibyte message at every possible byte boundary. This catches errors that a happy-path TextDecoder.decode(chunk) example misses. It also tests truncation and frame budgets. Production callers must cap total lifetime, event count, snapshot size and rendered history separately from the per-frame cap.

The Node endpoint reads bounded batches from the SQLite event log and waits for the writable response's drain signal. That controls local buffering. It does not prove end-to-end flow control through a proxy. Use a durable log or an explicit overflow/resync policy rather than accumulating arbitrary messages in memory for a slow browser.

Disconnect, cancel and resume

Disconnect releases the client reader. Cancellation is an authenticated command to change the server run. Reconnect authorizes the viewer again, fetches a snapshot, and subscribes after its sequence. A terminal snapshot remains available even if the final event was never delivered. Duplicate delivery is normal; duplicate side effects are not repaired by the reducer.

Keep progress text replaceable, but preserve decisions and action receipts durably. For high-frequency streams, coalesce display updates per animation frame or short interval; never coalesce away an approval, failure or completion. Announce meaningful state changes through a polite live region rather than reading every token aloud.

Exercise and expected evidence

Inject invalid JSON, a sequence gap, duplicate events, slow consumers and a disconnect after commit. Passing behavior is explicit protocol failure or resync, bounded pending data, and a stable terminal result after reconnect. Record first useful state and completion latency separately. A buffered proxy can erase perceived progress while preserving correctness; diagnose this by comparing timestamps across boundaries.

Migration

Start with snapshot polling and deterministic state. Add streaming as a transport optimization. Preserve polling as a fallback while measuring reconnect success. Do not make stream availability a prerequisite for retrieving an accepted operation's receipt.

Executable reference and sources

Implementation · Behavioral evidence · Primary source.

Run the projects and assessment. Human review is pending.