Eye Training · Streams and Streaming Data

Data arrives in pieces, at its own pace, and may stop half-way. Every streaming bug forgets one of those three.

Part 2 · Surface 15. Bug counts are hidden and some snippets are clean. See Start Here for how to use these drills.

The snippets use Node streams (pipe, pipeline, write), the web ReadableStream that fetch returns, server-sent events, and LLM token streams. The checks are the same for all of them.

J · The basics

M · Three years in, these should jump out

S · Where seniors stand out

What your eye checks

Whenever data arrives in chunks, ask:

1. Is the whole thing there yet? Data events, read() and tokens arrive later; parse at end or done, not before.
2. Do chunk boundaries matter? Lines, JSON records and multi-byte characters can be split across chunks. Buffer the partial piece, and decode with { stream: true } or a StringDecoder.
3. Who handles errors? pipe doesn't forward them; pipeline does. Every stream needs an error path.
4. Is there backpressure? If write() returns false, wait for drain. Never load a whole table or file into memory to send it.
5. What happens when the client leaves? Listeners removed, upstream work (DB cursors, LLM calls) aborted, using res.on('close').
6. Is each piece a delta or the whole value? Append deltas; replace snapshots.
7. What does the stream end with? A [DONE] marker, a final partial line, a _flush.
8. What if it fails half-way? Headers and a 200 are already sent. Can the client tell the data is incomplete?
9. Can two streams write into the same place? A shared "current" target gets interleaved tokens.

Tells

Go deeper

• Streams and Async Iterators: backpressure, pipeline, and async iteration in depth
• Cancellation and Cleanup: Part 9, cancelling LLM calls so you stop paying
• Previous surface: Events, listeners and sockets · Next: fetch and the network