JavaScript

Overview

Every AI interface you have used streams. Text appears a few words at a time, and it feels responsive even though the model is not actually faster.

That effect is built from a handful of JavaScript features most tutorials never connect: async iterators, async generators, AbortController, and backpressure. This lesson covers them in the context they exist for — moving tokens from a provider to a screen, and being able to stop cleanly when the user changes their mind.

Theory

Promises give you one value. Async iterators give you many, over time. A model response is not one value that arrives late; it is a sequence. for await (const chunk of stream) is the language feature built exactly for that shape, and it is why streaming code reads like ordinary loops.

Async generators let you make streams, not just consume them. An async function* that yields as data arrives lets you wrap, filter, or transform a provider's stream and hand your own out — which is how you insert parsing or logging without buffering the whole reply.

AbortController cancels work, it does not just ignore results. Passing a signal to fetch actually closes the connection. Skipping the result of an abandoned request instead means you keep receiving and keep paying for tokens nobody will read.

Backpressure is what stops a fast producer breaking a slow consumer. If tokens arrive faster than the browser paints or the client reads, something has to absorb the difference. Streams handle this for you; manually buffering everything into an array does not, and it fails on exactly the long responses you most wanted to stream.

The event loop is why none of this blocks. Awaiting a stream yields the thread. Your server handles other requests while a model call is in flight — which is why one machine can hold hundreds of open completions.

Visual Diagram

Picture a horizontal pipeline: Model provider → HTTP streaming response → async iterable → your transform generator → UI render. Tokens flow left to right, one small piece at a time, each arrow representing a hand-off that happens over await, not all at once. Above the whole pipeline sits an AbortController, with a line running down to every stage — calling .abort() on it doesn't just stop the UI from rendering new tokens, it propagates backward and tears down the HTTP connection itself, which is the entire point: a cancelled generation should stop consuming provider compute, not just stop being displayed.

Examples

• A chat UI that shows partial responses appearing token-by-token, using for await...of over the model SDK's stream and calling a state setter on each chunk.
• A "stop generating" button wired to controller.abort(), which both halts the visible stream and cancels the underlying request to the provider, so the user isn't billed (or doesn't bill your app) for tokens no one will read.
• A word-buffering transform generator that receives raw, uneven token chunks from a provider and yields only complete words, so a UI doesn't render half a word and then "fix" it a frame later.
• A server route that pipes a model's stream straight through to the client using ReadableStream, so the server never buffers the full response in memory before forwarding it — critical for very long generations.

Code

// A transform async generator: takes a raw, unevenly-chunked token stream
// and yields whole words, buffering partial words between chunks. This is
// the kind of small utility that sits between "provider SDK" and "UI".
async function* bufferIntoWords(
  tokens: AsyncIterable<string>,
  signal?: AbortSignal,
): AsyncGenerator<string> {
  let buffer = "";

  for await (const token of tokens) {
    if (signal?.aborted) return; // stop pulling more tokens immediately

    buffer += token;
    const words = buffer.split(/(?<=\s)/); // split, keeping trailing spaces
    buffer = words.pop() ?? ""; // last chunk may be a partial word — hold it

    for (const word of words) {
      if (signal?.aborted) return;
      yield word;
    }
  }

  if (buffer) yield buffer; // flush whatever's left when the stream ends
}

/**
 * Streams a model response into the UI, word by word, and can be
 * cancelled mid-flight. This is the shape almost every streaming chat
 * UI reduces to once you strip away the framework around it.
 */
async function streamToUI(
  prompt: string,
  onWord: (word: string) => void,
  controller: AbortController,
): Promise<{ cancelled: boolean }> {
  try {
    const response = await fetch("/api/generate", {
      method: "POST",
      body: JSON.stringify({ prompt }),
      signal: controller.signal, // ties the fetch to the abort signal
    });

    if (!response.body) throw new Error("no response body");

    // Adapt a ReadableStream<Uint8Array> into an async-iterable of strings.
    const decoder = new TextDecoder();
    async function* toStrings() {
      for await (const chunk of response.body as unknown as AsyncIterable<Uint8Array>) {
        yield decoder.decode(chunk, { stream: true });
      }
    }

    for await (const word of bufferIntoWords(toStrings(), controller.signal)) {
      onWord(word);
    }

    return { cancelled: controller.signal.aborted };
  } catch (err) {
    if (controller.signal.aborted) return { cancelled: true };
    throw err;
  }
}

// Usage: the same controller cancels the fetch, the generator loop, and
// the word-buffering transform all at once — one abort, one teardown.
const controller = new AbortController();
streamToUI("Explain backpressure", (word) => process.stdout.write(word), controller);
// Later, e.g. on a new user message or unmount:
// controller.abort();

Real Product Example

ChatGPT's stop-generating and regenerate behavior is the clearest mainstream example of this whole lesson in one UI element. While a response is streaming, ChatGPT shows a stop control; clicking it doesn't just freeze the visible text — it terminates the underlying request, and the model stops producing tokens server-side almost immediately. Sending a new message while a previous response is still streaming does the same cancellation implicitly before starting the new generation. Under the hood this requires exactly the mechanism built above: the client holds a cancellation handle (functionally an AbortController or the server-framework equivalent) tied to the in-flight streaming connection, and triggering it propagates all the way back to the provider request, not just the local render loop. The reason this matters commercially, not just architecturally, is cost: tokens are billed as they're generated, so a UI that only hides an unwanted response instead of actually cancelling it is paying full price for output nobody will ever read, on every single interrupted generation, at whatever scale the product runs.

Common Mistakes

Senior Tips

Exercises

Mini Project

Cancellable token stream. Build a small, framework-free script (Node or browser console, your choice):

1. Write a fake fakeTokenStream(text: string): AsyncGenerator<string> that splits a string into fake "tokens" (single words) and yields one every ~150ms, using await new Promise(r => setTimeout(r, 150)) between yields.
2. Consume it with for await...of, printing each token as it arrives, and confirm you see text appear incrementally rather than all at once.
3. Wire in an AbortController: pass its signal into the generator, check signal.aborted inside the loop, and call .abort() from a separate timer (e.g. after 1 second) to confirm the stream stops mid-sentence.
4. Log a message distinguishing "stream finished naturally" from "stream was cancelled" — the caller needs to be able to tell those two outcomes apart.

Resources

• MDN — for await...of: the loop construct every streaming model SDK's output is designed to be consumed with.
• MDN — AbortController: the cancellation primitive used throughout this lesson's code, and the one you'll wire into every streaming request you build from here on.
• MDN — Streams API: the lower-level ReadableStream/WritableStream machinery that backpressure and byte-level stream handling are built on.
• MDN — JavaScript execution model: the formal explanation of the event loop and job queue referenced in Theory — worth a careful read once, not a skim.