Stream an LLM answer to the browser with Server-Sent Events (FastAPI + fetch)
Show a model's answer word by word: a FastAPI endpoint that relays Ollama as SSE, browser code that reads it with fetch and POST, and a working Stop button.

A language model writes its answer one piece at a time. If your web page waits for the whole answer, the user stares at a spinner for ten or twenty seconds. If you stream it, the first words appear as soon as the model produces them, and the user can stop a bad answer early. This guide builds the full path: a model server, a small Python backend, and the browser code that shows the text as it arrives, with a Stop button that really stops the model.
The stack: FastAPI 0.162.0 with its built-in Server-Sent Events support, httpx 0.28.1 to call the model, and plain JavaScript in the browser. The model runs locally with Ollama, so you can try everything without an API key. Versions were checked on PyPI on 30 October 2026.
Why Server-Sent Events
Server-Sent Events (SSE) is a simple text format over a normal HTTP response with Content-Type: text/event-stream. The server writes small blocks of field: value lines, each block ends with a blank line, and the connection stays open:
event: token
data: {"text": "Light "}
event: token
data: {"text": "is "}
event: done
data: {"prompt_tokens": 12, "output_tokens": 9}
The format is defined in the HTML standard, section 9.2. It goes in one direction only, from server to browser, which is exactly what a streaming answer needs. It is plain HTTP, and hosted model APIs use it too: Anthropic's Messages API, for example, streams with SSE. WebSockets would also work, but they add a second protocol to your stack for a job that does not need it.
Why not EventSource
Browsers have a built-in SSE client, EventSource. Its constructor accepts only a URL and one option, withCredentials (see the EventSourceInit dictionary in the standard). That means:
- it always sends a GET request, so the prompt would have to go in the URL, where it ends up in server logs and browser history, and long prompts hit URL length limits;
- you cannot add headers such as
Authorization; - it reconnects automatically when the connection drops, which for a chat answer means generating the same answer again, and paying for it again.
So for chat we use fetch() with a POST body and read the response stream ourselves. It takes about fifty lines of code.
The backend: FastAPI relays Ollama as SSE
Ollama's chat endpoint, POST /api/chat, streams by default. It does not use SSE: it sends one JSON object per line, each with a piece of message.content, and a final object with "done": true and token counts (prompt_eval_count, eval_count). Our backend reads those lines and re-sends them as SSE events with three names: token, done and error.
Since version 0.135.0 FastAPI has native SSE support: you set response_class=EventSourceResponse and yield events from the endpoint.
import json
import os
from collections.abc import AsyncIterable
import httpx
from fastapi import FastAPI
from fastapi.sse import EventSourceResponse, ServerSentEvent
from pydantic import BaseModel
OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://localhost:11434")
MODEL = os.environ.get("OLLAMA_MODEL", "llama3.2")
app = FastAPI()
http = httpx.AsyncClient(timeout=httpx.Timeout(10.0, read=120.0))
class Prompt(BaseModel):
text: str
@app.post("/api/chat/stream", response_class=EventSourceResponse)
async def chat_stream(prompt: Prompt) -> AsyncIterable[ServerSentEvent]:
body = {"model": MODEL, "messages": [{"role": "user", "content": prompt.text}], "stream": True}
try:
async with http.stream("POST", f"{OLLAMA_URL}/api/chat", json=body) as upstream:
if upstream.status_code != 200:
detail = (await upstream.aread()).decode(errors="replace")[:300]
yield ServerSentEvent(data={"status": upstream.status_code, "detail": detail}, event="error")
return
async for line in upstream.aiter_lines(): # Ollama streams one JSON object per line
if not line:
continue
chunk = json.loads(line)
if chunk.get("done"):
yield ServerSentEvent(data={"prompt_tokens": chunk.get("prompt_eval_count"),
"output_tokens": chunk.get("eval_count")}, event="done")
return
text = chunk.get("message", {}).get("content", "")
if text:
yield ServerSentEvent(data={"text": text}, event="token")
except httpx.HTTPError as exc:
yield ServerSentEvent(data={"detail": type(exc).__name__}, event="error")
Points worth noticing:
- Errors travel inside the stream. Once the first byte of the SSE response has left, the HTTP status is already 200 and cannot change. A missing model (Ollama answers 404) or a lost connection must become an
errorevent that the browser understands. - The
datafield is JSON. FastAPI encodesdataas JSON, so a token that contains a newline or a quote arrives intact. If you need to send text as it is,ServerSentEventalso hasraw_data. - The read timeout is long, the connect timeout is short. A model can take many seconds before the first token, especially when it has to be loaded into memory. Failing to connect, instead, should be fast.
- FastAPI handles the boring parts. According to its documentation it sets
Cache-Control: no-cache, sendsX-Accel-Buffering: noso that Nginx does not buffer the stream, and sends a keep-alive comment every 15 seconds when nothing else is sent. The HTML standard recommends that keep-alive because some proxies close idle connections.
The same pattern works with a hosted model API: replace the Ollama call with your provider's streaming call and keep the three event names. Keep the API key on the server; the browser only ever talks to your backend.
The browser: read SSE from fetch
fetch() gives you the response body as a stream of bytes. TextDecoderStream turns it into text, and the rest is the parsing algorithm from the HTML standard, "Interpreting an event stream": split into lines (CRLF, LF or CR), ignore lines that start with a colon, collect data lines, and dispatch the event at the blank line.
export async function* readSSE(response) {
const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
let buffer = '';
let type = '', data = [], id = null;
try {
while (true) {
const { value, done } = await reader.read();
if (done) return; // an event without its final blank line is discarded, as the spec says
buffer += value;
// Split on CRLF, LF or CR; a CR at the very end may be the first half of a CRLF, so keep it.
const lines = buffer.split(/\r\n|\r(?!$)|\n/);
buffer = lines.pop();
for (const line of lines) {
if (line === '') { // blank line: dispatch the event
if (data.length) yield { event: type || 'message', data: data.join('\n'), id };
type = ''; data = [];
continue;
}
if (line.startsWith(':')) continue; // comment, e.g. keep-alive pings
const i = line.indexOf(':');
const field = i === -1 ? line : line.slice(0, i);
let val = i === -1 ? '' : line.slice(i + 1);
if (val.startsWith(' ')) val = val.slice(1);
if (field === 'data') data.push(val);
else if (field === 'event') type = val;
else if (field === 'id' && !val.includes('\0')) id = val;
}
}
} finally {
reader.releaseLock();
}
}
The detail that most hand-written parsers miss: network chunks do not respect event boundaries. One chunk can end in the middle of a line, or between the \r and the \n of a line ending. That is why the code keeps the last, incomplete line in buffer and does not split on a \r at the very end. TextDecoderStream solves the same problem one level down, for multi-byte UTF-8 characters split across chunks.
Then a small function sends the prompt and turns events into callbacks:
export async function streamChat(url, prompt, { onToken, signal } = {}) {
const res = await fetch(url, {
method: 'POST',
headers: { 'content-type': 'application/json', accept: 'text/event-stream' },
body: JSON.stringify({ text: prompt }),
signal,
});
if (!res.ok) throw new Error(`HTTP ${res.status}`);
for await (const ev of readSSE(res)) {
const payload = JSON.parse(ev.data);
if (ev.event === 'token') onToken?.(payload.text);
else if (ev.event === 'error') throw new Error(`model error: ${JSON.stringify(payload)}`);
else if (ev.event === 'done') return payload; // token counts reported by the model server
}
throw new Error('stream ended before the "done" event');
}
Treat a stream that ends without done as an error. Proxies, load balancers and server restarts can cut a response in the middle, and without that check a truncated answer looks like a complete one.
Showing the text and the Stop button
In the page, append each piece as text, not as HTML:
const controller = new AbortController();
stop.addEventListener('click', () => controller.abort());
const usage = await streamChat('/api/chat/stream', promptText, {
onToken: (t) => out.append(t), // text node: never innerHTML with model output
signal: controller.signal,
});
element.append(string) creates a text node, so a model that writes <img src=x onerror=...> shows those characters instead of running them. If you render Markdown, run the output through a sanitizer first. A model's output is untrusted input, like anything a user types.
When the user presses Stop, controller.abort() cancels the fetch. The browser closes the connection, Uvicorn notices the disconnect, the endpoint's generator is cancelled, and the async with block closes the connection to Ollama. A local model stops computing; with a paid API you stop paying for tokens nobody will read. Our test checks this whole chain: after the abort, the fake model server sees its client go away before the answer is finished.
Test it without a model
The example folder contains fake_ollama.py, a 65-line stand-in for Ollama's streaming endpoint written with the Python standard library. ./test_stream.sh starts it with two copies of the backend and runs the same sse-client.js used by the page, this time in Node.js. It checks four things:
- the parser handles events split across chunks, a CRLF split between two chunks, comments and multi-line
data; - the full answer arrives in order, followed by the token counts;
- an unknown model (404 from Ollama) reaches the browser as an
errorevent; - aborting after the first token closes the connection to the model server.
stream-sse OK { prompt_tokens: 12, output_tokens: 9 } { completed: 1, client_closed: 1 }
The same test runs in GitHub Actions on every change. To try it with a real model: ollama pull llama3.2, then uvicorn server:app --port 8000, and open http://localhost:8000/.
Things that break streaming in production
The code above works on your laptop. In production, something between the browser and your server often collects the whole response before passing it on, and the answer appears all at once at the end. Check these in order:
- Reverse proxies. Nginx buffers proxied responses unless told otherwise. Its
proxy_bufferingdocumentation says theX-Accel-Buffering: noresponse header turns buffering off, and FastAPI sends it; other proxies and CDNs have their own settings for buffering and response timeouts. - Compression. A compression layer may wait for enough bytes before sending a compressed block. If tokens arrive in bursts, disable compression for
text/event-streamresponses and test again. - Idle timeouts. Load balancers close connections that stay silent for too long. The 15-second keep-alive comment helps, but a model that thinks for a long time before the first token can still hit a short timeout.
- Connection limits on HTTP/1.1. MDN notes that without HTTP/2 browsers allow only six open connections per domain, shared by all tabs. A user with several chat tabs open can block their own requests. Serve the app over HTTP/2 or close streams when they finish.
- CORS. If the page and the API are on different origins, the POST with a JSON body triggers a preflight request, and the API must answer it. Our HTTP request tester sends a request from your browser and explains CORS errors in plain language. For the status codes you may see on the way, see the HTTP status codes reference.
Limits of this approach
- No automatic resume. If the connection drops in the middle, the answer is lost and the user must ask again. FastAPI's documentation shows how to resume with event IDs and the
Last-Event-IDheader, but for model output that only works if the server stores the generated text. - Long-lived connections. Each answer keeps two connections open for its whole duration (browser to backend, backend to model). Size connection limits, worker counts and proxy timeouts for the number of answers you stream at the same time, not for requests per second.
- Rate limits still apply. A streaming call can fail with 429 before the first token like any other call; our guide on HTTP 429 and Retry-After shows how to retry it.
The complete, tested example for this article is in our public repository: adamant-solutions/ai-dev-examples/stream-sse