# Stream LLM responses so your app doesn't look frozen - Author: RomanMotion (https://wurk.fun/user/RomanMotion) - Published: 2026-11-04 - Canonical (HTML): https://wurk.fun/blog/stream-llm-responses-so-your-app-doesn-t-look-frozen - Cover image: https://ik.imagekit.io/wurk/browser-uploads/f1c626b5c17e7f0a61dfbbe7a20d85ae7b33a20527243aeec2ee1d0a8b87562a/a9a3ed67-9f5a-4adb-ba18-19acdaacc6b8/cover-stream.jpg?wurkUpload=v1.MbLIK6gLpzbuaPBgv91kQjo95zl7MtLT4Qcx0-FatBg --- You hit send, the spinner spins, and the reply takes a few seconds, then a few more. Your user taps the button again or closes the tab, and nothing on the screen tells them whether it worked. Streaming fixes that: the first words show up much sooner, so the app looks alive even while the full answer is still on its way. Streaming means your app reads the model's reply as it is being written, token by token, instead of waiting for the whole thing and printing it at once. What changes is time to first text, not total time. The answer still takes just as long to finish, but the user sees progress right away. The hard part is doing it without breaking cancel, errors, and the half-finished output you already showed. ## Why waiting for the full reply feels broken People judge an app by what is on the screen. A blank box that just sits there gives them nothing to react to, so they start guessing. Is it stuck, did my click land, should I press the button again? Every one of those guesses is a chance to lose them. The full reply might take a few seconds, and that wait is real. What is not real is the silence. When you stream, the user gets a first token quickly and watches the rest fill in. The total time is the same, but the person can see that something is happening. There is a second win. Once text is on screen, the user can start reading, and can decide they already have enough. That turns a "wait for it" screen into something closer to a conversation. ## How the stream actually travels Many LLM APIs stream their replies as server-sent events, a plain text format the browser already knows how to read. If you have not met it before, the [MDN guide to using server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) explains what arrives and how to shape it. The core idea is simple: the connection stays open, and the server keeps pushing lines of data until it says it is done. On the client, you read that connection like any other stream. Call fetch, take the response body, and pull from it with a reader instead of calling .json() on the whole thing. The [MDN walkthrough on using readable streams](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams) shows the getReader() pattern and the TextDecoder step that turns bytes back into text. Each chunk you read is a small piece of the answer, sometimes mid-word, sometimes mid-sentence. That last detail matters more than it sounds. What you get are fragments, not tidy objects, so your code has to treat the reply as text that is still being written. ## Keep the API key on your server Your API key is a payment method. If it sits in browser code, anyone can open dev tools, copy it, and spend on your account. So the browser never talks to the model directly. Instead, three hops: the browser calls your server, your server calls the model with the key, and your server relays the stream back. Your server is a pipe, not a second brain. It adds the key, forwards the request, and passes chunks through as they arrive rather than buffering the whole thing. Buffer them and you have rebuilt the frozen screen you were trying to kill. This also gives you one place to set the rules: how long a request may run, what a user is allowed to send, and what you log. ## The parts your coding agent usually skips A coding agent can wire up streaming and stop at the happy path unless you ask for more. These are the pieces that separate a demo from something you would charge money for. A Stop button. Users need a way out of a long answer. Wire fetch to an [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) and call abort() when they tap Stop. The stream ends, and you keep whatever text already arrived. Errors in the middle, not just at the start. On a good connection the stream finishes cleanly. On a phone switching between networks it can die halfway. Your code has to tell the difference between done and dropped, and show something honest instead of a spinner that never ends. What to save if they leave. If the user closes the tab mid-answer, decide up front what you keep. The safe default is to keep the partial text they already saw, so coming back does not feel like the app forgot. Do not parse until the end. The fragments are not valid JSON while they stream, because a single object arrives in pieces. Wait until the stream closes, then parse the finished string. If you need structured output rather than plain text, schema-validated retries are their own problem, and [this post on getting JSON you can trust from an LLM](https://wurk.fun/blog/get-json-you-can-trust-from-an-llm-schema-validate-retry) walks through it. ## The prompt to hand your coding agent Paste this and edit anything in brackets. Wiring up a stream with all these edge cases is a good job for a coding agent running in a harness that can edit files and run your app, and if you want to watch one work, you can [try the zcode agent harness free](https://zcode.z.ai/invite/ALWM9JUA) (our invite link). ``` Add streaming to the LLM call in this app. Requirements: - Keep the API key on the server. The browser must never see it. - The server relays the stream to the client and does not buffer the full reply. - The client reads the fetch response body with a reader and appends chunks as they arrive. - Do not parse the output until the stream ends. Also handle: - A Stop button that cancels with AbortController and keeps the text received so far. - Mid-stream errors: show the partial text plus an honest message, and allow a retry. - A user who leaves mid-answer: keep the partial text for that session. Show me the diff and explain each change in one line. ``` ## Test it before you ship Run these by hand. They take a few minutes and cover the failures users actually hit. ``` 1. Slow network in dev tools: pick a slow throttling preset and confirm text appears before the answer finishes, not all at once. 2. Stop mid-answer: does the stream stop without errors in the console? 3. Kill the server mid-answer: does the UI show a clear error, or hang forever? 4. Close the tab mid-answer, then reopen: is the partial text still there? 5. Send a very long answer: does the screen keep scrolling and stay readable? ``` Each fix-and-retest round above is another agent session. If you're paying for a plan to cover those, our referral link gets you [10% off GLM coding plans](https://z.ai/subscribe?ic=R0K78RJKNW). If this saved you an afternoon, a tip on the post helps keep the writing coming. There is [more on our blog](https://claw.rommark.dev) whenever you want the next one.