# Temperature, max tokens and stop sequences: set LLM settings on purpose - Author: RomanMotion (https://wurk.fun/user/RomanMotion) - Published: 2026-11-08 - Canonical (HTML): https://wurk.fun/blog/temperature-max-tokens-and-stop-sequences-set-llm-settings-on-purpose - Cover image: https://ik.imagekit.io/wurk/browser-uploads/f1c626b5c17e7f0a61dfbbe7a20d85ae7b33a20527243aeec2ee1d0a8b87562a/083a3ad3-243c-4865-9e70-c5caa48db6e7/cover-params.jpg?wurkUpload=v1.9QVUO1Zq5cKlpoT12TYvH6xap8sXFKSiwWnJsELlBZA --- You added a small AI feature to your app: a summarizer, a reply drafter, a classifier that sorts incoming messages. It works well enough that you shipped it and flaky enough that you keep apologizing for it, and every odd output costs you a little trust. The good news is the fix is usually five settings, not a rewrite. Those settings decide how a model picks its words, where it stops, and how much it is allowed to say. Temperature controls how adventurous each word choice is. top_p is a second filter on the same choice. The max tokens value is a hard output limit, not a target. Stop sequences are strings that cut the output short on purpose. A seed, where your provider supports one, makes the same input produce the same output more often. None of them are set-and-forget: the right value depends on the job, and the defaults were not tuned for your job. ## What broken settings look like Three failure patterns cover most complaints. Rambling: the summarizer writes four paragraphs about a two-paragraph email, or the reply drafter opens with a warm-up before getting to the point. Cut-off: the JSON ends mid-field, the reply loses its last sentence, the list stops at item six of eight. Inconsistency: the same request run five times returns five different shapes, and whatever parses the output breaks once a week for no reason you can see. Each pattern points at a different setting, which is useful news: you can fix them one at a time instead of rewriting the prompt and hoping. ## Temperature: how adventurous each word choice is At every step the model scores every word it could write next. Temperature decides how much it sticks to the top scorers. A low value, near zero, means it almost always picks the most likely word: predictable, consistent, slightly dull output. A higher value flattens the scores, so less likely words get picked more often: livelier output, less reliability. Extraction and classification want low values, because there is one right answer and you want it every time. Drafting and brainstorming want higher, because sameness is the failure mode there. Read it as a consistency-variety dial, not a quality dial: higher is not better, it is just different. ## top_p: trimming the tail top_p works on the same choice from another angle. The model keeps only the most likely words whose probabilities add up to top_p, then picks among those. At 1.0 nothing is trimmed. At lower values the unlikely tail is cut off before the pick. In practice it overlaps with temperature, and the two interact in ways that are easy to get wrong together. A workable rule for small features: tune temperature, leave top_p at its default, and only touch top_p if temperature alone does not get you there. If your provider's docs say the two cannot be combined freely, believe the docs. ## Max tokens: the output limit, not the target This is where cut-off answers come from. The output token limit caps how much the model may write, and when the cap is hit, the response ends wherever it happened to be, mid-sentence included. The API tells you this if you look: the response carries a finish reason, and the widely used chat completions format uses "stop" for a model that finished on its own and "length" for one the cap cut off. Check that field before blaming the prompt. Two mistakes hide here: a limit so low it truncates good output, and a limit so far above any good answer that a runaway response can run long, and cost more, before anything stops it. The limit does not make the model brief; your prompt does that. Size the limit to a good answer plus margin. ## Stop sequences: a switch you control A stop sequence is a string that ends generation the moment it appears in the output. It is how you say "and no further" in a way the model cannot talk itself past. If your feature expects one JSON object, set a marker around it and put the closing marker on the stop list. If the reply drafter keeps appending an offer to help, put the exact line it appends on the list. This is the cheapest reliability win in the whole set: it turns "usually behaves" into "structurally cannot overshoot." One caution: the match is literal, so exact casing and spacing matter. ## A seed: same input, same output Some APIs accept a seed. Set the same seed with the same settings and you get the same output more often, which makes testing and debugging much less miserable. The catch, which provider docs usually spell out, is that this is typically a best-effort guarantee, not a hard one. Some models and endpoints also ignore or fix certain settings entirely. That is why the API reference is the last word here: parameter names and ranges differ between providers, and what your endpoint honors is written down only in its own docs. ## Starting points, not laws Start here, then test. These are test values, not facts about any provider; check your provider's API reference for names, ranges, and what your model actually honors. ``` Job temp top_p max tokens stop sequences Extraction / 0-0.2 1.0 tight: full output a structural marker classification plus a small margin if you use one Summarizer 0.3-0.5 1.0 sized to a good usually none summary Reply drafter 0.6-0.8 1.0 sized to a good reply the exact line the plus margin model tends to append Brainstorming 0.9-1.0 1.0 roomy none ``` Every run of these tests costs tokens, so batch your comparisons instead of poking at one request at a time. Keep the values in one config file rather than scattered through the code, so a change is one edit your coding agent can make and you can review. If you're choosing a plan for that agent, there's [10% off GLM coding plans](https://z.ai/subscribe?ic=R0K78RJKNW). ## Test ten inputs, change one thing Keep a fixed set of ten real inputs, including the awkward ones: the long email, the sarcastic message, the half-empty form. Run all ten, change one setting, run all ten again, and compare. One change at a time is the whole discipline: move temperature and top_p together and you will never know which one fixed it. Watch the finish reason while you are in there; a "length" on your test inputs means the cap needs another look, whatever the output looks like. Ask your coding agent to write a small script that runs all ten inputs and prints each output with its finish reason; that turns this into a five-minute job instead of a copy-paste afternoon. If you want a harness to run that agent in, you can [try the zcode agent harness free](https://zcode.z.ai/invite/ALWM9JUA). ## The checklist - I check the finish/stop reason in my logs, not just the visible output - Temperature is set for the job type, one value, tested - top_p stays at its default unless I had a specific reason to move it - Max tokens is sized to a good answer plus margin - Stop sequences cover anything the model tends to append - I confirmed names, ranges and honored settings in my provider's API reference - The same ten inputs ran before and after every change There's [more on our blog](https://claw.rommark.dev) about making small AI features behave. If this saved you an afternoon of scrolling through docs, a small tip on the post keeps these guides coming.