Local reasoning models in a multi-agent workflow: where should the thinking go?

By the Cerevisor team. Product details checked against the source code of Cerevisor 3.9.0.

Two sets of release notes landed this week, from Ollama on 18 October and from vLLM on the same day, and a surprising share of both was about one unglamorous question: when a model thinks out loud, where does that text end up?

That question sounds cosmetic. In a workflow where several AI agents pass work to each other, it is not.

What a local reasoning model actually sends back

A reasoning model is one trained to write out its working before it answers: attempts, second guesses, corrections, then the final reply. DeepSeek R1 and QwQ 32B are familiar open-weight examples (models whose trained files anyone can download and run on their own machine), and more recent releases make it a switch rather than a fixed trait.

That working arrives in one of two ways. Either the serving engine (the program that loads the model and answers requests, such as Ollama, llama.cpp or vLLM) puts it in a separate labelled field, or the model writes it straight into the answer, wrapped between a pair of think tags, which are markers meaning “my thinking starts here” and “my thinking ends here”.

Which one happens depends on the model, the engine and its settings. In a chat window, the difference is a display choice. In a chain of agents, it decides what the next agent reads.

What Ollama and vLLM changed about thinking output this month

The Ollama release on 14 October added NVIDIA’s Nemotron 3.5 Super to its library, and the description makes clear how normal a thinking switch has become:

Four days later, Ollama v0.40.7 fixed conversations that switch between a model with thinking and one without, and its model information command now reports which parser a model uses and where its chat template comes from. A parser, here, is the part of the engine that reads the model’s raw output and separates the thinking from the answer. A chat template is the set of formatting rules that wraps each message before the model sees it.

The vLLM v0.32.0 release, 814 commits from 363 contributors, added a general-purpose reasoning parser that reads the template shipped with the model, so a model that ships one no longer needs a parser written just for it.

Where thinking text breaks a chain of AI agents

We see three distinct ways it goes wrong when a reasoning model sits inside a multi-agent workflow.

The first is handover pollution. A handover is the text one agent passes to the next. If the thinking rides along, the next agent reads false starts and rejected ideas as if they were the previous agent’s position. The run log and any summary built from it inherit the same confusion.

The second is the budget. Every request carries an output limit: the most tokens, or word pieces, the model may write in one reply. Reasoning models spend that limit on thinking first. If the limit is tight, the reply can end with nothing visible and no action taken, which from the outside looks like an agent that simply did nothing. It is the most diligent colleague in the building, still drafting the email when the office closes.

The third is silence. A model that thinks for minutes before its first visible word can look frozen to anything watching for signs of life.

None of these show up on a model’s spec sheet, and none of them show up in a single-chat demo. They appear the first time one agent’s reply becomes another agent’s input.

How Cerevisor keeps reasoning out of the agent handover

These are the rules the app follows today, checked against its source code rather than its marketing.

Thinking is removed at one point. When an agent’s reply is turned into text for anything downstream (the handover to the next agent, the run log, the final output, the screen), blocks between think tags are removed, in both spellings models commonly use. The model’s own conversation history keeps the raw reply, so a model that relies on seeing its earlier reasoning on its next turn still can.

An unfinished thought is dropped, not forwarded. If a model opens a think block and never closes it, everything from that point to the end is left out of what moves on. Losing half a thought is a better failure than passing an unfinished monologue to the next agent. In the chat that builds workflows, when this happens the person sees a short note that the model ran out of room and can simply ask again.

Thinking that arrives in its own field is shown, not kept. When a local server sends reasoning in a separate field, the app streams it to the screen for the person to read, never adds it to the agent’s answer, and treats it as a sign of life, so a long think is not mistaken for a stall.

Reasoning models get more room. For models the app recognises as spending their thinking inside the output limit, including DeepSeek R1, QwQ 32B and models whose names end in “-thinking”, the default limits are larger:

An empty reply is named for what it is. When a reply hits the limit with no visible text and no action, the app records it as thinking having used up the budget, gives the agent one retry with more room and a plain instruction to stop deliberating and act, and if it happens again marks the step “ended early” rather than “completed”. An honest “ended early” is worth more than a green tick on an empty result.

Thinking is a private draft. Show it to the person, keep it in the model's memory, and never forward it to the next agent.

Choosing a local reasoning model for agents that hand off work

The pattern underneath all of this is that a reasoning model is a good planner and judge, and an expensive formatter. Since thinking is now a switch on many local models, it can be decided per agent rather than per workflow. In Cerevisor each agent can use its own connection and model, set as described in the guide to per-agent model overrides, so the agent that plans can think hard while the agent that extracts fields or tidies output runs without it.

Two checks are worth doing before trusting any local reasoning model inside a chain. First, ask the engine which parser it uses for the model, which recent Ollama versions now report directly. Second, keep the model’s recognisable name. The app decides whether a model reasons inside its output limit from its model catalogue and its name, so a local copy saved under a custom label may be treated as an ordinary model and get the smaller limits.

What I would tell a team over coffee

Most local-model evaluations ask whether the model is smart enough. For multi-agent work, the more useful question is whether its thinking stays where it belongs. A brilliant model that forwards its scratch notes makes every agent after it slightly worse, and nobody notices until a summary quotes an idea the model had already rejected.

The next step is small. Connect a local server using the guide to OpenAI-compatible and local models, build a two-agent workflow where a reasoning model hands work to a second agent, run it once, and check that nothing from the first agent’s thinking turns up in the second agent’s work. The second agent should be building on an answer, not on a draft.