qwen3 reasoning parser: streamed content starts with "\n\n" after </think> and streamed reasoning keeps its trailing newline; non-streaming strips both #712

What happened

With thinking on (--reasoning-parser qwen3), the model writes …reasoning\n</think>\n\n<answer or tool call>. Non-streaming strips the whitespace on both sides of </think>. Streaming does not, so the same turn looks different on /v1/chat/completions depending on stream:

So a thinking-on streamed answer starts with a blank line, and a tool-call-only turn sends a whitespace-only content delta. A client that creates a text part when the first content delta arrives therefore shows an empty assistant message before each tool call. The /v1/messages and /v1/responses streams go through the same path (generate_events), so their text blocks get the same "\n\n".

Minimal reproduction (CPU only, PYTHONPATH=python):

from freetoken.server.reasoning_parser import ReasoningParser
from freetoken.server.function_call_parser import FunctionCallParser
from freetoken.server.api_models import Tool

toks = ["Think", ".", "\n", "</think>", "\n\n", "Answer", "."]
p = ReasoningParser("qwen3", force_reasoning=True)
d = [p.parse_stream_chunk(t) for t in toks] + [p.flush()]
print("stream:     reasoning=%r content=%r" % ("".join(r for r, _ in d), "".join(c for _, c in d)))
r, c = ReasoningParser("qwen3", force_reasoning=True).parse_non_stream("".join(toks))
print("non-stream: reasoning=%r content=%r" % (r.strip(), c))   # .strip(): generation.py:823

# tool-call-only turn, streamed through the qwen3_coder detector as generate_events does
tools = [Tool(type="function", function={"name": "ls", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}}})]
toks = ["Plan", "\n", "</think>", "\n\n", "<tool_call>", "\n<function=ls>\n<parameter=path>\n.\n</parameter>\n</function>\n</tool_call>"]
rp, fp, text = ReasoningParser("qwen3", force_reasoning=True), FunctionCallParser(tools, "qwen3_coder"), []
for t in toks:
    _, c = rp.parse_stream_chunk(t)
    if c: text += [e[1] for e in fp.parse_stream_events(c) if e[0] == "text"]
print("stream content deltas before the call:", text)
_, c = ReasoningParser("qwen3", force_reasoning=True).parse_non_stream("".join(toks))
print("non-stream content:", repr(FunctionCallParser(tools, "qwen3_coder").parse_non_stream(c).normal_text))

Output on main 9b68e00:

stream:     reasoning='Think.\n' content='\n\nAnswer.'
non-stream: reasoning='Think.' content='Answer.'
stream content deltas before the call: ['\n\n']
non-stream content: ''

Cause (current main, 9b68e00)

• python/freetoken/server/reasoning_parser.py:134 and :151-153: BaseReasoningParser.detect_and_parse (non-streaming) strips the whole text and then normal_text.strip().
• python/freetoken/server/reasoning_parser.py:176-183: parse_streaming_increment returns the text after </think> unstripped. Its rstrip() applies only to reasoning in the same chunk as </think>. When the "\n" arrives in an earlier delta than </think> (the usual case, since </think> is a single added token), it has already been streamed as reasoning by then.
• python/freetoken/server/generation.py:823: the non-streaming result also .strip()s the reasoning. The streaming loop (:706-723) forwards the deltas as they are. In the tool path, suppress_ws (:667) only drops whitespace after a call has closed, not before the first one. openai_api.py:285 sends each ContentDelta as is.

The code is in BaseReasoningParser, so the qwen3, glm, minimax and deepseekv32 parsers all inherit it.

Proposed fix

Make the streaming path follow the non-streaming one in BaseReasoningParser:

• After </think>, drop leading whitespace from content until the first non-whitespace character. Flush a whitespace-only remainder at the end as nothing.
• While reasoning, hold back a trailing run of whitespace instead of streaming it. Emit it if more reasoning follows. Drop it when </think> closes the block.

Both are small, bounded holds, so streaming latency does not change. A stream-vs-non-stream test over the same token sequence (text answer and tool-call-only turn) would keep the two paths in line.

Environment

Seen serving nvidia/Qwen3.8-Flash-Next-NVFP4 on an RTX 5080 16 GB; the reproduction above is CPU-only on main 9b68e00.

Related: #711 (same parser, re-entry on a later <think>), #719 (chat logprobs design, which depends on both modes agreeing).