Skip to content

fix(streaming): keep chat moderation results in the accumulated snapshot - #3808

Open
hsusul wants to merge 1 commit into
openai:mainfrom
hsusul:fix/stream-snapshot-moderation
Open

fix(streaming): keep chat moderation results in the accumulated snapshot#3808
hsusul wants to merge 1 commit into
openai:mainfrom
hsusul:fix/stream-snapshot-moderation

Conversation

@hsusul

@hsusul hsusul commented Sep 6, 2026

Copy link
Copy Markdown
  • I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Component: src/openai/lib/streaming/chat/_completions.pyChatCompletionStreamState._accumulate_chunk.

Problem. ChatCompletionChunk and ChatCompletion both carry a moderation field, and its docstring says it is "Present on the moderation chunk when moderated completions are requested" — i.e. it arrives on its own chunk, after the content chunks. The streaming accumulator carries usage and system_fingerprint forward from every chunk but never reads moderation, so the field is silently dropped from the accumulated completion.

The result is that client.chat.completions.stream(...) (and ChatCompletionStreamState used directly) reports moderation=None for a stream that actually delivered moderation results, while the equivalent non-streaming client.chat.completions.create(...) response keeps them. Callers that gate on moderation therefore see "no moderation ran" only when they stream.

moderation was added by 87e46c25 feat(api): responses.moderation and chat_completions.moderation, which updated the generated types, resources and tests but did not touch this hand-written accumulator.

Repro (no key, no network — the accumulator is fed synthetic chunks):

from openai._compat import model_parse
from openai.types.chat import ChatCompletionChunk
from openai.lib.streaming.chat import ChatCompletionStreamState

RESULT = {
    "categories": {"violence": False},
    "category_applied_input_types": {"violence": ["text"]},
    "category_scores": {"violence": 0.01},
    "flagged": False,
    "model": "omni-moderation-latest",
    "type": "moderation_result",
}
MODERATION = {
    "input": {"type": "moderation_results", "model": "omni-moderation-latest", "results": [RESULT]},
    "output": {"type": "moderation_results", "model": "omni-moderation-latest", "results": [RESULT]},
}

def chunk(**extra):
    return model_parse(ChatCompletionChunk, {
        "id": "chatcmpl-test", "object": "chat.completion.chunk",
        "created": 0, "model": "gpt-test", "choices": [], **extra,
    })

state = ChatCompletionStreamState()
state.handle_chunk(chunk(choices=[{"index": 0, "delta": {"role": "assistant", "content": "Hi"},
                                   "finish_reason": "stop", "logprobs": None}]))
state.handle_chunk(chunk(moderation=MODERATION, system_fingerprint="fp_x",
                         usage={"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}))

completion = state.get_final_completion()
print(completion.system_fingerprint)  # 'fp_x'      -> carried
print(completion.usage is not None)   # True        -> carried
print(completion.moderation)          # None        -> dropped

Before / after

stream shape before after
moderation on the first chunk preserved preserved (unchanged)
moderation on a later, dedicated moderation chunk None preserved
moderation followed by a chunk without moderation preserved preserved (unchanged)
no moderation anywhere in the stream None None (unchanged)

The first-chunk case already worked by accident: _convert_initial_chunk_into_snapshot splats the whole first chunk into the snapshot, so moderation survived only when it happened to be on chunk 1.

Root cause. _accumulate_chunk ends with

completion_snapshot.usage = chunk.usage
completion_snapshot.system_fingerprint = chunk.system_fingerprint

moderation is the one completion-level field that a later chunk can introduce and that is not in that list.

The fix / why it is minimal. Three lines next to the existing carry-forwards:

if chunk.moderation is not None:
    completion_snapshot.moderation = cast(
        Moderation,
        construct_type(type_=Moderation, value=chunk.moderation.to_dict()),
    )
  • Guarded by is not None rather than assigned unconditionally like usage/system_fingerprint, because the moderation chunk is not necessarily the last chunk; an unconditional assignment would clear a moderation payload that arrived earlier.
  • Routed through construct_type because chat_completion_chunk.Moderation and chat_completion.Moderation are separate generated models with the same shape. Assigning the chunk instance straight onto the snapshot field stores the wrong class and makes the snapshot unserializable (TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer' from to_dict()); one of the added tests asserts completion.to_dict()["moderation"] round-trips.
  • No change to usage / system_fingerprint behavior, to event emission, ordering, parsing, or any public signature. No other chunk field is touched.

Why this is hand-maintained, not generated code. src/openai/lib/streaming/chat/_completions.py has no File generated from our OpenAPI spec header and lives under src/openai/lib/. The generated types (types/chat/chat_completion.py, types/chat/chat_completion_chunk.py) are already correct and are not modified here — the defect is only in the SDK-owned accumulator that has to bridge them.

Tests (tests/lib/chat/test_completions_streaming.py, following the existing test_stream_obfuscation_stays_on_raw_chunks pattern — synthetic chunks, no HTTP, no key):

  • test_stream_snapshot_keeps_moderation_from_a_later_chunk — the regression; fails on main.
  • test_stream_snapshot_keeps_moderation_from_the_first_chunk — the path that already worked stays working.
  • test_stream_snapshot_moderation_survives_a_later_chunk_without_moderation — guards against an unconditional overwrite.
  • test_stream_snapshot_has_no_moderation_when_the_stream_has_none — no field invented when the stream has none.

Modes covered. ChatCompletionStreamState is the single accumulator behind both ChatCompletionStream and AsyncChatCompletionStream, so sync and async share this code path and this fix; the tests drive the state object directly rather than duplicating an async wrapper that would exercise the same lines. Non-streaming parse/create are unaffected. No stream lifecycle, context-manager, or close behavior is involved.

Additional context & links

Validation (macOS, Python 3.10.16 from uv, branch at 3884855):

  • ./scripts/lintpassed (Ruff All checks passed!, Pyright 0 errors, 0 warnings, 0 informations, mypy Success: no issues found in 1614 source files, import openai ok).
  • ./scripts/formatpassed, no changes to the two files in this PR.
  • .venv/bin/python -m pytest tests/lib/chat/test_completions_streaming.py -k moderation -q -p no:xdist -o addopts=""1 failed, 3 passed with the source change reverted (AssertionError: assert None is not None on completion.moderation), 4 passed with it applied.
  • ./scripts/test tests --ignore=tests/test_uv_workflows.py --ignore=tests/test_upload_examples.py9366 passed, 144 skipped (Pydantic v2) and 9352 passed, 158 skipped (Pydantic v1).

Limits, stated plainly:

  • tests/test_uv_workflows.py and tests/test_upload_examples.py were excluded from the full run because they fail in my checkout for environment reasons, not because of this change: test_uv_workflows.py shells out to git remote get-url origin from a temp cwd and gets fatal: not a git repository, and test_upload_examples.py times out generating large fixture files. I confirmed the identical failures with this commit's source change reverted, so they are pre-existing here.
  • No live API calls were made and no API key was used; every assertion in the new tests is driven by locally constructed chunks.
  • I did not run the Node-tooling test suite (*.test.cjs) — it is unrelated to this change.

`ChatCompletionStreamState._accumulate_chunk` carried `usage` and
`system_fingerprint` forward from each chunk but ignored `moderation`.
Moderation results arrive on their own chunk after the content chunks, so
`get_final_completion()` and `current_completion_snapshot` reported
`moderation=None` for streams that requested moderated completions, while
the equivalent non-streaming response kept the field.

Carry `moderation` into the snapshot when a chunk provides it. It is copied
through `construct_type` because `ChatCompletionChunk.Moderation` and
`ChatCompletion.Moderation` are distinct models, and it is only applied when
present so a later chunk without moderation cannot clear it.
@hsusul
hsusul requested a review from a team as a code owner September 6, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant