Skip to content

fix(ai): track failed OpenAI Responses stream events - #1261

Open
Artyfowl1710 wants to merge 1 commit into
PostHog:mainfrom
Artyfowl1710:codex/openai-failed-response-telemetry
Open

Artyfowl1710 wants to merge 1 commit into
PostHog:mainfrom
Artyfowl1710:codex/openai-failed-response-telemetry

Conversation

@Artyfowl1710

Copy link
Copy Markdown

💡 Motivation and Context

An OpenAI Responses stream can return response.failed as a terminal event over a successful HTTP connection. The Python SDK forwards that event to the caller but currently records the generation without $ai_is_error. Dashboards that use this field miss these failures.

The sync and async Responses wrappers now record the terminal event through the existing streaming error path and continue forwarding it to callers. The existing privacy-mode formatter redacts the provider message. Exceptions raised during iteration retain their existing precedence.

💚 How did you test it?

  • Added sync and async regression tests using the OpenAI SDK with httpx.MockTransport and an HTTP 200 SSE response. The tests verify the event delivered to callers, generation failure metadata, and provider error redaction in privacy mode.
  • Ran pytest -q -p no:cacheprovider --timeout=30 posthog/test/ai/openai posthog/test/ai/test_stream_midstream_error.py -k "openai or capture_streaming_event" with OpenAI 2.54.0 and Python 3.12 on Ubuntu. The result was 144 passed, 3 skipped, and 11 deselected.
  • Ruff 0.12.2 lint and format checks passed for the changed Python files.

📝 Checklist

  • All commits are signed.
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change.

If releasing new changes

A patch changeset is included for pypi/posthog.

🤖 Agent context

Autonomy: Fully autonomous

OpenAI Codex identified the gap, prepared the change, and ran pytest and Ruff via the shell. The regression tests use the OpenAI SDK's streaming HTTP parser and cover both wrappers. The implementation uses the existing capture and redaction logic.

Prepared with OpenAI Codex assistance.
@Artyfowl1710
Artyfowl1710 requested a review from a team as a code owner November 16, 2026 13:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant