Skip to content

fix(ai-content-moderation): moderate the OpenAI developer role - #13780

Open
shreemaan-abhishek wants to merge 3 commits into
apache:masterfrom
shreemaan-abhishek:fix/moderation-developer-role
Open

fix(ai-content-moderation): moderate the OpenAI developer role#13780
shreemaan-abhishek wants to merge 3 commits into
apache:masterfrom
shreemaan-abhishek:fix/moderation-developer-role

Conversation

@shreemaan-abhishek

Copy link
Copy Markdown
Contributor

Description

The request-side extractors in the OpenAI protocol adapters matched role == "system" exactly, so text in a developer message was never collected for moderation. developer is OpenAI's replacement for system on o1 and later models and on the Responses API, so a client can put instructions there and have them proxied to the LLM unchecked by ai-aliyun-content-moderation and ai-aws-content-moderation:

{
  "model": "gpt-5",
  "messages": [
    { "role": "developer", "content": "..." },
    { "role": "user", "content": "hi" }
  ]
}

Only hi was sent to the moderation service.

Both roles carry the system prompt, so extract_system_content in openai-chat.lua and openai-responses.lua now collects them together, and the existing system value in request_check_roles covers both. There is no new enum value and no schema change. Anthropic and Bedrock carry the system prompt in body.system and need no change; ai-lakera-guard uses extract_request_content and was never affected.

Behavior under the default request_check_roles: ["user"] is unchanged: developer follows system, so it is moderated only when system is selected.

Which issue(s) this PR fixes:

N/A

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

The request-side extractors in the OpenAI Chat and Responses adapters
matched `role == "system"` exactly, so a prompt placed in a `developer`
message - OpenAI's replacement for `system` on newer models - reached the
LLM unmoderated by ai-aliyun-content-moderation and ai-aws-content-moderation.

Both roles carry the system prompt, so extract_system_content now collects
them together and the existing `system` selector in request_check_roles
covers both. No new schema value.
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Aug 5, 2026
@shreemaan-abhishek

Copy link
Copy Markdown
Contributor Author

Heads-up for whoever merges second: this appends TEST 51/52 to t/plugin/ai-aws-content-moderation.t, and #13773 appends its own TEST 51+ to the same tail, so expect a trivial numbering conflict there.

# Conflicts:
#	t/plugin/ai-aws-content-moderation.t
@shreemaan-abhishek

Copy link
Copy Markdown
Contributor Author

Merged master now that #13773 has landed, and resolved the expected conflict in t/plugin/ai-aws-content-moderation.t (my blocks renumbered to TEST 67-70, none of #13773's touched).

Since the AWS plugin is role-selective as of #13773, the AWS coverage is now meaningful rather than incidental:

  • TEST 67/68: with request_check_roles: ["user", "tool", "system"], a harmful developer message is blocked.
  • TEST 69/70: with ["user"], developer is skipped, and grep_error_log confirms only the user text reached Comprehend.

The request_check_roles rows in the ai-aws-content-moderation docs (EN + ZH) that #13773 added now carry the same developer clause as the aliyun ones. Both moderation suites and make lint pass locally.

@shreemaan-abhishek

Copy link
Copy Markdown
Contributor Author

Added end-to-end coverage for the Responses API path in t/plugin/ai-aliyun-content-moderation.t, so the developer fix is exercised through protocol detection and the moderation plugin rather than by calling the extractor directly:

  • TEST 83/84: with request_check_roles: ["user", "tool", "system"], an input array whose developer item is harmful and whose user item is benign is blocked.
  • TEST 85/86: with the default ["user"], the same request passes through.

Reverting extract_system_content in openai-responses.lua to item.role == "system" fails TEST 84, so it guards the fix. Suite and make lint pass locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants