RHIDP-14083: extend customization profile for safety shield prompts - #2374
RHIDP-14083: extend customization profile for safety shield prompts#2374Jdubrick wants to merge 3 commits into
Conversation
WalkthroughQuestion-validity fields are now nullable and resolve from custom profiles or built-in defaults. ChangesQuestion-validity resolution
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CustomProfile
participant Configuration
participant QuestionValidity
participant Moderation
CustomProfile->>Configuration: Load validation and invalid-response values
Configuration->>QuestionValidity: Pass resolved prompt configuration
QuestionValidity->>Moderation: Submit rendered agent input or raw responses input
Moderation-->>QuestionValidity: Return validation result
Suggested reviewers: 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/models/successful_responses.md`:
- Around line 576-577: Update the PROFILE_CONFIG references in the validation
and invalid_resp table entries to use Markdown-safe subscript notation, such as
double-quoted keys or escaped brackets, so markdownlint MD052 passes while
preserving the documented configuration paths.
In `@tests/unit/app/endpoints/test_shields.py`:
- Line 106: Update the test covering the changed model_prompt in the endpoint
response to assert that the returned prompt includes the configured “Is this
question valid? ${message}” value, alongside the existing model_id assertion.
Anchor the assertion to the response validation in the test so it verifies the
endpoint preserves the updated prompt.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6d84467a-998d-43ab-bb2c-0c201d99ff99
📒 Files selected for processing (19)
docs/devel_doc/openapi.jsondocs/models/successful_responses.jsondocs/models/successful_responses.mddocs/user_doc/config.mddocs/user_doc/shields_guide.mdexamples/lightspeed-stack-shields.yamlsrc/models/api/responses/successful/catalog.pysrc/models/config.pysrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pytests/profiles/empty_qv_strings/profile.pytests/profiles/no_qv_keys/profile.pytests/profiles/test/profile.pytests/unit/app/endpoints/test_shields.pytests/unit/models/responses/test_successful_responses.pytests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.pytests/unit/qv_config.pytests/unit/test_configuration.pytests/unit/utils/test_prompts.pytests/unit/utils/test_pydantic_ai.py
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
- GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-7-on-pull-request
⚠️ CI failures not shown inline (1)
GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-8-on-pull-request: Failed
Conclusion: failure
Konflux kflux-prd-rh02/lightspeed-stack-0-8-on-pull-request has <b>failed</b>.
<ul>
<li><b>Namespace</b>: <a href="https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant">lightspeed-core-tenant</a></li>
<li><b>PipelineRun:</b> <a href="https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-24kll">lightspeed-stack-0-8-on-pull-request-24kll</a></li>
</ul>
<hr>
<h4>Task Statuses:</h4>
<table>
<tr><th>Status</th><th>Duration</th><th>Name</th></tr>
<tr>
<td>🟢 Succeeded</td>
<td>6 seconds</td><td>
[init](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-24kll/logs/init)
</td></tr>
<tr>
<td>🟢 Succeeded</td>
<td>16 seconds</td><td>
[clone-repository](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-24kll/logs/clone-repository)
</td></tr>
<tr>
<td>🟢 Succeeded</td>
<td>10 seconds</td><td>
[prefetch-dependencies](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-24kll/logs/prefetch-dependencies)
</td></tr>
<tr>
<td>🔴 Failed</td>
<td>12 seconds</td><td>
[build-images](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-24kll/logs/build-images)
</td></tr>
</table>
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (Custom checks)
**/*: Flag meaningful O(n^2)+ algorithms on non-trivial inputs, including handlers and Kubernetes list operations.
Flag N+1 patterns that list items and then query once per item, including Kubernetes API and database access.
Flag expensive work inside loops, including API calls, JSON parsing, and regex compilation.
Flag unbounded growth in caches, watchers, or buffers when eviction or limits are missing.
Flag missing pagination or limits on list operations and API endpoints.
Flag secrets or tokens logged in plaintext or hardcoded in source.
Flag API endpoints missing authentication or authorization.
Flag injection vulnerabilities, including SQL injection, command injection, and path traversal.
Flag sensitive data leaked in API responses, WebSocket messages, or logs.
Flag Kubernetes Secrets and Red Hat secrets missing OwnerReferences.Never commit secrets or keys; use environment variables for sensitive data.
Files:
tests/profiles/no_qv_keys/profile.pytests/profiles/empty_qv_strings/profile.pytests/unit/qv_config.pydocs/user_doc/config.mdsrc/models/api/responses/successful/catalog.pyexamples/lightspeed-stack-shields.yamltests/profiles/test/profile.pydocs/models/successful_responses.jsontests/unit/utils/test_pydantic_ai.pydocs/user_doc/shields_guide.mdsrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pytests/unit/app/endpoints/test_shields.pydocs/models/successful_responses.mddocs/devel_doc/openapi.jsontests/unit/test_configuration.pysrc/models/config.pytests/unit/utils/test_prompts.pytests/unit/models/responses/test_successful_responses.pytests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Use absolute imports for internal Python modules.
Every module must begin with a descriptive docstring explaining its purpose.
Uselogger = get_logger(__name__)fromlog.pyfor module logging.
Define shared constants in the centralconstants.pymodule and annotate constants withFinal[type].
All functions must have complete parameter and return type annotations and descriptive docstrings.
Use snake_case, descriptive, action-oriented names for functions, such asget_,validate_, andcheck_.
Avoid modifying mutable parameters in place; return a new data structure instead.
Useasync deffor I/O operations and external API calls.
HandleAPIConnectionErrorfrom Llama Stack integrations.
Use standard logger levels appropriately:debugfor diagnostics,infofor general execution,warningfor unexpected or potentially problematic conditions, anderrorfor serious failures.
All classes must have descriptive docstrings, use PascalCase names, and provide complete, specific type annotations for class attributes.
Use Google Python docstring conventions, includingParameters,Returns,Raises, andAttributessections when applicable.
Files:
tests/profiles/no_qv_keys/profile.pytests/profiles/empty_qv_strings/profile.pytests/unit/qv_config.pysrc/models/api/responses/successful/catalog.pytests/profiles/test/profile.pytests/unit/utils/test_pydantic_ai.pysrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pytests/unit/app/endpoints/test_shields.pytests/unit/test_configuration.pysrc/models/config.pytests/unit/utils/test_prompts.pytests/unit/models/responses/test_successful_responses.pytests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.py
tests/unit/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
tests/unit/**/*.py: Use pytest for unit tests; do not use unittest.
Useconftest.pyfor shared fixtures,pytest-mockfor mocks, andpytest.mark.asynciofor asynchronous tests.
Files:
tests/unit/qv_config.pytests/unit/utils/test_pydantic_ai.pytests/unit/app/endpoints/test_shields.pytests/unit/test_configuration.pytests/unit/utils/test_prompts.pytests/unit/models/responses/test_successful_responses.pytests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.py
src/models/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/models/**/*.py: Pydantic configuration models must extendConfigurationBase; data models must extendBaseModel.
Use@field_validatorand@model_validatorfor Pydantic model validation; model validators should usetyping_extensions.Selfwhere applicable.
Files:
src/models/api/responses/successful/catalog.pysrc/models/config.py
🧠 Learnings (6)
📚 Learning: 2026-06-24T13:45:37.249Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 1971
File: src/utils/markdown_repair.py:31-36
Timestamp: 2026-06-24T13:45:37.249Z
Learning: In the lightspeed-stack repository, docstrings must use the section header name "Parameters:" (not "Args:") for function arguments, even if the project references Google Python docstring conventions. Ensure docstrings follow the project’s established "Parameters:" header format for any documented function parameters.
Applied to files:
tests/profiles/no_qv_keys/profile.pytests/profiles/empty_qv_strings/profile.pytests/unit/qv_config.pysrc/models/api/responses/successful/catalog.pytests/profiles/test/profile.pytests/unit/utils/test_pydantic_ai.pysrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pytests/unit/app/endpoints/test_shields.pytests/unit/test_configuration.pysrc/models/config.pytests/unit/utils/test_prompts.pytests/unit/models/responses/test_successful_responses.pytests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.py
📚 Learning: 2026-01-12T10:58:40.230Z
Learnt from: blublinsky
Repo: lightspeed-core/lightspeed-stack PR: 972
File: src/models/config.py:459-513
Timestamp: 2026-01-12T10:58:40.230Z
Learning: In lightspeed-core/lightspeed-stack, for Python files under src/models, when a user claims a fix is done but the issue persists, verify the current code state before accepting the fix. Steps: review the diff, fetch the latest changes, run relevant tests, reproduce the issue, search the codebase for lingering references to the original problem, confirm the fix is applied and not undone by subsequent commits, and validate with local checks to ensure the issue is resolved.
Applied to files:
src/models/api/responses/successful/catalog.pysrc/models/config.py
📚 Learning: 2026-02-25T07:46:33.545Z
Learnt from: asimurka
Repo: lightspeed-core/lightspeed-stack PR: 1211
File: src/models/responses.py:8-16
Timestamp: 2026-02-25T07:46:33.545Z
Learning: In the Python codebase, requests.py should use OpenAIResponseInputTool as Tool while responses.py uses OpenAIResponseTool as Tool. This difference is intentional due to differing schemas for input vs output tools in llama-stack-api. Apply this distinction consistently to other models under src/models (e.g., ensure request-related tools use the InputTool variant and response-related tools use the ResponseTool variant). If adding new tools, choose the corresponding InputTool or Tool class based on whether the tool represents input or output, and document the rationale in code comments.
Applied to files:
src/models/api/responses/successful/catalog.pysrc/models/config.py
📚 Learning: 2026-07-06T15:26:18.398Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2071
File: src/models/config.py:2416-2422
Timestamp: 2026-07-06T15:26:18.398Z
Learning: In this repo’s Python code under src/**, don’t treat differences in string concatenation style as a style inconsistency when Black has effectively forced (or made clearer) use of explicit `+` string concatenation in multi-line logger/string expressions. If adjacent-literal implicit concatenation is avoided/changed specifically to accommodate Black’s formatting in these call sites, accept the `+` usage and don’t recommend converting it solely for consistency with nearby blocks that use implicit concatenation.
Applied to files:
src/models/api/responses/successful/catalog.pysrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pysrc/models/config.py
📚 Learning: 2026-07-17T19:25:05.325Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2166
File: src/utils/saved_prompts.py:129-157
Timestamp: 2026-07-17T19:25:05.325Z
Learning: For any endpoint that handles saved prompts and calls `src/utils/saved_prompts.py::create_saved_prompt`, treat the endpoint as the validation boundary. Before calling `create_saved_prompt`, validate the incoming saved-prompt name and content, specifically using `validate_saved_prompt_name` and then persist (store) the normalized value it returns. Do not call `create_saved_prompt` with unvalidated/raw name/content.
Applied to files:
src/models/api/responses/successful/catalog.pysrc/pydantic_ai_lightspeed/capabilities/question_validity/_capability.pysrc/models/config.py
📚 Learning: 2026-05-20T08:09:30.641Z
Learnt from: max-svistunov
Repo: lightspeed-core/lightspeed-stack PR: 1580
File: docs/design/llama-stack-config-merge/poc-results/library-mode/synthesized-run.yaml:107-110
Timestamp: 2026-05-20T08:09:30.641Z
Learning: In Llama-stack config YAMLs, when defining a Llama Guard safety shield entry, set `provider_shield_id` to the *guard model identifier* (e.g., `meta-llama/Llama-Guard-3-8B`). Do not use a chat/generative model id (e.g., `openai/gpt-4o-mini`): a chat-model id (or `native_override`) indicates only an override landed and does **not** mean the safety shield is actually gating queries. Ensure any E2E coverage for the related implementation (JIRA/E2E tests) exercises a real Llama Guard model to verify that the shield is effective.
Applied to files:
examples/lightspeed-stack-shields.yaml
🪛 markdownlint-cli2 (0.23.2)
docs/models/successful_responses.md
[warning] 576-576: Reference links and images should use a label that is defined
Missing link or image reference definition: "'validation'"
(MD052, reference-links-images)
[warning] 577-577: Reference links and images should use a label that is defined
Missing link or image reference definition: "'invalidresp'"
(MD052, reference-links-images)
🔇 Additional comments (20)
src/models/config.py (1)
1639-1686: LGTM!Also applies to: 1697-1713, 2724-2743, 3257-3293
tests/profiles/empty_qv_strings/profile.py (1)
1-9: LGTM!tests/profiles/no_qv_keys/profile.py (1)
1-7: LGTM!tests/unit/qv_config.py (1)
1-23: LGTM!tests/unit/test_configuration.py (1)
253-580: LGTM!tests/unit/utils/test_prompts.py (1)
338-354: LGTM!src/pydantic_ai_lightspeed/capabilities/question_validity/_capability.py (1)
119-155: LGTM!Also applies to: 173-173, 216-236, 238-254
tests/unit/pydantic_ai_lightspeed/capabilities/question_validity/test_capability.py (1)
13-25: LGTM!Also applies to: 111-121, 163-192, 203-245, 305-665, 693-768
tests/unit/utils/test_pydantic_ai.py (1)
25-25: LGTM!Also applies to: 77-77, 134-134, 300-300, 369-369
docs/devel_doc/openapi.json (3)
1443-1443: LGTM!Also applies to: 21275-21275
13876-13926: LGTM!
18826-18865: 🗄️ Data Integrity & IntegrationNo change needed for
GET /v1/shieldsnull fields.
GET /v1/shieldsserializes resolvedconfiguration.shields, andConfiguration.resolve_question_validity_shield_prompts()fills omittedQuestionValidityConfig.model_promptandinvalid_question_responsebefore exposure.docs/models/successful_responses.json (1)
1527-1545: LGTM!Also applies to: 4493-4504, 5860-5860
docs/models/successful_responses.md (1)
1956-1957: LGTM!docs/user_doc/config.md (1)
284-289: LGTM!Also applies to: 772-773
docs/user_doc/shields_guide.md (1)
84-137: LGTM!Also applies to: 150-152, 164-173
examples/lightspeed-stack-shields.yaml (1)
22-29: LGTM!src/models/api/responses/successful/catalog.py (1)
98-98: LGTM!tests/profiles/test/profile.py (1)
35-38: LGTM!tests/unit/models/responses/test_successful_responses.py (1)
196-196: LGTM!
asimurka
left a comment
asimurka
left a comment
There was a problem hiding this comment.
LGTM in overall, I pointed out one nit.
| profile_invalid_resp = profile.get_invalid_resp() | ||
|
|
||
| for shield in self.shields: | ||
| match shield.config: |
There was a problem hiding this comment.
This could be simplified to isinstance
There was a problem hiding this comment.
Thank you @asimurka, I've updated it to reflect using isInstance
When model_prompt or invalid_question_response are omitted, fill them at configuration load from the profile (then LCORE defaults) so GET /v1/shields and agent QV see effective text without duplicating YAML. Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>
Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>
c48a776 to
62f8134
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/models/config.py`:
- Around line 1643-1667: Update the newly added nullable annotations in the
configuration model, including the validation and invalid_resp fields and the
additional ranges noted in the review, from Optional[T] to the equivalent T |
None syntax. Preserve all field defaults, metadata, and behavior.
- Around line 3301-3317: Update the validator method containing the shields loop
to avoid mutating self.shields or any shield.config in place. For each
QuestionValidityConfig, create copied shield and configuration objects with
resolved model_prompt and invalid_question_response values, preserve other
shield data unchanged, and return a copied Configuration containing the rebuilt
shield list while leaving the original validator input untouched.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 840618ca-e36b-433d-bf07-2168d3865dd0
📒 Files selected for processing (7)
docs/devel_doc/openapi.jsondocs/models/successful_responses.jsondocs/models/successful_responses.mdsrc/models/api/responses/successful/catalog.pysrc/models/config.pytests/unit/app/endpoints/test_shields.pytests/unit/utils/test_pydantic_ai.py
📜 Review details
⏰ Context from checks skipped due to timeout. (12)
- GitHub Check: integration_tests (3.12)
- GitHub Check: build-pr
- GitHub Check: unit_tests (3.13)
- GitHub Check: integration_tests (3.13)
- GitHub Check: unit_tests (3.12)
- GitHub Check: E2E: library mode / ci / group 2
- GitHub Check: E2E Tests for Lightspeed Evaluation job
- GitHub Check: E2E: library mode / ci / group 1
- GitHub Check: E2E: server mode / ci / group 1
- GitHub Check: E2E: server mode / ci / group 2
- GitHub Check: E2E: library mode / ci / group 3
- GitHub Check: E2E: server mode / ci / group 3
⚠️ CI failures not shown inline (1)
GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-8-on-pull-request: Failed
Conclusion: failure
Konflux kflux-prd-rh02/lightspeed-stack-0-8-on-pull-request has <b>failed</b>.
<ul>
<li><b>Namespace</b>: <a href="https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant">lightspeed-core-tenant</a></li>
<li><b>PipelineRun:</b> <a href="https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-r6vcn">lightspeed-stack-0-8-on-pull-request-r6vcn</a></li>
</ul>
<hr>
<h4>Task Statuses:</h4>
<table>
<tr><th>Status</th><th>Duration</th><th>Name</th></tr>
<tr>
<td>🟢 Succeeded</td>
<td>4 seconds</td><td>
[init](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-r6vcn/logs/init)
</td></tr>
<tr>
<td>🟢 Succeeded</td>
<td>15 seconds</td><td>
[clone-repository](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-r6vcn/logs/clone-repository)
</td></tr>
<tr>
<td>🔴 Failed</td>
<td>18 seconds</td><td>
[prefetch-dependencies](https://konflux-ui.apps.kflux-prd-rh02.0fk9.p1.openshiftapps.com/ns/lightspeed-core-tenant/pipelinerun/lightspeed-stack-0-8-on-pull-request-r6vcn/logs/prefetch-dependencies)
</td></tr>
</table>
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (Custom checks)
**/*: Flag meaningful O(n^2)+ algorithms on non-trivial inputs, including handlers and Kubernetes list operations.
Flag N+1 patterns that list items and then query once per item, including Kubernetes API and database access.
Flag expensive work inside loops, including API calls, JSON parsing, and regex compilation.
Flag unbounded growth in caches, watchers, or buffers when eviction or limits are missing.
Flag missing pagination or limits on list operations and API endpoints.
Flag secrets or tokens logged in plaintext or hardcoded in source.
Flag API endpoints missing authentication or authorization.
Flag injection vulnerabilities, including SQL injection, command injection, and path traversal.
Flag sensitive data leaked in API responses, WebSocket messages, or logs.
Flag Kubernetes Secrets and Red Hat secrets missing OwnerReferences.
Files:
docs/models/successful_responses.mdtests/unit/app/endpoints/test_shields.pydocs/models/successful_responses.jsondocs/devel_doc/openapi.jsonsrc/models/config.pytests/unit/utils/test_pydantic_ai.pysrc/models/api/responses/successful/catalog.py
tests/unit/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Use pytest for unit tests, shared fixtures in
conftest.py,pytest-mockfor mocks,pytest.mark.asynciofor async tests, and maintain at least 60% unit-test coverage.
Files:
tests/unit/app/endpoints/test_shields.pytests/unit/utils/test_pydantic_ai.py
src/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/**/*.py: Use absolute imports for internal modules and follow the prescribed FastAPI and Llama Stack import conventions.
All modules must begin with descriptive docstrings; uselogger = get_logger(__name__)fromlog.pyfor module logging; package__init__.pyfiles must contain brief package descriptions.
Define shared constants in the centralconstants.pymodule, add descriptive comments, and annotate constants withFinal[type].
Use complete type annotations for function parameters, return types, class attributes, and type aliases; prefer specific types overAny, use modern union syntax, and usetyping_extensions.Selffor model validators.
All functions and classes require descriptive Google-style docstrings, including appropriateParameters,Returns,Raises, andAttributessections.
Use descriptive snake_case, action-oriented function names such asget_,validate_, andcheck_; use PascalCase class names with standard suffixes such asConfiguration,Error/Exception,Resolver, andInterface.
Avoid modifying input parameters in place; return a newly constructed data structure instead.
Useasync deffor I/O operations and external API calls; API endpoints should raise FastAPIHTTPExceptionwith appropriate status codes and handle Llama StackAPIConnectionError.
Usefrom log import get_loggerand standard logger levels:debugfor diagnostics,infofor general execution,warningfor unexpected conditions or potential problems, anderrorfor serious failures.
Configuration models must extendConfigurationBase, setextra="forbid"to reject unknown fields, use Pydantic validators for custom validation, and use types such asOptional[FilePath],PositiveInt, andSecretStrwhere appropriate.
Abstract interfaces must useABCand@abstractmethoddecorators.
Never commit secrets or keys; use environment variables for sensitive data.
Files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
src/models/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Pydantic data models must extend
BaseModel; configuration models must extendConfigurationBase; use@model_validatorand@field_validatorfor validation.
Files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
🧠 Learnings (5)
📚 Learning: 2026-06-24T13:45:37.249Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 1971
File: src/utils/markdown_repair.py:31-36
Timestamp: 2026-06-24T13:45:37.249Z
Learning: In the lightspeed-stack repository, docstrings must use the section header name "Parameters:" (not "Args:") for function arguments, even if the project references Google Python docstring conventions. Ensure docstrings follow the project’s established "Parameters:" header format for any documented function parameters.
Applied to files:
tests/unit/app/endpoints/test_shields.pysrc/models/config.pytests/unit/utils/test_pydantic_ai.pysrc/models/api/responses/successful/catalog.py
📚 Learning: 2026-01-12T10:58:40.230Z
Learnt from: blublinsky
Repo: lightspeed-core/lightspeed-stack PR: 972
File: src/models/config.py:459-513
Timestamp: 2026-01-12T10:58:40.230Z
Learning: In lightspeed-core/lightspeed-stack, for Python files under src/models, when a user claims a fix is done but the issue persists, verify the current code state before accepting the fix. Steps: review the diff, fetch the latest changes, run relevant tests, reproduce the issue, search the codebase for lingering references to the original problem, confirm the fix is applied and not undone by subsequent commits, and validate with local checks to ensure the issue is resolved.
Applied to files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
📚 Learning: 2026-02-25T07:46:33.545Z
Learnt from: asimurka
Repo: lightspeed-core/lightspeed-stack PR: 1211
File: src/models/responses.py:8-16
Timestamp: 2026-02-25T07:46:33.545Z
Learning: In the Python codebase, requests.py should use OpenAIResponseInputTool as Tool while responses.py uses OpenAIResponseTool as Tool. This difference is intentional due to differing schemas for input vs output tools in llama-stack-api. Apply this distinction consistently to other models under src/models (e.g., ensure request-related tools use the InputTool variant and response-related tools use the ResponseTool variant). If adding new tools, choose the corresponding InputTool or Tool class based on whether the tool represents input or output, and document the rationale in code comments.
Applied to files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
📚 Learning: 2026-07-06T15:26:18.398Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2071
File: src/models/config.py:2416-2422
Timestamp: 2026-07-06T15:26:18.398Z
Learning: In this repo’s Python code under src/**, don’t treat differences in string concatenation style as a style inconsistency when Black has effectively forced (or made clearer) use of explicit `+` string concatenation in multi-line logger/string expressions. If adjacent-literal implicit concatenation is avoided/changed specifically to accommodate Black’s formatting in these call sites, accept the `+` usage and don’t recommend converting it solely for consistency with nearby blocks that use implicit concatenation.
Applied to files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
📚 Learning: 2026-07-17T19:25:05.325Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2166
File: src/utils/saved_prompts.py:129-157
Timestamp: 2026-07-17T19:25:05.325Z
Learning: For any endpoint that handles saved prompts and calls `src/utils/saved_prompts.py::create_saved_prompt`, treat the endpoint as the validation boundary. Before calling `create_saved_prompt`, validate the incoming saved-prompt name and content, specifically using `validate_saved_prompt_name` and then persist (store) the normalized value it returns. Do not call `create_saved_prompt` with unvalidated/raw name/content.
Applied to files:
src/models/config.pysrc/models/api/responses/successful/catalog.py
🪛 markdownlint-cli2 (0.23.2)
docs/models/successful_responses.md
[warning] 595-595: Reference links and images should use a label that is defined
Missing link or image reference definition: ""validation""
(MD052, reference-links-images)
[warning] 596-596: Reference links and images should use a label that is defined
Missing link or image reference definition: ""invalidresp""
(MD052, reference-links-images)
🔇 Additional comments (8)
src/models/config.py (1)
1277-1277: LGTM!Also applies to: 1640-1641, 1678-1687, 3280-3300, 3475-3529
tests/unit/utils/test_pydantic_ai.py (1)
25-25: LGTM!Also applies to: 78-78, 135-135, 301-301, 370-370, 426-444
docs/devel_doc/openapi.json (1)
1443-1443: LGTM!Also applies to: 14057-14084, 19018-19039, 21459-21459
docs/models/successful_responses.md (2)
595-596: Make thePROFILE_CONFIGreferences Markdown-safe.Lines 595-596 still trigger
markdownlint-cli2MD052. Markdown parses the bracketed keys as reference labels. Put each full expression in a code span.Proposed fix
-| validation | string | Read-only. Loaded from the custom profile Python module (PROFILE_CONFIG["system_prompts"]["validation"]), not from lightspeed-stack.yaml. Used when a question_validity shield omits model_prompt. | -| invalid_resp | string | Read-only. Loaded from the custom profile Python module (PROFILE_CONFIG["query_responses"]["invalid_resp"]), not from lightspeed-stack.yaml. Used when a question_validity shield omits invalid_question_response. | +| validation | string | Read-only. Loaded from the custom profile Python module (`PROFILE_CONFIG["system_prompts"]["validation"]`), not from lightspeed-stack.yaml. Used when a question_validity shield omits model_prompt. | +| invalid_resp | string | Read-only. Loaded from the custom profile Python module (`PROFILE_CONFIG["query_responses"]["invalid_resp"]`), not from lightspeed-stack.yaml. Used when a question_validity shield omits invalid_question_response. |Source: Linters/SAST tools
55-73: LGTM!Also applies to: 222-222, 1801-1801, 1975-1976, 2431-2448, 2577-2626
docs/models/successful_responses.json (1)
64-82: LGTM!Also applies to: 132-132, 462-474, 1562-1580, 3988-4000, 4537-4548, 5712-5755, 5948-5948, 5988-6008, 6026-6056
src/models/api/responses/successful/catalog.py (1)
9-10: LGTM!Also applies to: 13-41, 130-130
tests/unit/app/endpoints/test_shields.py (1)
106-106: LGTM!Also applies to: 138-140
| validation: Optional[str] = Field( | ||
| default=None, | ||
| init=False, | ||
| title="Question validity classifier prompt", | ||
| description=( | ||
| "Read-only. Loaded from the custom profile Python module " | ||
| "(PROFILE_CONFIG['system_prompts']['validation']), not from " | ||
| "lightspeed-stack.yaml. Used when a question_validity shield omits " | ||
| "model_prompt." | ||
| ), | ||
| json_schema_extra={"readOnly": True}, | ||
| ) | ||
|
|
||
| invalid_resp: Optional[str] = Field( | ||
| default=None, | ||
| init=False, | ||
| title="Invalid question response", | ||
| description=( | ||
| "Read-only. Loaded from the custom profile Python module " | ||
| "(PROFILE_CONFIG['query_responses']['invalid_resp']), not from " | ||
| "lightspeed-stack.yaml. Used when a question_validity shield omits " | ||
| "invalid_question_response." | ||
| ), | ||
| json_schema_extra={"readOnly": True}, | ||
| ) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Use modern union syntax for new nullable types.
Replace each new Optional[...] annotation with the equivalent T | None form.
As per coding guidelines, “use modern union syntax.”
Also applies to: 1698-1715, 2615-2624, 2735-2755, 3043-3053
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/models/config.py` around lines 1643 - 1667, Update the newly added
nullable annotations in the configuration model, including the validation and
invalid_resp fields and the additional ranges noted in the review, from
Optional[T] to the equivalent T | None syntax. Preserve all field defaults,
metadata, and behavior.
Source: Coding guidelines
| for shield in self.shields: | ||
| if not isinstance(shield.config, QuestionValidityConfig): | ||
| continue | ||
| qv_config = shield.config | ||
| if qv_config.model_prompt is None: | ||
| qv_config.model_prompt = ( | ||
| profile_validation | ||
| if profile_validation is not None | ||
| else constants.DEFAULT_MODEL_PROMPT | ||
| ) | ||
| if qv_config.invalid_question_response is None: | ||
| qv_config.invalid_question_response = ( | ||
| profile_invalid_resp | ||
| if profile_invalid_resp is not None | ||
| else constants.DEFAULT_INVALID_QUESTION_RESPONSE | ||
| ) | ||
| return self |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Return copied shield configuration instead of mutating the validator input.
This validator writes resolved values into shield.config. Build copied shield and QuestionValidityConfig objects, then return a copied Configuration with the resolved shield list.
As per coding guidelines, “Avoid modifying input parameters in place; return a newly constructed data structure instead.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/models/config.py` around lines 3301 - 3317, Update the validator method
containing the shields loop to avoid mutating self.shields or any shield.config
in place. For each QuestionValidityConfig, create copied shield and
configuration objects with resolved model_prompt and invalid_question_response
values, preserve other shield data unchanged, and return a copied Configuration
containing the rebuilt shield list while leaving the original validator input
untouched.
Source: Coding guidelines
2 participants
Description
When a question_validity shield omits model_prompt or invalid_question_response, we now fill those at configuration load from the customization profile (then LCORE defaults). That way GET /v1/shields and the agent QV path see the effective text without copying the same prompts into YAML. This lets users include long model_prompts in the profile, rather than stuffing them inline in the yaml.
Resolution order:
Profile keys come from the profile Python module (PROFILE_CONFIG), not from YAML under customization:.
Example (omit the prompt fields and let the profile supply them):
With a profile that looks like:
After load, both fields are set to the profile values. GET /v1/shields returns those resolved strings.
Type of change
pyproject.toml+uv.lock]requirements.*.txtfor Konflux]Tools used to create PR
Identify any AI code assistants used in this PR (for transparency and review context)
Related Tickets & Documents
Checklist before requesting a review
Testing
Summary by CodeRabbit
New Features
${message}substitution during agent execution; responses moderation evaluates the raw input.Documentation