Repository navigation
[AAASM-6415] 📝 (readme): Document required LangChain callback wiring - #387
Chisanan232 wants to merge 7 commits into
Conversation
init_assembly() activates the LangChain adapter and creates its AssemblyCallbackHandler but installs no hook into LangChain, so a denied tool still runs unless the handler is passed in callbacks. Correct the Quick Start snippet and the claims that said otherwise. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
…r wired Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…adapter Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
…page Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
|
|
OWNER-MERGE-REVIEW head=60c5bbe56ed67399812c148b256a6e779e62b39d verdict=PASS blocking=0 reviewer=claude-independent-agent Independent exact-head review (separate read-only agent) of the docs-only correction for D5 / AAASM-6415, over four iterations (60aae12 -> ea9b261 -> 47b8cf4 -> f3a39cf -> 60c5bbe): final verdict PASS, no Critical/High/Medium. PR = README.md, docs/index.md, docs/concepts/architecture.md, docs/concepts/index.md, docs/examples/framework-support.md, docs/examples/langchain-basic-agent.md and one test file; no runtime change. Claims now limited to what was verified against source (init_assembly() installs no hook into LangChain; handler must be passed via callbacks; denied tool with wiring raises ToolExecutionBlockedError and has no side effect, negative control proves the gap). Not verified, and stated in the docs: the AgentExecutor config={callbacks} propagation (langchain-classic not installed in the verification env); decisions come from the repo's fake native runtime. Open product question (silent fail-open on LangChain without callbacks: docs-only vs auto-wire vs runtime warning) is recorded in the PR body for the owner and is NOT resolved by this PR. D5 is not claimed resolved by this merge. Merging requires the owner's formal GitHub approval on this head (repo rule). 🤖 Generated with Claude Code |



Outcome
The README Quick Start said
init_assembly()sends every LangChain tool call through the policy gate. That is false:init_assembly()activates the LangChain adapter and creates oneAssemblyCallbackHandler, but installs no hook into LangChain. A denied tool still runs unless the handler is passed incallbacks. This PR corrects the docs and adds a test that pins the behaviour. No runtime or public API change.Evidence
runtime.py:auto_inject_callback_handleronly stores the handler singleton; no LangChain hook is registered anywhere in the SDK (QA finding D5, AAASM-6415).test/unit/test_quickstart_langchain_denied_tool_wiring.pyruns the realinit_assembly(realRuntimeQueryInterceptor, repo's fake native runtime as decision source), takes the handler from the langchain adapter on the returned context, and invokes a reallangchain_coreStructuredToolwhose body writes a file:test_denied_tool_without_wiring_still_executes(negative control, documents the gap): deny, handler not passed, file written.test_denied_tool_with_wiring_has_no_side_effect: deny, handler inconfig={"callbacks": [handler]},ToolExecutionBlockedError, no file.test_allowed_tool_with_wiring_executes: positive control.mypy agent_assemblyreports 5 pre-existing errors in files this PR does not touch (4 are missing native_corestub, not run against a built extension here).docs/index.md"What it wraps".Not verified / limits
AgentExecutor+create_react_agent) was not executed:langchain,langchain-classic,langchain-communityare not in this environment. The mechanism is verified with alangchain_coretool via the callback manager; passingconfig={"callbacks": [handler]}toAgentExecutor.invokerelies on LangChain's standard config propagation to tools.langchainmodule insys.modulesonly to satisfyFrameworkAdapter.is_available(); the adapter, handler and invocation path are real.native/aa-ffi-python/Cargo.toml:aa-corerev0044d515d020. No Rust build was done.StructuredTool, notAgentExecutor; propagation ofconfig={"callbacks": [...]}through an agent executor is not verified.init_assembly()should auto-wire LangChain (e.g. via a configure hook) is a separate design question.Needs owner decision (no code here)
Silent fail-open:
init_assembly()succeeds and reports the LangChain adapter active, yet tools are ungoverned unlesscallbacksis passed. Options: (a) docs-only plus a follow-up ticket (this PR), (b) auto-wire via LangChainregister_configure_hook(runtime contract change), (c) a runtime/doctor warning when the handler has not been seen on a tool run.🤖 Generated with Claude Code
https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4