Skip to content

feat(plugins): add context-aware hooks for memory pipelines - #2623

Open
consultantQ wants to merge 11 commits into
MemTensor:dev-v2.0.37from
consultantQ:feat/add-more-plugin-hooks
Open

consultantQ wants to merge 11 commits into
MemTensor:dev-v2.0.37from
consultantQ:feat/add-more-plugin-hooks

Conversation

@consultantQ

Copy link
Copy Markdown

Description

This PR is a prerequisite for the upcoming SmartComment tracing integration.
SmartComment uses these hooks and their correlation metadata to build execution graphs across memory extraction, storage, search, and scheduler operations.

The integration is developed on the feat/smartcomment-integration branch and will be submitted as a separate PR after this change is merged.

The hooks introduced here are general-purpose extension points and add no dependency on SmartComment.

This PR extends the plugin hook system across search, memory extraction, textual memory writes, and scheduler operations.

Changes:

  • Add HookContext with request, user, task, cube, and operation metadata so plugins can correlate events across processing stages.
  • Preserve compatibility with existing callbacks that do not accept hook_context.
  • Add hooks after search threshold filtering and deduplication, plus a post-processing failure hook. Extend existing search hooks with correlation context.
  • Add success/failure hooks for memory extraction, textual memory writes, and scheduler fine-transfer/add/delete/soft-delete operations.
  • Fix concurrent fine-transfer results being returned in completion order, with failed or None results omitted. Results now preserve input order and retain empty groups for missing outputs, keeping each result aligned with its source memory.
  • Expose per_input, batch, or unknown grouping metadata so plugins can interpret reader outputs correctly.
  • Add hook documentation and regression tests covering callback compatibility, result replacement, failure handling, and input/output alignment.

Dependencies: No new third-party dependencies.

Related Issue (Required): None (new feature, no prior issue)

Type of change

  • New feature
  • Bug fix: preserve input-to-output alignment in concurrent fine transfer
  • Documentation update

How Has This Been Tested?

Validation results on commit 5cbd7d4b:

  • Focused hook regression suite: 57 passed.
  • Full test suite: 1056 passed, 9 skipped, 39 warnings
  • Ruff lint and formatting checks passed for all 20 changed Python files.
  • OpenAPI schemas matched upstream/dev-v2.0.37 using mocked server
    initialization: 26 paths and 59 schemas, with no differences.

The commands below describe the focused test suite:

poetry run pytest -q \
  tests/plugins/test_hook_context.py \
  tests/api/test_search_hook_context.py \
  tests/api/test_search_pipeline_hooks.py \
  tests/mem_reader/test_fine_transfer_order.py \
  tests/mem_reader/test_hook_pipeline.py \
  tests/mem_scheduler/test_mem_read_grouping.py \
  tests/mem_scheduler/test_operation_observer.py \
  tests/multi_mem_cube/test_single_cube_hooks.py
  • Unit tests executed and results recorded
  • Formatting and lint checks completed
  • OpenAPI schema comparison completed with mocked server initialization
  • Full test suite executed

Checklist

  • I have performed a self-review of my own code.
  • I have commented my code in hard-to-understand areas.
  • I have added and verified tests covering the changes.
  • I have created a related documentation issue/PR in MemOS-Docs, if applicable.
  • I have linked the related issue.
  • I have mentioned the intended reviewer.

Reviewer Checklist

  • Related issue and scope verified
  • Required checks passed
  • Tests reviewed

qzs and others added 10 commits September 4, 2026 23:00
Port the reviewed Hook-only tests from smartcomment-integration.
Keep upstream Search tests intact apart from required hook_context
assertions, and isolate new behavior tests with an opt-in fixture.
Cover Reader grouping/order and all five MemRead operation sites.

Validation on feat/add-more-plugin-hooks: 49 targeted tests passed;
related regression: 266 passed, 1 skipped. Individual test files,
Ruff, staged pre-commit, and diff checks passed. Full make test
stopped after 180 seconds at the existing Ollama network request.
- Normalize scheduler context to dictionaries and remove target parameter
- Include reader and deletion arguments in operation inputs
- Remove archive observations and redundant search context initialization
- Update hook documentation and regression tests
Copilot AI balanced review requested due to automatic review settings November 17, 2026 07:35
@Memtensor-AI Memtensor-AI added area:api 云服务 / FastAPI / OpenAPI / MCP area:memcube GeneralMemCube / cube 生命周期 / cube 配置 area:memory 记忆存储、检索、更新、召回逻辑 area:scheduler 调度模块 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Nov 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Both sync and async wrappers mishandle caller-supplied hook_context, causing TypeError for methods that reject the keyword or receive it positionally, and passing the wrong context to hooks in the positional case.

2 open issues
What changed in this PR

This PR extends MemOS plugin hooks across memory pipelines, providing correlation metadata for future tracing integrations without adding a SmartComment dependency.

Changes:

  • Add HookContext and context-aware callbacks while retaining legacy callback support.
  • Add search-stage hooks and success/failure hooks for extraction, textual writes, and scheduler operations.
  • Preserve fine-transfer input/output alignment and document grouping contracts with regression tests.
File Description
tests/​plugins/​test_hook_context.py Tests context metadata and callback compatibility.
tests/​multi_mem_cube/​test_single_cube_hooks.py Tests textual-write hooks and result replacement.
tests/​mem_scheduler/​test_operation_observer.py Tests scheduler operation hooks and correlation.
tests/​mem_scheduler/​test_mem_read_grouping.py Tests reader grouping metadata.
tests/​mem_reader/​test_hook_pipeline.py Tests extraction success and failure hooks.
tests/​mem_reader/​test_fine_transfer_order.py Tests ordered results and empty output slots.
tests/​conftest.py Adds hook-registration isolation fixture.
tests/​api/​test_search_pipeline_hooks.py Updates expected search hook parameters.
tests/​api/​test_search_hook_context.py Tests search context, replacement, and failures.
src/​memos/​plugins/​README.md Introduces context-aware hook usage.
src/​memos/​plugins/​hooks.py Adds context handling and legacy callback adaptation.
src/​memos/​plugins/​HOOKS.md Documents pipeline hook contracts.
src/​memos/​plugins/​hook_defs.py Declares new hooks and context parameters.
src/​memos/​plugins/​hook_context.py Defines correlation metadata and its builder.
src/​memos/​plugins/​base.py Corrects registration-method wording in the docstring.
src/​memos/​plugins/​__init__.py Exports context APIs.
src/​memos/​multi_mem_cube/​single_cube.py Adds hooks around textual-memory writes.
src/​memos/​mem_scheduler/​task_schedule_modules/​operation_observer.py Adds scheduler operation observation.
src/​memos/​mem_scheduler/​task_schedule_modules/​handlers/​mem_read_handler.py Instruments memory operations and reports grouping.
src/​memos/​mem_scheduler/​task_schedule_modules/​base_handler.py Adds a shared operation-observer factory.
src/​memos/​mem_reader/​simple_struct.py Adds extraction hooks and preserves transfer alignment.
src/​memos/​api/​handlers/​search_handler.py Adds contextual search-stage and failure hooks.

Review effort: Balanced

Comment thread src/memos/plugins/hooks.py Outdated
Comment on lines +212 to +213
if _accepts_hook_context(func):
call_kwargs["hook_context"] = hook_context
Comment thread src/memos/plugins/hooks.py Outdated
Comment on lines +212 to +213
if _accepts_hook_context(func):
call_kwargs["hook_context"] = hook_context
@Memtensor-AI

Memtensor-AI commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2623
Task: 793c0c6b7d8eebe4
Base: dev-v2.0.37
Head: feat/add-more-plugin-hooks
Head SHA: a33c8b0e8ed0b129061f74a43246b5e25380529a

🔍 OpenCodeReview found 2 issue(s) in this PR.

⚠️ Review warnings (3)

  • completed primary review; remaining group budget is insufficient for another measured pass and finalization
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files

1. tests/conftest.py (L9-L10)

The fixture only snapshots/restores the registration table _hooks, but memos.plugins.hook_defs._specs is also module-level mutable state populated by define_hook. Tests using this fixture define hard-coded hook names/specs inside test bodies (e.g. define_hook("test.hook-context.compatibility", ...)), and those specs persist for the whole pytest session. Because define_hook is idempotent (first definition wins), a later test reusing the same name silently gets the earlier spec instead of its own, and names stay "declared" so trigger_hook no longer warns for them — leaving the isolation incomplete/order-dependent.

Consider snapshotting _specs as well as _hooks:

💡 Suggested Change

Before:

    saved = {name: list(callbacks) for name, callbacks in _hooks.items()}
    _hooks.clear()

After:

    saved = {name: list(callbacks) for name, callbacks in _hooks.items()}
    from memos.plugins.hook_defs import _specs

    saved_specs = dict(_specs)
    _hooks.clear()

2. src/memos/plugins/hook_context.py (L23)

HookContext is declared with frozen=True and the default eq=True, so Python auto-generates __hash__ over every field. Since attributes is a plain dict, hash(build_hook_context(source="x")) raises TypeError: unhashable type: 'dict', even though the frozen dataclass advertises hashability/immutability. Any plugin that groups, caches, or keys contexts (set, dict key, lru_cache) will fail at runtime. Exclude attributes from the generated hash while keeping it in __eq__.

💡 Suggested Change

Before:

    attributes: Mapping[str, Any] = field(default_factory=dict)

After:

    attributes: Mapping[str, Any] = field(default_factory=dict, hash=False)

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api 云服务 / FastAPI / OpenAPI / MCP area:memcube GeneralMemCube / cube 生命周期 / cube 配置 area:memory 记忆存储、检索、更新、召回逻辑 area:scheduler 调度模块 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants