Skip to content

Release v2.0.36 - #2568

Merged
syzsunshine219 merged 31 commits into
mainfrom
dev-v2.0.36
Nov 9, 2026
Merged

syzsunshine219 merged 31 commits into
mainfrom
dev-v2.0.36

Conversation

@endxxxx

@endxxxx endxxxx commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Merge dev-v2.0.36 into main for MemOS v2.0.36. The final release-branch commit sets both the package version and memos.__version__ to 2.0.36.

This release includes the PostgreSQL graph storage updates (#2531), knowledge memory and recall changes (#2479, #2436), and fixes for PolarDB recall (#2551), RabbitMQ shutdown (#2532), API key validation (#2491), and rate limiting (#2490).

How Has This Been Tested?

  • Unit tests — 24 passed in tests/graph_dbs/test_postgres_400_compat.py and tests/mem_scheduler/test_rabbitmq_shutdown.py.
  • Verified that both version fields are 2.0.36, ran make format and git diff --check, and simulated the merge with main.

Checklist

  • Reviewed the release diff and the two-file version commit.
  • Verified the release branch merges cleanly with the current main locally.

MatthewZhuang and others added 30 commits September 22, 2026 16:09
Co-authored-by: Cursor <cursoragent@cursor.com>
Playground only kept the first cube in text_mem, so personal memories
were dropped when a knowledge-base cube came first.

Co-authored-by: Cursor <cursoragent@cursor.com>
Relative markdown paths such as images/<hash>.jpg were forwarded to the vision API and rejected as invalid base64.

Co-authored-by: Cursor <cursoragent@cursor.com>
Relative markdown paths such as images/<hash>.jpg were forwarded to the vision API and rejected as invalid base64.

Co-authored-by: Cursor <cursoragent@cursor.com>
Persist summary-material and raw-file neighbor relationships as node metadata so knowledge storage, retrieval deduplication, deletion, merging, and feedback locking remain functional on backends without graph edges. Raw file creation now follows file sources independently of prompt selection.

Constraint: SUMMARY, MATERIAL, PRECEDING, and FOLLOWING edge operations are unavailable on the replacement graph backend.
Rejected: Remove legacy edge helpers | retained unused compatibility code as requested.
Confidence: high
Scope-risk: moderate
Directive: Keep knowledge_material_ids and knowledge_summary_ids bidirectional when adding, merging, or deleting knowledge nodes.
Tested: 10 focused test functions executed directly; Ruff passed on all changed files.
Not-tested: pytest runner crashes with exit 139 in the existing Anaconda environment; uv environment setup timed out fetching lxml.
Persist summary-material and raw-file neighbor relationships as node metadata so knowledge storage, retrieval deduplication, deletion, merging, and feedback locking remain functional on backends without graph edges. Raw file creation now follows file sources independently of prompt selection.

Constraint: SUMMARY, MATERIAL, PRECEDING, and FOLLOWING edge operations are unavailable on the replacement graph backend.
Rejected: Remove legacy edge helpers | retained unused compatibility code as requested.
Confidence: high
Scope-risk: moderate
Directive: Keep knowledge_material_ids and knowledge_summary_ids bidirectional when adding, merging, or deleting knowledge nodes.
Tested: 10 focused test functions executed directly; Ruff passed on all changed files.
Not-tested: pytest runner crashes with exit 139 in the existing Anaconda environment; uv environment setup timed out fetching lxml.

Co-authored-by: 黑布林 <11641432+heiheiyouyou@user.noreply.gitee.com>
* feat: align PostgreSQL graph storage with current v2.0.36 APIs

Store the existing graph interface on MD5 tenant shards so this branch can run on PostgreSQL without attribute-tree tables.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style: format PostgreSQL graph files for Ruff

The upstream check stops on ruff format before the tests run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: retrigger pull request checks

The Windows Python 3.11 dependency install failed once and needs another run.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
The Docker requirements and lockfiles still shipped CA bundles from before the October 2026 distrust.

Co-authored-by: jiachengzhen <jiacz@memtensor.cn>
Co-authored-by: Cursor <cursoragent@cursor.com>
…_read_handler (#2512)

## Description

Fix an ID-to-memory misalignment and truncation bug in
`MemReadMessageHandler._process_memories_with_reader` when constructing
legacy playground web log events (`addMemory` / `updateMemory`).

In `_process_memories_with_reader`:
1. `mem_group` filters out `RawFileMemory` items
(`memory.metadata.memory_type != "RawFileMemory"`), and
`enhanced_mem_ids = text_mem.add(mem_group, user_name=user_name)`
returns IDs corresponding 1-to-1 with `mem_group`.
2. However, the playground legacy event loop zipped `enhanced_mem_ids`
with the unfiltered `flattened_memories` (`zip(enhanced_mem_ids,
flattened_memories, strict=False)`). Whenever `RawFileMemory` items were
present in `flattened_memories`, they consumed the `enhanced_mem_ids`
belonging to normal memories, shifted every subsequent ID-to-memory
mapping, and caused trailing memories in `mem_group` to be silently
dropped by `zip(..., strict=False)`.

Aligning the `zip` target with `mem_group` (`zip(enhanced_mem_ids,
mem_group, strict=False)`) ensures each returned `item_id` maps to the
exact memory added to `text_mem`.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Added unit test
`test_playground_legacy_events_align_enhanced_ids_with_filtered_mem_group`
in `tests/mem_scheduler/test_mem_read_handler_fast_cleanup.py` covering
a mixed batch `[RawFileMemory, LongTermMemory (add), UserMemory (update
with merged_from)]` and asserting exact `ref_id`, `id`, `memory`, and
`memory_type` alignment in both `addMemory` and `updateMemory` events
without truncation.

```bash
pytest tests/mem_scheduler/test_mem_read_handler_fast_cleanup.py -vv
ruff check src/memos/mem_scheduler/task_schedule_modules/handlers/mem_read_handler.py tests/mem_scheduler/test_mem_read_handler_fast_cleanup.py
```

- [x] Unit Test

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [ ] I have created related documentation issue/PR in
[MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) |
我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档
issue/PR(如果适用)
- [ ] I have linked the issue to this PR (if applicable) | 我已将 issue
链接到此 PR(如果适用)
- [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人
…nses API (#2511)

## Description

Fix `IndexError: list index out of range` in
`OpenAIResponsesLLM.generate` and `AzureResponsesLLM.generate` when
`response.output` does not contain any `ResponseReasoningItem` (for
example, when `enable_thinking=False` by default or when calling
non-reasoning models).

Currently, both methods evaluate `summary = output_reasoning[0].summary`
unconditionally right after filtering `output_reasoning`, which raises
`IndexError` whenever `output_reasoning` is empty (`[]`). Guarding the
lookup with `output_reasoning[0].summary if output_reasoning else None`
safely handles both missing `ResponseReasoningItem` entries and empty
`summary` lists.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Ran unit tests covering `OpenAIResponsesLLM.generate` and
`AzureResponsesLLM.generate` across three scenarios:
1. Responses without any `ResponseReasoningItem` (`output_reasoning ==
[]`)
2. Responses with `ResponseReasoningItem(summary=[])`
3. Responses with non-empty reasoning `summary`
(`remove_think_prefix=False` and `remove_think_prefix=True`)

```bash
pytest tests/llms/test_openai_responses_llm.py -vv
ruff check src/memos/llms/openai_new.py tests/llms/test_openai_responses_llm.py
```

- [x] Unit Test

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [ ] I have created related documentation issue/PR in
[MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) |
我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档
issue/PR(如果适用)
- [ ] I have linked the issue to this PR (if applicable) | 我已将 issue
链接到此 PR(如果适用)
- [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人
## Description

Neo4j-backed recall can return an empty result after `TextualMemoryItem`
rejects a JSON-encoded `metadata.internal_info`. The native ingest
reader can also skip the same nodes when constructing
`TreeNodeTextualMemoryMetadata`.

The chunking path creates `internal_info` as a dictionary. Neo4j
serializes dictionary properties as JSON, but its read conversion does
not restore this field before Pydantic validation. A short conversation
with `internal_info=None` does not exercise the failure.

Add a before-validator on `TextualMemoryMetadata.internal_info`, shared
by the search and reader metadata models. Decode string values once and
require a JSON object. Existing dictionaries and `None` keep their
behavior; malformed JSON, encoded arrays/scalars/null, and
double-encoded strings remain validation errors. No storage migration,
write-path change, request/response route change, new dependency, LLM
request, or retrieval-policy change is required.

Related issue: #2288. Earlier PR #2289 was closed without merging; the
failure remains reproducible on current `main` (`a7367d0`). This change
deliberately retains validation errors for malformed values instead of
silently replacing them with `None`.

## Type of change

- [x] Bug fix

## How Has This Been Tested?

- Before the fix: new regression file has **5 failed, 15 passed**,
reproducing the JSON-string validation error.
- After the fix: new regression file has **20 passed**.
- In an isolated Python 3.12 / Pydantic 2.12.4 backend container, with
networking disabled, no credentials and no business volumes:

  ```sh
PYTHONPATH=src python -B -m pytest -p no:cacheprovider -o
filterwarnings= -o addopts= -q tests/memories/textual/
  ```

**80 passed**. Coverage includes all four metadata subclasses, search
`from_dict`, direct reader metadata construction, unchanged
source/session/content fields, repeated storage JSON round trips,
existing dictionary/None values, and rejection of malformed or
non-object JSON.

The image does not include pytest-asyncio, producing an existing unknown
`asyncio_mode` configuration warning; one existing naive-memory test
also reports a Pydantic serialization warning.

- `make format`: **passed** (`All checks passed!`; `631 files left
unchanged`), using the repository-pinned Ruff 0.11.8.
- `pre-commit run --files src/memos/memories/textual/item.py
tests/memories/textual/test_item.py`: **passed** (all applicable hooks).

## Checklist

- [x] Self-reviewed the diff; only the metadata validator and its
regression tests change.
- [x] Added an explanatory comment at the storage/model boundary.
- [x] Added tests that fail before the fix and pass afterward.
- [x] Linked the existing issue. No public API or documentation change
is required.

Maintainer review and upstream CI are pending; the full repository suite
was not run locally.
…mit bucket (#2428)

_get_client_key in src/memos/api/middleware/rate_limit.py tested
whether request.headers['Authorization'] started with 'krlk_', but real
clients send 'Authorization: Bearer krlk_<hex>' per RFC 6750. The prefix
check never matched and every request — including ones carrying a valid
API key — fell through to IP-based limiting. Behind a shared egress IP
all keyed clients shared one bucket, so the per-key rate-limit was dead.

The fix strips the 'Bearer ' scheme (case-insensitive, per RFC 7235)
before the prefix check, so 'Bearer krlk_<hex>' correctly maps to
'ratelimit:key:<first 20 of stripped key>'. Backwards compatibility is
preserved: a bare 'krlk_<hex>' header still matches.

- New tests/api/test_rate_limit_bearer.py with 7 cases covering Bearer
  prefix, lowercase bearer, non-key Bearer tokens, missing header,
  X-Forwarded-For preference, and the bare-krlk_ backwards-compat path.
- No new imports, no new dependencies, no public API change.

Fixes #2428.
validate_key_format exists as a separate copy in two modules — src/memos/api/utils/api_keys.py (canonical home, used by the key-creation utilities) and src/memos/api/middleware/auth.py (the middleware that gates every authenticated request). Both used `int(hex_part, 16)` to verify the 64-char hex tail, which silently passes for:

- a leading `+` or `-` sign (`int("+a..a", 16)` == ...),
- a `0x` / `0X` prefix (`int("0xa..", 16)` == ...),
- `_` separators (PEP 515: `int("a_a", 16)`),
- surrounding whitespace.

So the documented format `krlk_<64-lowercase-hex>` was not actually enforced — keys like `krlk_+a..a`, `krlk_0x..`, or `krlk_a_a` were accepted by the middleware, hashed verbatim, and looked up against the database, bypassing the canonical form that `generate_api_key()` produces.

This commit:

- Replaces the `int(hex_part, 16)` check in the canonical api_keys.py
  implementation with an anchored regex (`[0-9a-f]{64}`) so only the exact
  lowercase 64-hex form is valid.
- Imports the fixed implementation into auth.py and keeps the public
  `validate_key_format` name as a thin re-export, so the middleware
  shares the same logic as the canonical utils copy. This prevents the
  two copies from drifting again — the original bug class.

New tests/api/test_validate_key_format.py covers 21 cases:

- canonical format and `generate_api_key()` output always pass
- 15 non-canonical variants are rejected (signs, 0x prefix, underscore,
  uppercase, whitespace, wrong length, missing prefix, empty)
- both copies (utils and middleware) enforce the same contract

Fixes #2427.
…mit bucket (#2428) (#2490)

## Summary

`_get_client_key` in `src/memos/api/middleware/rate_limit.py` tested
whether `request.headers["Authorization"]` started with `krlk_`, but
real clients send `Authorization: Bearer krlk_<hex>` per RFC 6750. The
prefix check never matched and every request — including ones carrying a
valid API key — fell through to IP-based limiting.

## Root cause

```python
auth_header = request.headers.get("Authorization", "")
if auth_header.startswith("krlk_"):    # ← false for "Bearer krlk_..."
    return f"ratelimit:key:{auth_header[:20]}"
```

Real `Authorization` values are `Bearer <token>`; the `Bearer ` prefix
is what made this branch unreachable. Behind a shared egress IP all
keyed clients shared one bucket and the per-key rate-limit was dead
code.

## Fix

Strip the `Bearer ` scheme (case-insensitive, per RFC 7235) before the
prefix check, so `Bearer krlk_<hex>` correctly maps to
`ratelimit:key:<first 20 of stripped key>`. Backwards compatibility is
preserved: a bare `krlk_<hex>` header still matches.

## Tests

New `tests/api/test_rate_limit_bearer.py`, 7 cases:

1. `test_bearer_krlk_key_uses_per_key_bucket` — red against unpatched
`main`, green with the fix
2. `test_bearer_krlk_key_first_twenty_chars_of_actual_key` — confirms
the key slice
3. `test_lowercase_bearer_scheme` — RFC 7235 case-insensitivity
4. `test_bearer_with_non_key_token_falls_back_to_ip` — non-API-key
Bearer tokens still go to IP
5. `test_missing_authorization_header_falls_back_to_ip`
6. `test_x_forwarded_for_header_preferred_over_ip`
7. `test_krlk_prefix_without_bearer_scheme_still_matches` — backwards
compat

## Verification

```
PYTHONPATH=src pytest tests/api/test_rate_limit_bearer.py -v
============================== 7 passed in 2.50s ==============================
```

Ruff: `ruff format --check` and `ruff check` both clean on the touched
test file; the source file has the same 2 pre-existing BLE001 findings
as on `main` (this PR introduces 0 new lint issues).

## Out of scope

- No new dependencies
- No public API / OpenAPI contract changes
- No DB schema, migration, or graph model changes
- No Makefile / pyproject.toml / pre-commit / GitHub workflow changes

Closes #2428
…#2491)

## Summary

`validate_key_format` exists as a separate copy in two modules. Both
used `int(hex_part, 16)` to verify the 64-char hex tail, which silently
passes for leading signs, `0x` prefixes, `_` separators, and surrounding
whitespace — so the documented format `krlk_<64-lowercase-hex>` was not
actually enforced at the middleware boundary.

## Root cause

Two copies of the same buggy validator:

- `src/memos/api/utils/api_keys.py:48` — used by the key-creation
utilities
- `src/memos/api/middleware/auth.py:65` — gates every authenticated
request

Both check the tail via `int(hex_part, 16)`. CPython accepts:

| input | `int(..., 16)` result |
| ----- | ----------------- |
| `+aaaa…` (64 chars) | valid |
| `0xaaaa…` (62 chars) | valid |
| `aaaa_bbbb…` (with `_`) | valid (PEP 515) |
| ` AAAA…` / `AAAA… ` | valid |

So `krlk_+a..a`, `krlk_0x..`, `krlk_a_a`, `krlk_A..A`, etc. were all
accepted by the middleware, hashed verbatim, and looked up against the
database — bypassing the canonical form that `generate_api_key()`
produces (lowercase, no prefixes, no separators, no signs, no
whitespace).

## Fix

Two changes that together eliminate the bug class:

1. **Canonical validator** — `src/memos/api/utils/api_keys.py`: replace
the `int(hex_part, 16)` check with an anchored regex
`re.fullmatch(r"[0-9a-f]{64}", hex_part)`. Only the exact
64-lowercase-hex form is valid.
2. **Dedup** — `src/memos/api/middleware/auth.py`: import the canonical
implementation under the name `validate_key_format_canonical`, keep the
public symbol `validate_key_format` as a one-line re-export. This
prevents the two copies from drifting again (the original bug class).

## Tests

New `tests/api/test_validate_key_format.py`, 21 cases:

- canonical format and `generate_api_key()` output (20 random samples)
always pass
- 15 non-canonical variants are rejected: signs, `0x` prefix, underscore
separators, uppercase hex, leading/trailing whitespace, wrong length,
missing `krlk_` prefix, `Bearer ` prefix, empty string
- both `utils` and `middleware` copies enforce the same contract

## Verification

```
PYTHONPATH=src pytest tests/api/test_validate_key_format.py -v
============================== 21 passed in 3.31s ==============================
```

Ruff: `ruff format --check` and `ruff check` both clean on touched files
(0 new findings vs `main`; the 3 pre-existing `DTZ`/`datetime.utcnow`
warnings in api_keys.py are out of scope).

## Out of scope

- No new dependencies
- No public API / OpenAPI contract changes
- No DB schema, migration, or graph model changes
- No Makefile / pyproject.toml / pre-commit / GitHub workflow changes

Closes #2427
## Description

Stopping a scheduler that has never initialized RabbitMQ logs
`AttributeError: 'OptimizedScheduler' object has no attribute
'_io_loop_thread'` during API shutdown. This includes local-queue
deployments and initialization skipped in CI. No restart is required to
trigger it.

The constructor initializes `_rabbitmq_io_loop_thread`, but both
`initialize_rabbitmq()` and `rabbitmq_close()` use `_io_loop_thread`.
Initialize the same field in the constructor. The production change is
one line; connection cleanup, thread joins, shutdown ordering, and
exception handling are unchanged. No new dependencies or configuration
are required.

Related issue: N/A; the reproduction and regression tests are included
here.

## Type of change

- [x] Bug fix (non-breaking)

## How Has This Been Tested?

- [x] Unit tests: `pytest tests/mem_scheduler/test_rabbitmq_shutdown.py
-q` — **5 passed**. Without the fix, **3 failed, 2 passed**. Covers
uninitialized/repeated close, CI initialization skip, connection cleanup
with a real IO thread, native optimized-scheduler lifecycle cleanup, and
propagation of unexpected join errors.
- [x] Test steps: isolated API shutdown with RabbitMQ uninitialized,
followed by restart and a second shutdown — both exits were clean;
graph/vector/business snapshots were unchanged. No generation or
embedding requests were made.
- [x] `make format` — all Ruff checks passed; 631 files unchanged.
- [x] Pre-commit checks on both changed files — passed.

Additional directory run: `pytest tests/mem_scheduler/ -q` — **70
passed, 1 skipped, 1 failed**. The existing
`test_dynamic_cache_layers_access` requires PyTorch, which is absent in
the test image; the same failure was reproduced against the unmodified
source. Pytest also warns that the image lacks the asyncio plugin. The
full repository suite was not run.

## Checklist

- [x] Self-reviewed the one-line fix and focused tests.
- [x] Added tests demonstrating the failure and preserving existing
cleanup behavior.
- [x] No documentation/API/schema changes are needed for this internal
field-name correction.
## Description

Fixes #2550.

PolarDB lightweight vector-search projections can return a numeric user
ID as an already-decoded string. Decoding it again with `json.loads`
turns it into an integer, causing `TextualMemoryItem` validation to fail
and potentially leaving knowledge-base retrieval empty after successful
ingestion.

Preserve identifier strings (including leading zeros), normalize native
integer identifiers to strings, and keep JSON-quoted strings and nulls
compatible. Non-identifier arrays, objects, booleans and numeric values
retain their existing decoding. The identifier field set is a
module-level frozenset, avoiding per-field allocations during recall. No
dependency, schema or route changes.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

- [x] Unit Test: `poetry run pytest tests/graph_dbs/ -q` → 77 passed, 3
skipped.
- [x] Test Script Or Test Steps: `make test` → 1005 passed, 9 skipped;
`make format` and changed-file pre-commit passed.
- [x] Pipeline verification: all 16 Python tests CI jobs passed on final
head `b1a7d5fee60c06923933550c9c9c0755c54e383c` (Linux, Windows, macOS
14/15; Python 3.10–3.13). Live cloud API testing is not applicable
before deployment.

Regression cases exercise a mocked database row through real lightweight
vector search and real recall-item construction, covering digit strings,
native integers, JSON-quoted strings, leading zeros and nulls. Before
the fix, the numeric string/integer cases fail with Pydantic
`metadata.user_id` string-type errors (2 failed, 26 passed); after the
fix, all relevant tests pass. All fixtures are synthetic and external
services are mocked.

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [x] Documentation issue/PR: not applicable; no public API changes.
- [x] I have linked the issue to this PR: #2550.
- [x] Reviewer mentioned: @endxxxx.

## Reviewer Checklist

- [x] Closes #2550.
- [x] Made sure Checks passed — all 16 required CI jobs passed on the
final head.
- [x] Tests have been provided.
@syzsunshine219
syzsunshine219 merged commit 3a16e5f into main Nov 9, 2026
25 of 33 checks passed
@syzsunshine219
syzsunshine219 deleted the dev-v2.0.36 branch November 9, 2026 12:30
@Memtensor-AI
Memtensor-AI restored the dev-v2.0.36 branch November 19, 2026 06:10
@Memtensor-AI
Memtensor-AI deleted the dev-v2.0.36 branch November 19, 2026 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants