Repository navigation
Conversation
chunk() built a Chunk and then rebound it to the bare restored-URL string, so the method always returned list[str] while discarding token_count and sentences. Restore the URLs into the Chunk's text and return the object. Fixes MemTensor#2433
🤖 Open Code ReviewTarget: PR #2434 🔍 OpenCodeReview found 1 issue(s) in this PR. 1.
|
|
|
This PR has been automatically marked as stale due to inactivity. |
|
This PR has been automatically closed due to inactivity. |
Fixes #2433
Summary
SentenceChunker.chunkinsrc/memos/chunkers/sentence_chunker.pybuilds a properChunkand then immediately overwrites it with the restored-URL string:So the method always returns
list[str]despite declaringlist[str] | list[Chunk]:token_countandsentencesare silently discarded, and callers using the documentedChunkAPI (.text/.token_count/.sentences) get anAttributeError. The existingtest_sentence_chunker.pymasks this by accepting both return types.Fix
Restore the URLs into the
Chunk's text and append the object:Regression test
Added
tests/chunkers/test_sentence_chunker_chunk_objects.py:chunks[0]is astr) and passes with the fixChunkwith restored URL text plus preservedtoken_countandsentencestests/chunkers/suite (25 tests) passes with it, and the test's package stub only fills in missing modules so it does not shadow the real package for other testsruff check(repo-pinned 0.11.8) passes on both touched files.AI Disclosure