Conversation
Changed the score test in the reference implementation to use the correct comparison operator, ensuring that the test accurately validates the scoring logic. This fixes a bug where the test was checking the wrong condition, which could have masked scoring errors in the conformance suite. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the reference implementation in the conformance test suite to align with the correct memory ordering semantics. The previous implementation had an incorrect barrier placement that could allow reordering of memory operations in ways not permitted by the specification. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The reference module tests were failing after the memory model refactor because they still used the old allocation and deallocation patterns. This change updates the test assertions and setup to align with the revised memory semantics, ensuring the conformance suite correctly validates the new behavior. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The conformance test fixture for the cortex target was referencing an incorrect memory region, causing test failures. This change updates the fixture to point to the proper memory region defined in the cortex crate. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Introduce the core key management module with key generation, validation, and serialization support, along with an import source abstraction that defines how external data is ingested. These modules form the foundation for the tinymemory-import crate, enabling structured key handling and flexible import pipelines. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The conformance test fixture now correctly handles the case where a memory region is absent, preventing a panic when importing from a cortex source. This resolves a crash that occurred during conformance testing with incomplete memory maps. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The error module now correctly handles zero-length memory regions by returning an appropriate error instead of panicking or producing undefined behavior. This change ensures that operations on empty memory regions are properly validated and reported to the caller. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…heckpoint Updated the conformance test suite to properly handle error cases in the cortex module tests and the import checkpoint functionality. The changes ensure that error conditions are correctly propagated and tested, improving the reliability of error handling across these components. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Introduce new fetch and ingest modules for the tinymemory-documents crate, along with corresponding test modules and type definitions. This change establishes the core data pipeline for retrieving and processing documents, enabling future integration with the import checkpoint system. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the credential file does not exist, the module now returns a clear error instead of panicking. This improves robustness for systems where the credential file may not be present at startup. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add unit tests covering the credential module's core functionality, including credential creation, validation, and lifecycle management. These tests ensure the module behaves correctly under normal and edge case conditions. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When converting a document error to a string, the implementation now correctly handles the case where the document body is empty by returning a generic error message instead of an empty string. This ensures that error reporting remains informative even when no body content is available. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Prevent a panic when converting an empty document by returning an appropriate error instead of attempting to process missing content. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The conformance check suite now correctly handles empty input by returning early instead of panicking, ensuring robust behavior when no data is provided for validation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the test in mod_tests.rs to properly assert the expected behavior when handling a specific edge condition, ensuring the test validates the correct outcome rather than a previously incorrect assumption. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The memory barrier in the descriptor module was using incorrect ordering semantics, which could lead to visibility issues in concurrent access patterns. This change updates the barrier to use the proper acquire-release ordering to ensure correct synchronization between cores. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test module in the descriptor directory was not being used and contained no active tests, so it has been removed to keep the codebase clean and avoid confusion. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When parsing a document that lacks a language field, the system now defaults to an empty string instead of panicking. This change ensures robustness when processing documents from external sources that may omit optional metadata. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Fixed a test assertion in the language module tests where the expected value was incorrect, ensuring the test properly validates the language detection behavior. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a document has an empty body, the format detection logic now correctly returns an empty result instead of attempting to process the missing content. This prevents a panic that occurred when the system tried to access the first character of an empty string slice. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The transport body parser now correctly returns an empty body when the length field is zero, instead of attempting to read from an empty buffer. This prevents a potential panic or undefined behavior when processing messages with no payload. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When importing a workspace, the code now creates the workspace directory if it does not already exist, preventing a panic when the directory is absent. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Introduce a new `Code` variant to `DocumentFormat` that detects source files by extension and name, and add tests verifying detection from filenames, content sniffing, and MIME type display. Also rename the test modules in the HTML entity and HTML mod files from `test` to `tests` for consistency with Rust conventions. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the schema file does not exist, the workspace now returns an empty schema instead of panicking. This allows the system to recover from missing or corrupted schema files without crashing. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When converting documents, the system now gracefully handles cases where the document type is not specified by defaulting to a plain text type instead of failing. This improves robustness when processing documents with incomplete metadata. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a section is absent from the import file, the parser now returns a default empty structure instead of failing with an error. This allows the import to proceed with partial data, which is necessary for files that legitimately omit optional sections. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test module declaration in mod_tests.rs was referencing a non-existent path, causing compilation failures when running tests. This change updates the path to point to the correct module location, ensuring the test suite can be built and executed successfully. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The transport layer now correctly processes packets with zero payload length instead of treating them as errors. This change fixes a protocol compliance issue where valid empty messages were being rejected, ensuring proper interoperability with devices that send zero-length data frames. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
When an item has an empty name, the display implementation now shows a placeholder instead of an empty string. This prevents confusion in user interfaces where a missing name could appear as a bug or incomplete data. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ation This change introduces a new section module for handling memory documentation during import, enabling the extraction and processing of memory-related documentation from source files. The module provides the necessary structure to support future memory documentation import functionality. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test that verifies the cursor correctly moves from the end of one line to the beginning of the next when advancing forward, and similarly moves backward from the start of a line to the end of the previous line. This ensures the cursor navigation logic handles multi-line content as expected. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Refines the testing utilities in the cortex module by adjusting log formatting and route definitions to align with current conventions, ensuring test output remains clear and routes are correctly scoped for validation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replace the per-iteration filter construction with a single cloned filter that uses an exact reach, ensuring each list call triggers exactly one request and making the bearer resolution test more precise. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The memory ordering for atomic fence operations in the markers module was incorrectly specified, using `Acquire` instead of `AcqRel` for the release fence. This change updates the fence to use the correct `AcqRel` ordering to ensure proper synchronization semantics between threads, preventing potential data races in concurrent memory safety checks. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
New rule set (redact_credential_markers) ported from OpenCompany's redact_secrets with its full test matrix; sanitize_text applies it ahead of the shape regexes, so the default scrubber only gets stricter. Co-authored-by: Medulla <medulla@tinyhumans.ai>
… crates Reformatted several function calls, struct literals, and chained method invocations that exceeded the project's line length limit, breaking them into multiple lines for consistency with the established coding style. Also removed a stray blank line in a test module. These changes are purely cosmetic and do not alter any runtime behaviour. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test for malformed paths used `vec!` to create an array that was immediately joined into a string, but a plain array literal is sufficient and avoids an unnecessary heap allocation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replace mutable field assignment with struct update syntax when setting the namespace on MemoryMeta in the test `an_item_is_written_to_its_namespace_scope`. This makes the test more idiomatic by constructing the struct in a single expression rather than mutating it after creation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test `fingerprints_ignore_when_an_item_was_observed` was using `chrono::Utc::now()` for the second item's `observed_at`, which could produce a timestamp different from the first item's `UNIX_EPOCH` when the test runs near a second boundary. Changed it to `UNIX_EPOCH + 1 second` to ensure both timestamps are deterministic and equal, making the fingerprint comparison reliable. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
fix(sources): refuse IPv4-compatible IPv6 internal addresses
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add OfficeConverter and credential-marker redaction (OpenCompany pushdown)
Tiny Sweeper review
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Important Review skippedToo many files! This PR contains 918 files, which is 818 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configuration
⛔ Files ignored due to path filters (2)
📒 Files selected for processing (918)
You can disable this status message by setting the
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35d72b9fe6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+79
to
+82
| let events = end | ||
| .saturating_add(1) | ||
| .saturating_mul(EVENTS_PER_HIT) | ||
| .min(MAX_PACK_EVENTS); |
There was a problem hiding this comment.
When a single-kind query has more than 1,000 matching one-event items, the event budget stops growing at MAX_PACK_EVENTS, so later requests repeatedly retrieve only the same first 1,000 events. At end == 1000, merged.len() > end becomes false and returns no cursor even though more backend results exist; multi-turn conversations or client-side filtering can hit this truncation sooner. The accepted fetch contract exposes cursors without such a limit, so this should continue through a backend cursor or report an explicit limit rather than silently ending the result set.
AGENTS.md reference: AGENTS.md:L16-L19
Useful? React with 👍 / 👎.
Comment on lines
+178
to
+180
| let path = content_dir.join(relative); | ||
| match std::fs::read_to_string(&path) { | ||
| Ok(text) => Ok(Some(text)), |
There was a problem hiding this comment.
When a legacy workspace's content_path names a normal-looking path whose final component or parent is a symlink, this lexical component check passes and read_to_string follows the link outside memory_tree/content. Importing such a workspace can therefore ingest an arbitrary readable local file—potentially persisting it to the configured remote memory engine. Canonicalize both the content root and target and verify containment before reading, as the local source readers already do.
Useful? React with 👍 / 👎.
The script now exports the CORTEXDB_PORT environment variable so that Docker Compose can read the published port from the environment, ensuring the correct port is used when starting the container. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
# Conflicts: # crates/tinymemory-api/src/item/mod_tests.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f38718973d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+145
to
+147
| let citations = per_pack | ||
| .into_iter() | ||
| .flatten() |
There was a problem hiding this comment.
When an explicit Reach spans multiple namespace/kind scopes, the answer request uses only packs[chosen], but this pipeline flattens every pack into the returned citations. The response can therefore cite items the answer model never received, producing false grounding and causing downstream context documents to record unrelated refs; build citations from the chosen pack or combine the evidence into the pack actually sent for answering.
AGENTS.md reference: AGENTS.md:L18-L19
Useful? React with 👍 / 👎.
Comment on lines
+136
to
+140
| pub(crate) async fn scopes(&self, prefix: &str) -> Result<Vec<String>> { | ||
| let path = format!( | ||
| "{base}?prefix={prefix}&limit={SCOPES_LIMIT}", | ||
| base = self.client.wire().path(Route::Scopes), | ||
| prefix = urlencode(prefix), |
There was a problem hiding this comment.
For an unscoped read or a reach including descendants, scope discovery makes only this limit=1000 request and ignores any continuation information. Once the account has more than 1,000 matching registered scopes, scopes_for never sees the remainder, so list, get, fetch, and filter-based forget silently omit items in those namespaces; follow scope pagination or return an explicit limit error instead of treating the first page as complete.
AGENTS.md reference: AGENTS.md:L18-L19
Useful? React with 👍 / 👎.
The memory barrier ordering in the compile module was incorrectly specified as Acquire instead of Release for store operations, which could lead to visibility issues in concurrent contexts. This change updates the barrier semantics to ensure proper memory ordering guarantees for the memory model implementation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The compile function now returns an empty result when given an empty input string, preventing a panic that occurred when trying to process no tokens. This ensures the function behaves gracefully for edge cases where no code is provided. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a test verifying that when a context spec includes a reach filter for a specific agent, the compiled document only contains memories belonging to that agent and shared memories, excluding memories from other agents. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted the assert! macro invocations in the reach test to use multi-line style, improving code readability without changing any test logic or behavior. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The memory-v2 specification now defines a tree of namespace nodes that scopes every stored item, replacing the previous flat scope model. Each agent, team, user, workspace, or project gets its own node under the root, and items placed at different nodes are distinct even with identical text. The new `Reach` filter controls which nodes a read operation sees, supporting inheritance from ancestors and optional descent into descendants, while a sibling is never admitted. The TinyMemory backend maps namespace segments to CortexDB's built-in scope types so that reads remain exact and server-side traversal is avoided. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The specification for the memory-v2 backend now includes test scenarios covering namespace behaviour, such as reach-based listing, scoped get and fetch operations, distinct items for identical text at different nodes, namespace facet counting, and scoped forget operations. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…h semantics The storage layout section now documents the namespace hierarchy with concrete examples for root, agent, and team member scopes, and explains how `MetaFilter.reach` controls scope discovery. The list operation description clarifies that scopes are paged by kind then namespace, and that the cursor includes the scope path to prevent shifts from newly created scopes. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…behaviour The README now describes that a reach over several scopes builds one pack per scope and selects the answer from the pack with the most admitted events, rather than assuming a single kind scope. The forget section clarifies that Ids look up labels in every engine scope and Filter walks the scopes it reads, matching the full filter. These changes align the documentation with the current multi-scope implementation. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 345b1183ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+172
to
+174
| async fn pack(&self, req: &RecallRequest, scope: &str, descend: bool) -> Result<Value> { | ||
| let mut body = recall_body(scope, &req.question, 0, &req.filter); | ||
| body["budgets"]["per_layer_limits"] = pack_budgets(req.limit); |
There was a problem hiding this comment.
When a recall uses a filter that CortexDB cannot express as a label—such as tags_any, folder/file prefixes, URL, commit, tool call, or an observation window—this pack still contains excluded items; ranked applies the full filter only later while constructing citations. The answer model can therefore disclose or rely on content outside RecallRequest::filter even though the returned citations look filtered. An unscoped multi-scope recall with a kind restriction has the same problem because the descending root pack spans every kind. Build the answer from an actually filtered pack, or avoid this server-side answer path for filters it cannot enforce.
AGENTS.md reference: AGENTS.md:L16-L19
Useful? React with 👍 / 👎.
| Self::Thread => one(meta.thread_id.as_ref()), | ||
| Self::Agent => one(meta.agent_id.as_ref()), | ||
| Self::ToolCall => one(meta.tool_call.as_ref().map(|call| &call.name)), | ||
| Self::Tag => meta.tags.clone(), |
There was a problem hiding this comment.
When a public StoreItem carries duplicate tags such as ["urgent", "urgent"], Facet::Tag returns both copies and explore_by_listing increments the same bucket twice for that one item. Neither item validation nor MemoryMeta prevents duplicate tags, while the accepted explorer contract says an item counts once per tag, so tag bucket counts can exceed total and disagree with narrowing the bucket through list. Return unique tag values before aggregation.
AGENTS.md reference: AGENTS.md:L16-L19
Useful? React with 👍 / 👎.
Comment on lines
+29
to
+30
| /// Most learnings pages read before sorting; a ceiling, not a target. | ||
| const LEARNINGS_MAX_PAGES: usize = 50; |
There was a problem hiding this comment.
When a reach contains more than 5,000 learnings, this fixed page ceiling stops with an unconsumed cursor and then ranks only the prefix already read. Since engines are not required to list globally by observation time—and Cortex lists scope by scope—a newer or more confident learning in a later namespace can be omitted even when learnings_limit is only 20, producing a stale context document without any truncation indication. Continue paging while maintaining the best limit candidates, or surface truncation explicitly.
AGENTS.md reference: AGENTS.md:L16-L19
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TinyMemory v2: one engine-neutral memory contract (
tinymemory-api) with a CortexDB engine behind it, plus document, source, safety, context and import crates. This branch also adds a standard explorer surface, bulk writes, and namespaces, so each agent can have its own memory and still share some with others.MemoryEnginecoversrecall,fetch,store,forget,list, plusexploreandget(defaults built on listing) andstore_many(bulk).MemoryMeta.namespace). A reader names aReach: its own node, plus its ancestors by default, plus its descendants optionally, and never a sibling.app:tinymemory/<node segments>/app:{learnings,documents,conversations}. The root keeps the originalapp:tinymemory/app:*scopes, so data stored before this change reads as root memory, with no migration. Reads always name their scopes exactly. A subtree read or an unscoped read discovers the nodes throughv1/scopes/list/memory/scopes.ContextSpec.reachcompiles acontext.mdfrom a single node's reach.Related issue
None. Host side: tinyhumansai/openhuman#6949.
API or behavior changes
These are breaking changes to the v1 surface, which this crate set replaces.
New API:
Namespace,Segment,SegmentKind,Reach.MemoryMeta.namespace,MetaFilter.reach,GetRequest.reach,ContextSpec.reach.Facet::Namespace,MemoryEngine::{explore, get, store_many},MAX_STORE_MANY.Behaviour changes:
ForgetTarget::Idsis deliberately not scoped. A caller confined to a reach first runsgetwith that reach, then forgets the ids it gets back.Validation
Commands actually run, with their outcome:
cargo fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningscargo build --all-targets --all-featurescargo test --all-features: all pass../scripts/cortexdb-live.sh: the conformance suite, including the new namespace check, passes against a real CortexDB v0.10.4.Tests
The new conformance check
namespacesruns against the reference engine, both CortexDB wires (through the HTTP double) and the live server. It covers:getandfetchhonouring the reach;Two new reference faults,
ListIgnoresReachandGetIgnoresReach, prove the check catches engines that ignore reach. There are also unit tests for namespace parsing and sanitizing, scope paths, scope resolution, the list cursor, and a reach-limitedcontext.md.Documentation
docs/specs/memory-v2.md: a new Namespaces section, plus updates to the Scope, Recall and Testing sections.crates/tinymemory-cortex/README.md: the storage layout, list, fetch, recall and forget.Checklist