Round-trip vectorsdb collection dimension through migrations#210
Merged
Conversation
A restored vectorsdb collection lost its dimension: the Collection resource never carried it, so archives dropped it at write time, and the destination both created the per-database metadata collection from the generic databases schema (which lacks the attribute) and wrote collection documents without it. Restored collections reported dimension null and creating additional collections on a restored vectorsdb database failed structure validation. Collection now carries a nullable dimension (serialized and exported from the Appwrite source), and the destination accepts an optional collectionStructures map keyed by database type so vectorsdb databases get their own metadata schema. Types with an entry get dimension written on entity import; archives written before dimension was serialized get a placeholder that syncVectorDimension corrects from the vector column size when the embeddings field is imported. DocumentsDB/VectorsDB fromArray also stop warning on entity lines whose nested database object predates the database DSN key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Greptile SummaryThis PR preserves vectorsdb collection dimensions during Appwrite migrations. The main changes are:
Confidence Score: 4/5The vectorsdb restore path can persist the wrong collection dimension when multiple vector attributes are imported.
src/Migration/Destinations/Appwrite.php, src/Migration/Resources/Database/Collection.php Important Files Changed
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
src/Migration/Destinations/Appwrite.php:1332
**Vector Dimension Overwritten**
When a vectorsdb collection has more than one vector attribute, this sync runs for each one and writes the collection-level `dimension` from the current attribute size. A later non-`embeddings` vector field with a different size can overwrite the restored collection metadata, so new inserts or collection operations validate against the wrong dimension.
### Issue 2 of 2
src/Migration/Resources/Database/Collection.php:76
**Null Dimension Changes Archives**
`Collection::jsonSerialize()` now writes `dimension: null` for every collection that does not set a dimension, including non-vectors collections and older archive data. Any strict archive reader or downstream schema that only accepts the previous collection shape can reject otherwise valid non-vectors collection exports because the field is present instead of omitted.
Reviews (1): Last reviewed commit: "feat: round-trip vectorsdb collection di..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| { | ||
| if ( | ||
| $type !== UtopiaDatabase::VAR_VECTOR | ||
| || $resource->getTable()->getDatabase()->getType() !== Resource::TYPE_DATABASE_VECTORSDB |
Contributor
There was a problem hiding this comment.
When a vectorsdb collection has more than one vector attribute, this sync runs for each one and writes the collection-level dimension from the current attribute size. A later non-embeddings vector field with a different size can overwrite the restored collection metadata, so new inserts or collection operations validate against the wrong dimension.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/Migration/Destinations/Appwrite.php
Line: 1332
Comment:
**Vector Dimension Overwritten**
When a vectorsdb collection has more than one vector attribute, this sync runs for each one and writes the collection-level `dimension` from the current attribute size. A later non-`embeddings` vector field with a different size can overwrite the restored collection metadata, so new inserts or collection operations validate against the wrong dimension.
**Knowledge Base Used:**
- [Destinations](https://app.greptile.com/appwrite/-/custom-context/knowledge-base/utopia-php/migration/-/docs/destinations.md)
- [Resource Model Classes](https://app.greptile.com/appwrite/-/custom-context/knowledge-base/utopia-php/migration/-/docs/resources.md)
How can I resolve this? If you propose a fix, please make it concise.
Member
Author
There was a problem hiding this comment.
Fixed: syncVectorDimension now only writes when the collection's dimension is still the placeholder (0), so it fills in the value for pre-dimension archives but never overwrites an archived dimension — a second vector column import is a no-op.
| public function jsonSerialize(): array | ||
| { | ||
| return array_merge(parent::jsonSerialize(), [ | ||
| 'dimension' => $this->dimension, |
Contributor
There was a problem hiding this comment.
Collection::jsonSerialize() now writes dimension: null for every collection that does not set a dimension, including non-vectors collections and older archive data. Any strict archive reader or downstream schema that only accepts the previous collection shape can reject otherwise valid non-vectors collection exports because the field is present instead of omitted.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/Migration/Resources/Database/Collection.php
Line: 76
Comment:
**Null Dimension Changes Archives**
`Collection::jsonSerialize()` now writes `dimension: null` for every collection that does not set a dimension, including non-vectors collections and older archive data. Any strict archive reader or downstream schema that only accepts the previous collection shape can reject otherwise valid non-vectors collection exports because the field is present instead of omitted.
**Knowledge Base Used:**
- [Resource Model Classes](https://app.greptile.com/appwrite/-/custom-context/knowledge-base/utopia-php/migration/-/docs/resources.md)
- [Migration Sources](https://app.greptile.com/appwrite/-/custom-context/knowledge-base/utopia-php/migration/-/docs/sources.md)
How can I resolve this? If you propose a fix, please make it concise.
Member
Author
There was a problem hiding this comment.
Fixed: jsonSerialize now omits dimension when unset, so documentsdb and other non-vector collection lines keep their previous shape.
This was referenced Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Resources\Database\Collectioncarries a nullabledimension(fromArray+jsonSerialize), and the Appwrite source exports it when listing tables, so archive entity lines preserve it.Destinations\Appwriteaccepts an optionalcollectionStructuresconstructor map keyed by database type. When the imported database's type has an entry,createDatabasebuilds the per-database metadata collection from that structure instead of the generic one, andcreateEntitywrites the type-specific metadata (vectorsdbdimension).dimensionwas serialized carry none, socreateEntitywrites a schema-satisfying placeholder andsyncVectorDimensioncorrects it from the vector column's size when theembeddingsfield is imported.DocumentsDB/VectorsDB::fromArrayno longer warn when the nested database object lacks thedatabaseDSN key (older archive lines).Why
Restoring a vectorsdb backup produced a working database whose collection reported
dimension: null, and creating additional collections on it failed structure validation: the vectorsdb metadata schema requiresdimensionbut the destination created the metadata collection from the genericdatabasesschema and never wrote the value. Companion to appwrite-labs/cloud#4908.Existing callers are unaffected: without a
collectionStructuresentry, behavior is unchanged.Tests
tests/Migration/Unit/Resources/CollectionTest.phpcovers the dimension round trip (fromArray → jsonSerialize → fromArray) and the null default. Full unit suite green, Pint and PHPStan (level 3) clean. End-to-end coverage lands in cloud'stestVectorsDBBackupAndRestore.🤖 Generated with Claude Code