fix(plugin-import-export): export relationship fields natively for JSON format - #17512
Open
nathanlentz wants to merge 11 commits into
Open
fix(plugin-import-export): export relationship fields natively for JSON format#17512nathanlentz wants to merge 11 commits into
nathanlentz wants to merge 11 commits into
Conversation
…ON format hasMany and polymorphic relationship fields were flattened into field_N_id / field_N_relationTo sibling keys for every export format, but only CSV import reverses that flattening. Importing a collection's own JSON export therefore dropped every hasMany and polymorphic relationship. The export hooks now check the format and return the relationship value as-is for JSON — an id, an array of ids, or relationTo/value pairs. CSV output is unchanged.
Four of the five relationship tests either paraphrased the implementation's null guards or duplicated the JSON roundtrip coverage already in test/plugin-import-export/int.spec.ts, and all five went through a local helper that hand-rolled the beforeExport arg object behind an untyped cast. Kept the one case nothing else covers: an orphaned entry must leave a gap in the CSV columns so surviving relationships keep their source index. It now runs through applyFieldHooks, the real caller, so the arg shape comes from production code rather than the test.
Contributor
📦 esbuild Bundle Analysis for payloadThis analysis was generated by esbuild-bundle-analyzer. 🤖
Largest pathsThese visualization shows top 20 largest paths in the bundle.Meta file: packages/next/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js
Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js
DetailsNext to the size is how much the size has increased or decreased compared with the base branch of this PR.
|
paulpopus
requested changes
Jul 27, 2026
paulpopus
left a comment
paulpopus
left a comment
Member
There was a problem hiding this comment.
In addition to the comments below:
I want you to double check in the UI and if there are any changes to the preview table and if there are, then write e2e tests covering for regressions in this area
nathanlentz
requested review from
AlessioGr,
JarrodMFlesch and
jacobsfletch
as code owners
July 28, 2026 15:08
nathanlentz
commented
Jul 28, 2026
Collaborator
Author
Yes, the preview table changed. I created a new way to present those relationships for JSON imports and added e2e tests. |
nathanlentz
added a commit
that referenced
this pull request
Jul 28, 2026
…llection Ports the review feedback from #17512: renders relationship and upload preview cells through a dedicated RelationshipCell grouped by target collection, drops dangling references from exports rather than exporting nulls the import would reject, and adds unit, integration, and e2e coverage.
paulpopus
requested changes
Jul 29, 2026
Comment on lines
+61
to
+84
| const result = applyFieldHooks({ | ||
| type: 'beforeExport', | ||
| // Exports populate at depth 1, so `value: null` is an orphaned reference — | ||
| // the target doc was deleted out from under it. | ||
| data: { | ||
| rel: [ | ||
| { relationTo: 'users', value: null }, | ||
| { relationTo: 'posts', value: 'p1' }, | ||
| ], | ||
| }, | ||
| fieldHooks: getExportFieldFunctions({ fields }), | ||
| fields, | ||
| format: 'csv', | ||
| operation: 'export', | ||
| req: mockReq, | ||
| }) | ||
|
|
||
| // The surviving entry stays at index 1 — shifting it to 0 would silently | ||
| // rewrite column names for every consumer of the CSV. | ||
| expect(result).toEqual({ | ||
| rel: null, | ||
| rel_1_id: 'p1', | ||
| rel_1_relationTo: 'posts', | ||
| }) |
Member
There was a problem hiding this comment.
This test doesn't exercise the CSV pipeline it's named for.
This drives the hasMany-polymorphic handler through applyFieldHooks with format: 'csv', but real CSV export never calls applyFieldHooks — it exclusively uses flattenObject, and the two disagree on what a hook's null return means for an array field:
applyFieldHooks.ts:if (typeof transformed !== 'undefined') { result[field.name] = transformed }→ writesrel: null.flattenObject.ts(array branch):if (result === null) { return }→ exits without ever writingrow[fieldPath], sorelis absent from the row entirely.
The asserted shape { rel: null, rel_1_id: 'p1', rel_1_relationTo: 'posts' } isn't what a real generated CSV row looks like — it would have no rel key at all. The column-pinning behavior (index 1 stays index 1) is validated correctly, but the rel: null half of the assertion wouldn't catch a real regression in flattenObject.
Suggest calling flattenObject directly here (matching what createExport.ts/export/handlePreview.ts actually do for CSV) and dropping/correcting the rel: null expectation.
Comment on lines
+408
to
+444
| describe('hasMany with gaps in the column indices', () => { | ||
| // Export pins each column to its source index, so an entry that could not be | ||
| // resolved to an id leaves a gap. Import must absorb the gap rather than | ||
| // emitting a null array entry. | ||
| const hasManyFields: FlattenedField[] = [ | ||
| { | ||
| name: 'rels', | ||
| type: 'relationship', | ||
| hasMany: true, | ||
| relationTo: ['posts', 'pages'], | ||
| } as FlattenedField, | ||
| ] | ||
|
|
||
| const expected = { | ||
| rels: [{ relationTo: 'posts', value: 'p1' }], | ||
| } | ||
|
|
||
| it('should absorb a leading gap left by an unresolvable entry', () => { | ||
| const data = { | ||
| rels_1_id: 'p1', | ||
| rels_1_relationTo: 'posts', | ||
| } | ||
|
|
||
| expect(unflattenObject({ data, fields: hasManyFields, req: mockReq })).toEqual(expected) | ||
| }) | ||
|
|
||
| it('should absorb a gap padded with empty strings by schema columns', () => { | ||
| const data = { | ||
| rels_0_id: '', | ||
| rels_0_relationTo: '', | ||
| rels_1_id: 'p1', | ||
| rels_1_relationTo: 'posts', | ||
| } | ||
|
|
||
| expect(unflattenObject({ data, fields: hasManyFields, req: mockReq })).toEqual(expected) | ||
| }) | ||
| }) |
Member
There was a problem hiding this comment.
This block covers only a leading gap (index 0 missing, or index 0 padded with empty strings; index 1 present). There's no case for a gap strictly between two present entries (index 0 present, index 1 missing, index 2 present). The current behavior is correct, but there's no test to ensure no regression in the future.
Comment on lines
+117
to
139
| registerHandler(({ format, siblingData, value }) => { | ||
| if (!Array.isArray(value)) { | ||
| return undefined | ||
| } | ||
| return undefined | ||
| // `index` is carried so CSV columns stay pinned to the source position: an entry | ||
| // that cannot be resolved to an id leaves a gap rather than shifting its siblings. | ||
| const rels = value.flatMap((val, index) => { | ||
| if (!isPolymorphicRelValue(val)) { | ||
| return [] | ||
| } | ||
| const id = getPolymorphicRelId(val) | ||
| return id === undefined ? [] : [{ id, index, relationTo: val.relationTo }] | ||
| }) | ||
|
|
||
| if (format === 'json') { | ||
| return rels.map((rel) => ({ relationTo: rel.relationTo, value: rel.id })) | ||
| } | ||
| rels.forEach(({ id, index, relationTo }) => { | ||
| siblingData[`${fullKey}_${index}_id`] = id | ||
| siblingData[`${fullKey}_${index}_relationTo`] = relationTo | ||
| }) | ||
| return null | ||
| }) |
Member
There was a problem hiding this comment.
No test (unit or integration) exercises the hasMany-polymorphic + JSON + dangling-reference combination for this handler. Suggest adding a unit test similar to the existing CSV "pin columns" test above but with format: 'json', asserting the dangling entry is dropped and the surviving entry's {relationTo, value} shape is correct.
Collaborator
Author
There was a problem hiding this comment.
Added. Same fields/data as the CSV pin-columns test, with format: 'json'. One comment though. it
asserts [null, {relationTo, value}] rather than the dangling entry being dropped, following your
note on line 113 about preserving nulls so JSON matches CSV.
Comment on lines
96
to
113
| if (!Array.isArray(field.relationTo)) { | ||
| registerHandler(({ siblingData, value }) => { | ||
| if (Array.isArray(value)) { | ||
| value.forEach((val, i) => { | ||
| const id = typeof val === 'object' && val ? (val as { id: unknown }).id : val | ||
| siblingData[`${fullKey}_${i}_id`] = id | ||
| }) | ||
| return null | ||
| registerHandler(({ format, siblingData, value }) => { | ||
| if (!Array.isArray(value)) { | ||
| return undefined | ||
| } | ||
| return undefined | ||
| const ids = value.map((val) => | ||
| typeof val === 'object' && val ? (val as { id: unknown }).id : val, | ||
| ) | ||
| // A dangling reference cannot be imported back, so it is dropped rather than | ||
| // exported as a null the import would reject as an invalid relationship. | ||
| if (format === 'json') { | ||
| return ids.filter((id) => id !== undefined && id !== null) | ||
| } | ||
| ids.forEach((id, i) => { | ||
| siblingData[`${fullKey}_${i}_id`] = id | ||
| }) | ||
| return null | ||
| }) |
Member
There was a problem hiding this comment.
JSON export will now change array length/position for dangling entries; CSV deliberately doesn't, and this asymmetry isn't documented.
A doc with rel: ['p1', 'p2'] where p1 is soft-deleted exports as CSV with both positions preserved (one as a gap), but as JSON rel: ['p2'] — length 1, not 2. Any other array field on the doc that a consumer expects to stay index-aligned with rel silently loses that alignment for JSON only.
I would lean on allowing exports to have null for now so that JSON matches with CSVs in behaviour. Since that is the underlying data shape. In the future we can do a "skip null" type of flag to handle these edge cases.
… exports Keep a dangling hasMany reference as null at its source index so JSON exports preserve the same length and positions as CSV, which pins each entry to its own column. Test coverage from review: - drive the hasMany polymorphic CSV column test through flattenObject, which is what real CSV export uses instead of applyFieldHooks - cover a gap strictly between two present column indices on import - cover hasMany polymorphic + JSON + dangling reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirrors #17480 (which targets
3.x) ontomain.Problem
hasManyand polymorphic relationship fields were flattened intofield_N_id/field_N_relationTosibling keys for every export format, but only CSV import reverses that flattening. Importing a collection's own JSON export therefore dropped everyhasManyand polymorphic relationship.Fix
The export hooks now check the format and return the relationship value as-is for JSON — an id, an array of ids, or
relationTo/valuepairs. CSV output is unchanged.JSON Export before the change:
[ { "id": 2, "name": "the wall", "description": "the north remembers", "events": null, "updatedAt": "2026-07-24T18:03:33.856Z", "createdAt": "2026-07-24T18:03:33.856Z", "events_0_id": 1, "events_1_id": 2 } ]JSON Export After Change:
[ { "id": 2, "name": "the wall", "description": "the north remembers", "events": [1,2], "updatedAt": "2026-07-24T18:03:33.856Z", "createdAt": "2026-07-24T18:03:33.856Z", } ]I also updated the Import Preview for JSON imports to render cells in a more formatted manner for polymorphic relationships