Repository navigation
feat(search-pipeline)!: apply a label source’s transforms to its referents - #1096
Merged
Merged
Conversation
ddeboer
force-pushed
the
feat/search-referent-document-labels
branch
from
November 16, 2026 16:16
208fb82 to
6fa1916
Compare
…rents - Pass the key and label quads of each searchable reference’s referents through the target type’s own transforms, inside the batch - Search the reference by the labels those transforms leave, as the target’s documents carry them - Record the decision in ADR 39 BREAKING CHANGE: a root type’s transforms also run on its referents in the stages of types whose searchable references name it, receiving nodes that carry only the key and label field. Make such a transform cope with that input, or drop searchable from the referring reference.
- Route the referents of an inline reference whose graph leaves the edge out, read off the inline node itself, through the endpoint’s target - Key the referents’ transforms by class, as the schema resolves a class-equal lookalike - Append routed quads in place instead of copying the array per quad - Move referentHopFields above the private helpers and restore keyedFieldOf’s doc comment
ddeboer
force-pushed
the
feat/search-referent-document-labels
branch
from
November 16, 2026 18:22
aad14f0 to
60cd630
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
searchablereference is now searched by its referents’ labels as the target type’s transforms make them, not only as the referring graph states them (#1093). Before projection, each stage passes its referents’ key and label quads through the target type’s own transforms, inside the batch, and projects what comes back. Decision and alternatives: ADR 39.In Linked Open Limburg, the
Termtransform replaces a keyed term’s label with the Authority’s, so the Term document for the CHT concept is called “boerenbont”. Works referencing that term were searched only by their publisher’s label for it, which is often missing or untagged (limburg/lol#318). With this change, those works are searched by the Authority’s label.This keeps memory bounded by
batchSize(ADR 12) and needs no stage order, so referents from any dataset are covered. A first version kept target documents’ labels in memory for the dataset pass, and review rejected it for breaking ADR 12. ADR 39 lists it as a rejected alternative.Limits:
SearchStageType.searchIndexerPipelinestages every schema type, but a caller wiringsearchStagesdirectly must include the target type, with its transforms on the default reader.locallookups such ascreatorstill refusesearchable.Part of #1058. It stays open for the
locallookups.BREAKING CHANGE: a root type’s transforms also run on its referents in the stages of types whose
searchablereferences name it. They receive nodes that carry only the key and label field, once per referring batch.Before: a
Termtransform saw onlyTermroots.After: it also sees each
aboutreferent of aCreativeWorkbatch whenaboutissearchable. Make the transform cope with a node holding only_sameAsandname, or dropsearchablefrom the reference.