Repository navigation
perf(engine): query optimizer pipeline & zero-copy N-Family bulk ingestion - #634
Merged
remiceres merged 5 commits intoOct 20, 2026
Merged
Conversation
…in SELECT - Implement Query.isDistinctRedundantWithGroupBy() with strict W3C semantics - Guard against non-projected grouping variables to preserve aggregate distinctness - Bypass distinct hash group allocation and submission lookup in Mappings - Add unit tests in QueryTest covering nominal, projected, and missing key cases - Reduce Q8 Bowlogna benchmark latency by 26% (1830 ms -> 1352 ms)
…dges Identify triple patterns constrained by inequality/equality comparison filters and position them as initial seeds during cardinality-guided join planning. This dramatically cuts intermediate candidate state on queries with selective post-filters (e.g. Bowlogna Q8 down from 1830ms to 969ms).
…eaming scanner Eliminate NTriples.g4 and NTriplesListener to remove per-line ANTLR ParseTree and TokenStream allocations. NTriplesParser now scans W3C RDF 1.1 N-Triples directly into Model/StorageManager with zero-copy buffer handling, dividing Bowlogna 1.23M loading time by 4 (from 45.5s down to 11.1s).
…ming scanner Eliminate NQuads.g4, NQuadsListener and LineRDFParser to remove all per-line ANTLR allocations across the N-Family format suite. NQuadsParser now scans quads and triples with zero-copy buffer handling directly into Model/StorageManager, achieving 100% W3C conformance on official RDF 1.1 N-Quads tests.
…nOrdering - Dynamically recompute pattern effective cost during greedy selection based on already bound variables from enclosing scopes and chosen edges. - Scale cardinality with BigDecimal precision (discount SPO subject probes by 1000, POS/OSP object lookups by 10, variable predicate by 10). - Maintain strict connectivity priority over disconnected Cartesian patterns. - Add 9 regression and edge-case unit tests covering precision, bound probes, and tie-breaking order. - Fixes #633 (Phase C).
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Cette Pull Request résout et finalise les travaux d'optimisation du moteur et d'ingestion massive pour l'atteinte des performances comparatives avec Apache Jena 6.2.0 et Eclipse RDF4J 5.3.2 sur le benchmark Bowlogna (1,23 million de triplets) dans le cadre du Cycle #53 (Itération 36).
Fixes #633.
Réalisations & Percées Techniques
Phase A1 — DISTINCT Bypass sur GROUP BY (
ce65602cc) :Query.java(isDistinctRedundantWithGroupBy()) lorsque toutes les variables de regroupement sont projetées dans leSELECT.MappingsetEval.Phase A2 — Selective Filter Pushdown (
2cd0b366c) :SelectiveFilterPushdown.javadans l'ordonnanceur d'arêtes.?var = constant).Phase D — Modernisation & Fast Streaming Scanners N-Triples / N-Quads (
16c79d432&f512e8740) :NTriples.g4,NQuads.g4et des parseurs/listeners d'arbres syntaxiques.NTriplesParser.javaetNQuadsParser.javaavec un scanner streaming zéro-copie optimisé.Phase C — Cost-Based Join Reordering Dynamique (
195607694) :CardinalityPatternOrdering.javaprenant en compte le fan-out réel selon les variables déjà liées par les arêtes précédentes ou les scopes englobants.BigDecimal.movePointLeft()(sujet lié /1000 pour probe SPO, objet lié /10 pour POS/OSP, prédicat variable /10).Qualité & Conformité
corese-core(dont 33 tests dédiés CBO).