Conversation
…dex in combinators This commit introduces several major improvements and bug fixes for v5.2.0 release. New features: - Add Scan() methods returning interleaved sequence of raw text segments and parsed matches via Or<StringSegment, ParsedRuleResultBase> - Add SplitSegments() methods to split input by parser rule matches (analogous to Regex.Split) - Add new overloads of MatchesToken() that return ParsedElement instead of just matched length - Add AnyCharTokenPattern — matches any single character - Add ParserTokenPattern — allows nesting one parser as a token in another parser - Add StringSegment struct — lightweight zero-copy string slice with lazy cached substring - Add SemanticException — structured exception with AST node reference, source position formatting, and nested children errors - Expose PositionalFormatter.Format() as public static method Bug fixes: - Fix startIndex in RepeatParserRule, SeparatedRepeatParserRule, SequenceParserRule and all combinator token patterns: initialPosition now correctly set to first child's startIndex instead of parsing start position, which fixes wrong offsets when SkipStrategies are used - Fix LiteralCharTokenPattern boundary check (position >= barrierPosition) - Fix RegexTokenPattern null-safety in ToString() - Fix CaptureTextTokenPattern: set intermediateValue directly on child element instead of creating new ParsedElement - Fix SkipWhitespacesTokenPattern: return child result directly instead of wrapping in a new ParsedElement with accumulated whitespace offset DX improvements: - Replace hard casts (T)value in ParsedRuleResultBase with soft 'is T' checks that throw SemanticException with detailed context instead of InvalidCastException - Same for GetIntermediateValue<T>, ConvertValue<T>, GetParsingParameter<T> Tests: - Add ReplaceAllMatchesTests (10 test cases) - Add ScanTests (14 test cases) - Add SplitSegmentsTests (8 test cases) - Update CombinatorTokensTests and KeywordTokenTests to use explicit out variable types Version: 5.1.0 -> 5.2.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New features:
segments and parsed matches via Or<StringSegment, ParsedRuleResultBase>
(analogous to Regex.Split)
of just matched length
another parser
lazy cached substring
source position formatting, and nested children errors
Bug fixes:
SequenceParserRule and all combinator token patterns: initialPosition
now correctly set to first child's startIndex instead of parsing start
position, which fixes wrong offsets when SkipStrategies are used
element instead of creating new ParsedElement
of wrapping in a new ParsedElement with accumulated whitespace offset
DX improvements:
checks that throw SemanticException with detailed context instead of
InvalidCastException
Tests:
out variable types