diff --git a/cpp/ql/integration-tests/query-suite/cpp-code-scanning.qls.expected b/cpp/ql/integration-tests/query-suite/cpp-code-scanning.qls.expected index d4b805999500..41e57565365b 100644 --- a/cpp/ql/integration-tests/query-suite/cpp-code-scanning.qls.expected +++ b/cpp/ql/integration-tests/query-suite/cpp-code-scanning.qls.expected @@ -28,6 +28,8 @@ ql/cpp/ql/src/Security/CWE/CWE-089/SqlTainted.ql ql/cpp/ql/src/Security/CWE/CWE-120/BadlyBoundedWrite.ql ql/cpp/ql/src/Security/CWE/CWE-120/VeryLikelyOverrunWrite.ql ql/cpp/ql/src/Security/CWE/CWE-131/NoSpaceForZeroTerminator.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql ql/cpp/ql/src/Security/CWE/CWE-134/UncontrolledFormatString.ql ql/cpp/ql/src/Security/CWE/CWE-190/ArithmeticUncontrolled.ql ql/cpp/ql/src/Security/CWE/CWE-190/ComparisonWithWiderType.ql diff --git a/cpp/ql/integration-tests/query-suite/cpp-security-and-quality.qls.expected b/cpp/ql/integration-tests/query-suite/cpp-security-and-quality.qls.expected index cb4e5f7b305a..18b55c92fa66 100644 --- a/cpp/ql/integration-tests/query-suite/cpp-security-and-quality.qls.expected +++ b/cpp/ql/integration-tests/query-suite/cpp-security-and-quality.qls.expected @@ -116,6 +116,8 @@ ql/cpp/ql/src/Security/CWE/CWE-120/UnboundedWrite.ql ql/cpp/ql/src/Security/CWE/CWE-120/VeryLikelyOverrunWrite.ql ql/cpp/ql/src/Security/CWE/CWE-121/UnterminatedVarargsCall.ql ql/cpp/ql/src/Security/CWE/CWE-131/NoSpaceForZeroTerminator.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql ql/cpp/ql/src/Security/CWE/CWE-134/UncontrolledFormatString.ql ql/cpp/ql/src/Security/CWE/CWE-190/ArithmeticUncontrolled.ql ql/cpp/ql/src/Security/CWE/CWE-190/ComparisonWithWiderType.ql diff --git a/cpp/ql/integration-tests/query-suite/cpp-security-extended.qls.expected b/cpp/ql/integration-tests/query-suite/cpp-security-extended.qls.expected index 7c5e3d98c5c7..4a98d6bc0fcd 100644 --- a/cpp/ql/integration-tests/query-suite/cpp-security-extended.qls.expected +++ b/cpp/ql/integration-tests/query-suite/cpp-security-extended.qls.expected @@ -49,6 +49,8 @@ ql/cpp/ql/src/Security/CWE/CWE-120/UnboundedWrite.ql ql/cpp/ql/src/Security/CWE/CWE-120/VeryLikelyOverrunWrite.ql ql/cpp/ql/src/Security/CWE/CWE-121/UnterminatedVarargsCall.ql ql/cpp/ql/src/Security/CWE/CWE-131/NoSpaceForZeroTerminator.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql +ql/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql ql/cpp/ql/src/Security/CWE/CWE-134/UncontrolledFormatString.ql ql/cpp/ql/src/Security/CWE/CWE-190/ArithmeticUncontrolled.ql ql/cpp/ql/src/Security/CWE/CWE-190/ComparisonWithWiderType.ql diff --git a/cpp/ql/lib/change-notes/2026-07-15-cpp-regex-tree-view.md b/cpp/ql/lib/change-notes/2026-07-15-cpp-regex-tree-view.md new file mode 100644 index 000000000000..19d6338088de --- /dev/null +++ b/cpp/ql/lib/change-notes/2026-07-15-cpp-regex-tree-view.md @@ -0,0 +1,4 @@ +--- +category: minorAnalysis +--- +* Added an internal C++ regular-expression parse library (`semmle.code.cpp.regex.RegexTreeView`) that implements the shared `RegexTreeViewSig` signature. This is the foundation for future ReDoS analyses for C++ (`cpp/polynomial-redos` and `cpp/redos`). No new queries are added in this release. diff --git a/cpp/ql/lib/change-notes/2026-07-16-cpp-regex-flow-configs.md b/cpp/ql/lib/change-notes/2026-07-16-cpp-regex-flow-configs.md new file mode 100644 index 000000000000..ce5196a17c21 --- /dev/null +++ b/cpp/ql/lib/change-notes/2026-07-16-cpp-regex-flow-configs.md @@ -0,0 +1,4 @@ +--- +category: minorAnalysis +--- +* Added internal C++ regex usage and flow modeling (`semmle.code.cpp.regex.RegexFlowConfigs`). The `RegexTreeView` `RegExp` class now only matches `StringLiteral`s that flow to a `std::basic_regex` construction/assignment or a `std::regex_match`/`std::regex_search`/`std::regex_replace`/`std::regex_iterator`/`std::regex_token_iterator` call. Construction-site flags in `std::regex_constants` are detected (`icase`, `multiline`, and the grammar flags), including bitwise-`|` combinations. Literals constructed with an explicit non-ECMAScript grammar flag (`basic`, `extended`, `awk`, `grep`, `egrep`) are excluded from the ECMAScript-only parser. No new queries are added in this release. diff --git a/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-bre-grammar.md b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-bre-grammar.md new file mode 100644 index 000000000000..fb2cef365c86 --- /dev/null +++ b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-bre-grammar.md @@ -0,0 +1,5 @@ +--- +category: minorAnalysis +--- +* The C++ regular-expression parser (`semmle.code.cpp.regex.internal.ParseRegExp`) now models POSIX Basic Regular Expressions (BRE) via a new concrete `BreRegExp` subclass. BRE is selected by the `std::regex_constants::basic` and `grep` flags. Where BRE inverts the metacharacter/escape convention relative to POSIX ERE and ECMAScript — backslash-prefixed `\(...\)` are capturing groups (bare `(...)` are literals), `\{n,m\}` is the interval quantifier (bare braces are literal text), and `+`, `?`, `|` are literal characters — the parser follows the POSIX specification. Positional rules for `*` (literal at the start of a subexpression), `^` (anchor only at start of subexpression) and `$` (anchor only at end of subexpression) are implemented; `\1`..`\9` numbered back-references are supported. The GNU-BRE extensions `\+`, `\?`, `\|` are not modeled — they are treated as escaped literals. Along with ECMAScript (`EcmaRegExp`) and POSIX ERE (`EreRegExp`), all six of the `std::regex_constants` grammar flags now have a concrete parser subclass and every regex is analyzed. +* The C++ ReDoS queries (`cpp/redos` and `cpp/polynomial-redos`) now analyze regexes constructed with the `std::regex_constants::basic` grammar flag; `grep` remains excluded via `isBacktrackingEngine` (POSIX tool-style engines are treated as linear-time), mirroring the existing `extended`-vs-`egrep`/`awk` split for ERE. diff --git a/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-non-backtracking-engine.md b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-non-backtracking-engine.md new file mode 100644 index 000000000000..8478b9267b2a --- /dev/null +++ b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-non-backtracking-engine.md @@ -0,0 +1,4 @@ +--- +category: minorAnalysis +--- +* The C++ ReDoS queries (`cpp/redos` and `cpp/polynomial-redos`) now treat regexes constructed with the `std::regex_constants::awk`, `grep`, or `egrep` grammar flags as non-backtracking, and exclude them from analysis. These POSIX tool-style grammars correspond to traditionally linear-time (DFA-based) matching semantics, so super-linear-backtracking ReDoS does not apply. The `basic` (BRE) and `extended` (ERE) grammars remain backtracking-eligible. Grammar selection and ReDoS-eligibility are exposed as independent axes via the new `isBacktrackingEngine` predicate in `semmle.code.cpp.regex.RegexFlowConfigs`. diff --git a/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-posix-brackets.md b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-posix-brackets.md new file mode 100644 index 000000000000..f6589dd67c02 --- /dev/null +++ b/cpp/ql/lib/change-notes/2026-07-17-cpp-regex-posix-brackets.md @@ -0,0 +1,5 @@ +--- +category: minorAnalysis +--- +* The C++ regular-expression parser (`semmle.code.cpp.regex.RegexTreeView`) now recognises POSIX bracket sub-expressions inside character classes as single character-matching atoms: POSIX character classes (`[[:alpha:]]`, `[[:digit:]]`, `[[:space:]]`, ...), collating symbols (`[[.a.]]`), and equivalence classes (`[[=a=]]`). These are ECMAScript-mode extensions accepted by `std::regex` but not part of ECMA-262 JavaScript, so they were previously mis-tokenized (the class was incorrectly closed at the inner `]`). POSIX character classes whose match set is `\d`, `\s`, or `\w` — or a subset of one (`digit`, `xdigit`, `space`, `blank`, `word`, `alpha`, `alnum`, `upper`, `lower`) — are mapped onto `\d`/`\s`/`\w` for the shared engine's escape-class reasoning. The remaining POSIX classes (`punct`, `cntrl`, `print`, `graph`) as well as collating and equivalence classes are left opaque: they still parse as single character-consuming class-member atoms, but the shared engine treats their character set as unknown rather than unsoundly equating it with `\w`. +* Documented that `std::regex_constants::multiline` is detected by `RegexFlowConfigs.qll` but not currently modeled in the parse tree — `^` and `$` are always treated as string-start/string-end anchors, matching the conservative choice made by other language ReDoS analyses. Precise multiline modeling would require extending the shared `RegexTreeViewSig` signature and is left as future work. diff --git a/cpp/ql/lib/qlpack.yml b/cpp/ql/lib/qlpack.yml index 5f4b92a191fc..775e8c717291 100644 --- a/cpp/ql/lib/qlpack.yml +++ b/cpp/ql/lib/qlpack.yml @@ -10,6 +10,7 @@ dependencies: codeql/mad: ${workspace} codeql/quantum: ${workspace} codeql/rangeanalysis: ${workspace} + codeql/regex: ${workspace} codeql/ssa: ${workspace} codeql/typeflow: ${workspace} codeql/tutorial: ${workspace} diff --git a/cpp/ql/lib/semmle/code/cpp/regex/RegexFlowConfigs.qll b/cpp/ql/lib/semmle/code/cpp/regex/RegexFlowConfigs.qll new file mode 100644 index 000000000000..adf218d9ccc6 --- /dev/null +++ b/cpp/ql/lib/semmle/code/cpp/regex/RegexFlowConfigs.qll @@ -0,0 +1,465 @@ +/** + * Provides classes and predicates for reasoning about C++ standard-library + * regular-expression usage. + * + * This module identifies which `StringLiteral`s flow (via global taint + * tracking) into a `std::basic_regex` construction or into one of the free + * matching functions (`std::regex_match`, `std::regex_search`, + * `std::regex_replace`) or iterators (`std::regex_iterator`, + * `std::regex_token_iterator`), and detects the construction-site flags of + * `std::regex_constants::syntax_option_type` and + * `std::regex_constants::match_flag_type` that affect matching semantics. + * + * The `regexMatchedAgainst` predicate mirrors the intent of the Java + * `RegexFlowConfigs.qll` library. + * + * All standard `std::regex` grammars are now modeled: ECMAScript (default), + * POSIX BRE (`basic`/`grep`), and POSIX ERE (`extended`/`egrep`/`awk`). + * Grammar selection and ReDoS-eligibility are independent axes - see + * `isBacktrackingEngine` for the latter. + */ + +import cpp +private import semmle.code.cpp.dataflow.new.DataFlow +private import semmle.code.cpp.dataflow.new.TaintTracking + +// --------------------------------------------------------------------------- +// std::basic_regex identification +// --------------------------------------------------------------------------- +/** + * A `std::basic_regex` class type (or instantiation thereof, e.g. `std::regex`, + * `std::wregex`). + */ +class StdBasicRegex extends Class { + StdBasicRegex() { + this.hasQualifiedName("std", "basic_regex") + or + this.(ClassTemplateInstantiation).getTemplate().hasQualifiedName("std", "basic_regex") + } +} + +/** + * Holds if `t`, after stripping references, const/volatile, and typedefs, + * denotes a `std::basic_regex` type. + */ +private predicate isBasicRegexType(Type t) { + exists(Type u | u = t.stripType() | + u instanceof StdBasicRegex + or + // Typedefs (e.g. `std::regex` = `std::basic_regex`) resolve via + // `getUnderlyingType()`. + u.(TypedefType).getBaseType().stripType() instanceof StdBasicRegex + ) +} + +/** + * Gets the parameter of `f` whose type is a reference (or plain) to + * `std::basic_regex`, i.e. the parameter that receives the regex object. + */ +private Parameter getRegexObjectParameter(Function f) { + result = f.getAParameter() and + isBasicRegexType(result.getType()) +} + +/** + * Gets a parameter of `f` whose type is a string-like: `const char*`, + * `const wchar_t*`, `std::basic_string`, or a `char`/`wchar_t` iterator + * pair. Used to identify the subject string of match/search/replace calls. + */ +private Parameter getStringLikeParameter(Function f) { + result = f.getAParameter() and + exists(Type u | u = result.getUnspecifiedType().stripType() | + // basic_string (by value or reference) + u.(Class).hasQualifiedName("std", "basic_string") + or + u.(ClassTemplateInstantiation).getTemplate().hasQualifiedName("std", "basic_string") + or + // C strings: const char* / const wchar_t* + exists(PointerType p | + p = u + or + p = u.(ReferenceType).getBaseType().stripType() + | + p.getBaseType().stripType() instanceof CharType + or + p.getBaseType().stripType() instanceof Wchar_t + ) + ) +} + +// --------------------------------------------------------------------------- +// Match / search / replace / iterator calls +// --------------------------------------------------------------------------- +/** + * A free function in namespace `std` that matches a subject against a regex: + * one of `regex_match`, `regex_search`, or `regex_replace`. + */ +private class StdRegexMatchFunction extends Function { + StdRegexMatchFunction() { + this.getNamespace().getName() = "std" and + this.getName() = ["regex_match", "regex_search", "regex_replace"] + } +} + +/** + * A class template instantiation of `std::regex_iterator` or + * `std::regex_token_iterator`. Their constructors take a range and a regex. + */ +private class StdRegexIterator extends Class { + StdRegexIterator() { + this.(ClassTemplateInstantiation) + .getTemplate() + .hasQualifiedName("std", ["regex_iterator", "regex_token_iterator"]) + or + this.hasQualifiedName("std", ["regex_iterator", "regex_token_iterator"]) + } +} + +/** + * Holds if `call` is a call site that matches a subject against a regex, + * where `regexArg` is the argument holding the regex object and `subjectArg` + * (if it exists) is the argument holding the subject string. + */ +predicate regexMatchCall(Call call, Expr regexArg, Expr subjectArg) { + exists(Function f, Parameter rp | + f = call.getTarget() and + ( + f instanceof StdRegexMatchFunction + or + // Iterator constructors. + f.(Constructor).getDeclaringType() instanceof StdRegexIterator + ) and + rp = getRegexObjectParameter(f) and + regexArg = call.getArgument(rp.getIndex()) + | + // First string-like parameter, if any, is the subject. + exists(Parameter sp | + sp = getStringLikeParameter(f) and + subjectArg = call.getArgument(sp.getIndex()) and + // Prefer the earliest such parameter (matches the standard argument + // order for these overloads). + not exists(Parameter sp2 | sp2 = getStringLikeParameter(f) and sp2.getIndex() < sp.getIndex()) + ) + ) +} + +// --------------------------------------------------------------------------- +// Regex-flow sinks: places where a pattern (a StringLiteral) is used as a regex +// --------------------------------------------------------------------------- +/** + * A regex-flow sink: an expression at which a value is used as the pattern + * for a `std::basic_regex` (construction, `.assign(...)`), or as the pattern + * argument of a free match/search/replace call that takes a raw pattern. + */ +class RegexPatternSink extends DataFlow::Node { + RegexPatternSink() { + // 1. Constructor argument 0 of std::basic_regex. + exists(ConstructorCall cc, Constructor c | + c = cc.getTarget() and + c.getDeclaringType() instanceof StdBasicRegex and + this.asExpr() = cc.getArgument(0) + ) + or + // 2. Argument 0 of a `basic_regex::assign(...)` call. + exists(FunctionCall fc, MemberFunction m | + m = fc.getTarget() and + m.getDeclaringType() instanceof StdBasicRegex and + m.hasName("assign") and + this.asExpr() = fc.getArgument(0) + ) + } +} + +// --------------------------------------------------------------------------- +// Fast-path: only track literals that look regex-y +// --------------------------------------------------------------------------- +/** + * A string literal that is a plausible ReDoS candidate: it contains at least + * one unbounded-repetition quantifier (`+`, `*`, or `{n,}`). Used as an + * optimisation to keep the taint-tracking configuration small; other regexes + * are not interesting for the polynomial-ReDoS analysis anyway. + */ +class ExploitableStringLiteral extends StringLiteral { + ExploitableStringLiteral() { + exists(string s | s = this.getValue() | + s.regexpMatch(".*[+*].*") or + s.regexpMatch(".*\\{[0-9]+,[0-9]*\\}.*") + ) + } +} + +/** + * A dataflow configuration tracking string literals that reach a regex + * pattern construction/assignment site. + */ +private module RegexPatternFlowConfig implements DataFlow::ConfigSig { + predicate isSource(DataFlow::Node node) { node.asExpr() instanceof ExploitableStringLiteral } + + predicate isSink(DataFlow::Node node) { node instanceof RegexPatternSink } +} + +private module RegexPatternFlow = TaintTracking::Global; + +// --------------------------------------------------------------------------- +// Public predicates +// --------------------------------------------------------------------------- +/** + * Holds if the `StringLiteral` `regex` flows to a modeled `std::regex` + * construction or usage site. + * + * As an optimisation, only regexes containing an unbounded-repetition + * quantifier (`+`, `*`, or `{n,}`) are considered. + */ +predicate usedAsRegex(StringLiteral regex) { + regex instanceof ExploitableStringLiteral and + RegexPatternFlow::flowFromExpr(regex) +} + +/** + * Holds if `regex` is a string literal used as a regular expression that is + * matched against the expression `str`. + * + * As an optimisation, only regexes containing an unbounded-repetition + * quantifier (`+`, `*`, or `{n,}`) are considered. + */ +predicate regexMatchedAgainst(StringLiteral regex, Expr str) { + exists(RegexPatternSink patternSink, Variable v | + RegexPatternFlow::flow(DataFlow::exprNode(regex), patternSink) and + // Recover the variable that is being constructed / assigned to. + ( + exists(ConstructorCall cc | + patternSink.asExpr() = cc.getArgument(0) and + cc.getEnclosingElement() = v.getInitializer() + ) + or + exists(FunctionCall fc | + patternSink.asExpr() = fc.getArgument(0) and + fc.getQualifier() = v.getAnAccess() + ) + ) and + // The regex variable is used as the regex argument to a match call. + exists(Expr regexArg | + regexMatchCall(_, regexArg, str) and + regexArg = v.getAnAccess() + ) + ) + or + // Also handle the pattern being passed inline to a match call (no named + // variable): rare in practice for std::regex, but supported for + // completeness. + exists(Expr regexArg | + regexMatchCall(_, regexArg, str) and + // The regex argument is a temporary `basic_regex(pattern)`. + exists(ConstructorCall cc | + cc.getTarget().getDeclaringType() instanceof StdBasicRegex and + cc.getParent*() = regexArg and + cc.getArgument(0) = regex.getFullyConverted() + or + cc.getTarget().getDeclaringType() instanceof StdBasicRegex and + cc.getParent*() = regexArg and + cc.getArgument(0) = regex + ) + ) +} + +// --------------------------------------------------------------------------- +// Construction-site flags +// --------------------------------------------------------------------------- +/** + * Holds if `ec` is a `std::regex_constants` enum constant with the given + * unqualified name. + */ +private predicate regexConstantsEnum(EnumConstant ec, string name) { + ec.hasName(name) and + ec.getDeclaringEnum().getNamespace().getName() = "regex_constants" and + ec.getDeclaringEnum().getNamespace().getParentNamespace().getName() = "std" +} + +/** + * Holds if `access` (an expression) is a reference to the `regex_constants` + * enum constant with the given `name`, possibly through implicit conversions. + */ +private predicate refersToRegexConstant(Expr access, string name) { + exists(EnumConstantAccess eca | eca = access.getAChild*() or eca = access | + regexConstantsEnum(eca.getTarget(), name) + ) +} + +/** + * Holds if `flagExpr` (a `syntax_option_type`/`match_flag_type` argument) is + * a bit-or expression, character constant, or single enum constant that + * includes the `regex_constants` flag with unqualified `name`. + * This handles both direct use (`std::regex_constants::icase`) and bitwise-OR + * combinations (`std::regex_constants::icase | std::regex_constants::multiline`). + */ +private predicate containsRegexFlag(Expr flagExpr, string name) { + refersToRegexConstant(flagExpr, name) + or + // Bitwise-OR combination: recurse into both operands. + containsRegexFlag(flagExpr.(BitwiseOrExpr).getAnOperand(), name) + or + // Operator| overload on the flag enum (some libc++ implementations expose + // `operator|` as a free function). Recurse into arguments. + exists(FunctionCall fc | + fc = flagExpr and + fc.getTarget().hasName("operator|") + | + containsRegexFlag(fc.getAnArgument(), name) + ) +} + +/** + * Gets a flag argument (`syntax_option_type` / `match_flag_type`) for the + * `basic_regex` construction (or `assign(...)`) whose pattern is `regex`. + * Returns nothing if no explicit flag argument is supplied. + */ +private Expr getConstructionFlagArg(StringLiteral regex) { + // basic_regex(pattern, flags) at variable construction. + exists(ConstructorCall cc, Variable v | + cc.getTarget().getDeclaringType() instanceof StdBasicRegex and + cc.getEnclosingElement() = v.getInitializer() and + RegexPatternFlow::flow(DataFlow::exprNode(regex), DataFlow::exprNode(cc.getArgument(0))) and + result = cc.getArgument(1) + ) + or + // Temporary basic_regex(pattern, flags). + exists(ConstructorCall cc | + cc.getTarget().getDeclaringType() instanceof StdBasicRegex and + RegexPatternFlow::flow(DataFlow::exprNode(regex), DataFlow::exprNode(cc.getArgument(0))) and + result = cc.getArgument(1) + ) + or + // basic_regex::assign(pattern, flags). + exists(FunctionCall fc | + fc.getTarget().(MemberFunction).getDeclaringType() instanceof StdBasicRegex and + fc.getTarget().hasName("assign") and + RegexPatternFlow::flow(DataFlow::exprNode(regex), DataFlow::exprNode(fc.getArgument(0))) and + result = fc.getArgument(1) + ) +} + +/** + * Holds if `regex` is constructed with the `std::regex_constants::icase` flag, + * either directly or as part of a bitwise-OR combination. + */ +predicate hasIgnoreCaseFlag(StringLiteral regex) { + containsRegexFlag(getConstructionFlagArg(regex), "icase") +} + +/** + * Holds if `regex` is constructed with the `std::regex_constants::multiline` + * flag (C++17 and later), either directly or as part of a bitwise-OR + * combination. + */ +predicate hasMultilineFlag(StringLiteral regex) { + containsRegexFlag(getConstructionFlagArg(regex), "multiline") +} + +/** + * Holds if `regex` is constructed with an explicit non-ECMAScript grammar + * flag (`basic`, `extended`, `awk`, `grep`, or `egrep`). + * + * This predicate is purely informational: nothing is excluded from analysis + * by grammar anymore, since every grammar the standard defines now has a + * concrete parser subclass (`EcmaRegExp`, `EreRegExp`, `BreRegExp`). + */ +predicate hasNonEcmaScriptGrammarFlag(StringLiteral regex) { + exists(string g | g = ["basic", "extended", "awk", "grep", "egrep"] | + containsRegexFlag(getConstructionFlagArg(regex), g) + ) +} + +// --------------------------------------------------------------------------- +// Grammar classification +// --------------------------------------------------------------------------- +/** + * The `std::regex` grammar dialects that the C++ regex parser is aware of. + * + * - `Ecma()` - ECMAScript, the default grammar used by `std::regex`. + * Selected either implicitly (no explicit grammar flag) or + * explicitly via `std::regex_constants::ECMAScript`. Modeled + * by `EcmaRegExp`. + * - `Bre()` - POSIX Basic Regular Expressions (selected via the `basic` + * or `grep` flags). Modeled by `BreRegExp`. + * - `Ere()` - POSIX Extended Regular Expressions (selected via the + * `extended`, `egrep`, or `awk` flags). Modeled by + * `EreRegExp`. + * + * All three cases are exercised by the parser today; every grammar has a + * concrete subclass, so `hasConcreteGrammar` holds for every regex the + * parser sees. + */ +newtype TRegexGrammar = + /** The ECMAScript grammar (the default for `std::regex`), modeled by `EcmaRegExp`. */ + Ecma() or + /** The POSIX Basic Regular Expression grammar (`basic`/`grep` flags), modeled by `BreRegExp`. */ + Bre() or + /** The POSIX Extended Regular Expression grammar (`extended`/`egrep`/`awk` flags), modeled by `EreRegExp`. */ + Ere() + +/** + * Gets the `std::regex` grammar dialect of `regex`, inferred from its + * construction-site `syntax_option_type` flag argument (if any). + * + * The mapping is: + * - `basic` / `grep` -> `Bre()` + * - `extended` / `egrep` / `awk` -> `Ere()` + * - anything else (default, explicit `ECMAScript`, or unresolved) -> `Ecma()` + * + * Every case now has a concrete parser subclass, so `hasConcreteGrammar` + * holds for the result of this predicate. + */ +TRegexGrammar regexGrammar(StringLiteral regex) { + if containsRegexFlag(getConstructionFlagArg(regex), ["basic", "grep"]) + then result = Bre() + else + if containsRegexFlag(getConstructionFlagArg(regex), ["extended", "egrep", "awk"]) + then result = Ere() + else result = Ecma() +} + +/** + * Holds if `grammar` has a concrete `RegExp` subclass and can therefore be + * admitted by the parser's characteristic predicate. All three grammars + * (`Ecma()`, `Ere()`, `Bre()`) are modeled today, so this holds for every + * grammar the standard defines. Kept as a helper so that any future grammar + * scaffolding can be admitted by adding a single disjunct here. + */ +predicate hasConcreteGrammar(TRegexGrammar grammar) { + grammar = Ecma() or grammar = Ere() or grammar = Bre() +} + +/** + * Holds if `regex` is constructed with an explicit ECMAScript grammar flag. + * This is the default, and also the case that the Phase 1 parser handles. + */ +predicate hasEcmaScriptGrammarFlag(StringLiteral regex) { + containsRegexFlag(getConstructionFlagArg(regex), "ECMAScript") +} + +/** + * Holds if `regex` is constructed with a POSIX tool-style grammar flag + * (`awk`, `grep`, or `egrep`) that we treat as linear-time / non-backtracking. + */ +private predicate hasNonBacktrackingGrammarFlag(StringLiteral regex) { + containsRegexFlag(getConstructionFlagArg(regex), ["awk", "grep", "egrep"]) +} + +/** + * Holds if `regex` is used with a `std::regex` matching engine that performs + * backtracking, so that super-linear-backtracking ReDoS is possible. + * + * `std::regex` backtracks for the ECMAScript, `basic` (BRE), and `extended` + * (ERE) grammars. The POSIX tool-style grammars `awk`, `grep`, and `egrep` + * are treated as linear-time (non-backtracking) matching semantics for the + * purposes of the ReDoS queries and are excluded here. + * + * Grammar selection and ReDoS-eligibility are independent axes: two flags + * can select the same grammar yet differ in ReDoS-eligibility (for example + * `extended` and `egrep` both parse as ERE, but only `extended` is + * backtracking-eligible). + */ +predicate isBacktrackingEngine(StringLiteral regex) { + usedAsRegex(regex) and + not hasNonBacktrackingGrammarFlag(regex) +} diff --git a/cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll b/cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll new file mode 100644 index 000000000000..f90dc8b66d66 --- /dev/null +++ b/cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll @@ -0,0 +1,1058 @@ +/** + * Provides a class hierarchy corresponding to a parse tree of C++ regular + * expressions, and an implementation of `RegexTreeViewSig` for use with the + * shared ReDoS analysis libraries. + * + * The target dialect is **ECMAScript** (the default for `std::regex`). + * + * Usage: + * ```ql + * import semmle.code.cpp.regex.RegexTreeView + * // RegExpTerm, RegExpGroup, etc. are available directly. + * ``` + */ + +import cpp +private import semmle.code.cpp.regex.internal.ParseRegExp +private import semmle.code.cpp.regex.RegexFlowConfigs as RFC +private import codeql.regex.RegexTreeView +// Export the implementation both as `RegexTreeView` (for use as a functor +// argument) and in the top-level scope (for direct import). +import Impl as RegexTreeView +import Impl + +/** Gets the root parse-tree term for `re`, if it has been parsed. */ +RegExpTerm getParsedRegExp(StringLiteral re) { result.getRegex() = re and result.isRootTerm() } + +// --------------------------------------------------------------------------- +// newtype TRegExpParent +// --------------------------------------------------------------------------- +/** + * An element that is either a regex literal (the root of a parse tree) or a + * regex term (a node in the parse tree). + * + * For sequences and alternations we require at least one child; otherwise the + * node is represented as its sole element instead. + */ +private newtype TRegExpParent = + /** A string literal used as a regular expression. */ + TRegExpLiteral(RegExp re) or + /** A quantified term. */ + TRegExpQuantifier(RegExp re, int start, int end) { re.qualifiedItem(start, end, _, _) } or + /** A sequence of two or more items. */ + TRegExpSequence(RegExp re, int start, int end) { + re.sequence(start, end) and + exists(seqChild(re, start, end, 1)) // require at least two children + } or + /** An alternation (`a|b`). */ + TRegExpAlt(RegExp re, int start, int end) { + re.alternation(start, end) and + exists(int part_end | re.alternationOption(start, end, start, part_end) and part_end < end) // require at least two alternatives + } or + /** A character class (`[...]`). */ + TRegExpCharacterClass(RegExp re, int start, int end) { re.charSet(start, end) } or + /** A character range inside a character class (`a-z`). */ + TRegExpCharacterRange(RegExp re, int start, int end) { re.charRange(_, start, _, _, end) } or + /** + * A POSIX bracket sub-expression inside a character class, such as + * `[:alpha:]`, `[.a.]`, or `[=a=]`. These are only recognized inside a + * `[...]` character class. + */ + TRegExpPosixBracket(RegExp re, int start, int end, string kind) { + re.posixBracketExpression(start, end, kind) + } or + /** A group (`(...)`, `(?:...)`, `(?...)`, etc.). */ + TRegExpGroup(RegExp re, int start, int end) { re.group(start, end) } or + /** A special (meta) character (`.`, `^`, `$`, `\b`, `\B`). */ + TRegExpSpecialChar(RegExp re, int start, int end) { re.specialCharacter(start, end, _) } or + /** A normal character or escape sequence (not a special character). */ + TRegExpNormalChar(RegExp re, int start, int end) { + re.normalCharacterSequence(start, end) + or + re.escapedCharacter(start, end) and + not re.specialCharacter(start, end, _) + } or + /** A back-reference (`\1`, `\k`). */ + TRegExpBackRef(RegExp re, int start, int end) { re.backreference(start, end) } + +// --------------------------------------------------------------------------- +// Helper: sequence children +// --------------------------------------------------------------------------- +pragma[nomagic] +private int seqChildEnd(RegExp re, int start, int end, int i) { + result = seqChild(re, start, end, i).getEnd() +} + +private RegExpTerm seqChild(RegExp re, int start, int end, int i) { + re.sequence(start, end) and + ( + i = 0 and + result.getRegex() = re and + result.getStart() = start and + exists(int itemEnd | + re.item(start, itemEnd) and + result.getEnd() = itemEnd + ) + or + i > 0 and + result.getRegex() = re and + exists(int itemStart | itemStart = seqChildEnd(re, start, end, i - 1) | + result.getStart() = itemStart and + re.item(itemStart, result.getEnd()) + ) + ) +} + +// --------------------------------------------------------------------------- +// Module Impl (implements RegexTreeViewSig) +// --------------------------------------------------------------------------- +/** An implementation that satisfies the `RegexTreeViewSig` signature. */ +module Impl implements RegexTreeViewSig { + // ------------------------------------------------------------------------- + // RegExpParent + // ------------------------------------------------------------------------- + /** + * An element containing a regular expression term: either the literal itself + * or a term node. + */ + class RegExpParent extends TRegExpParent { + /** Gets a textual representation of this element. */ + string toString() { result = "RegExpParent" } + + /** Gets the `i`th child term. */ + abstract RegExpTerm getChild(int i); + + /** Gets any child term. */ + RegExpTerm getAChild() { result = this.getChild(_) } + + /** Gets the number of child terms. */ + int getNumChild() { result = count(this.getAChild()) } + + /** Gets the last child term. */ + RegExpTerm getLastChild() { result = this.getChild(this.getNumChild() - 1) } + + /** Gets the underlying regex. */ + abstract RegExp getRegex(); + + /** Gets a primary QL class for this element. */ + string getAPrimaryQlClass() { result = "RegExpParent" } + + /** Gets a comma-separated list of primary QL classes for this element. */ + final string getPrimaryQlClasses() { result = concat(this.getAPrimaryQlClass(), ",") } + } + + // ------------------------------------------------------------------------- + // RegExpLiteral + // ------------------------------------------------------------------------- + /** A string literal used as a regular expression. */ + class RegExpLiteral extends TRegExpLiteral, RegExpParent { + RegExp re; + + RegExpLiteral() { this = TRegExpLiteral(re) } + + override RegExpTerm getChild(int i) { i = 0 and result.getRegex() = re and result.isRootTerm() } + + /** + * Holds if dot `.` matches all characters including newlines. + * + * ECMAScript `std::regex` has no dot-all flag: neither `icase` nor + * `multiline` change the behavior of `.`. This predicate therefore does + * not hold for any C++ `std::regex` literal. + */ + predicate isDotAll() { none() } + + /** + * Holds if matching is case-insensitive. Reflects whether the underlying + * `std::basic_regex` was constructed with + * `std::regex_constants::icase` (directly or in a bitwise-OR combination). + */ + predicate isIgnoreCase() { RFC::hasIgnoreCaseFlag(re) } + + /** + * Holds if `^` and `$` match at line boundaries (`std::regex_constants::multiline`, + * C++17+). + */ + predicate isMultiline() { RFC::hasMultilineFlag(re) } + + /** Gets a string representing all flags for this regex. */ + string getFlags() { + result = + strictconcat(string f | + this.isIgnoreCase() and f = "i" + or + this.isMultiline() and f = "m" + | + f order by f + ) + } + + override RegExp getRegex() { result = re } + + /** Gets the primary QL class for this element. */ + override string getAPrimaryQlClass() { result = "RegExpLiteral" } + } + + // ------------------------------------------------------------------------- + // RegExpTerm (base class for all parse-tree nodes) + // ------------------------------------------------------------------------- + /** + * A regular expression term - a node in the parse tree of a regex literal. + */ + class RegExpTerm extends RegExpParent { + RegExp re; + int start; + int end; + + RegExpTerm() { + this = TRegExpAlt(re, start, end) + or + this = TRegExpBackRef(re, start, end) + or + this = TRegExpCharacterClass(re, start, end) + or + this = TRegExpCharacterRange(re, start, end) + or + this = TRegExpPosixBracket(re, start, end, _) + or + this = TRegExpNormalChar(re, start, end) + or + this = TRegExpGroup(re, start, end) + or + this = TRegExpQuantifier(re, start, end) + or + this = TRegExpSequence(re, start, end) + or + this = TRegExpSpecialChar(re, start, end) + } + + /** Gets the outermost (root) term of this regular expression. */ + RegExpTerm getRootTerm() { + this.isRootTerm() and result = this + or + result = this.getParent().(RegExpTerm).getRootTerm() + } + + /** + * Holds if this term is part of a string literal that is (potentially) + * used as a regular expression. + */ + predicate isUsedAsRegExp() { any() } + + /** + * Holds if this is the root term of a regular expression (i.e., it covers + * the full string value). + */ + predicate isRootTerm() { start = 0 and end = re.getText().length() } + + override RegExpTerm getChild(int i) { + result = this.(RegExpAlt).getChild(i) + or + result = this.(RegExpBackRef).getChild(i) + or + result = this.(RegExpCharacterClass).getChild(i) + or + result = this.(RegExpCharacterRange).getChild(i) + or + result = this.(RegExpPosixBracket).getChild(i) + or + result = this.(RegExpNormalChar).getChild(i) + or + result = this.(RegExpGroup).getChild(i) + or + result = this.(RegExpQuantifier).getChild(i) + or + result = this.(RegExpSequence).getChild(i) + or + result = this.(RegExpSpecialChar).getChild(i) + } + + /** + * Gets the parent term (or the literal if this is the root term). + */ + RegExpParent getParent() { result.getAChild() = this } + + override RegExp getRegex() { result = re } + + /** Gets the start offset of this term in the string value. */ + int getStart() { result = start } + + /** Gets the end offset (exclusive) of this term in the string value. */ + int getEnd() { result = end } + + override string toString() { result = re.getText().substring(start, end) } + + /** + * Gets the source location of the enclosing string literal. + * Use `hasLocationInfo` to get offsets within the string. + */ + Location getLocation() { result = re.getLocation() } + + /** + * Holds if this term is found at the given source location. + * + * The location maps back to character offsets within the C++ string literal. + * Because the literal's start column already accounts for the opening `"`, + * we add 1 to skip the quote itself. + */ + predicate hasLocationInfo( + string filepath, int startline, int startcolumn, int endline, int endcolumn + ) { + exists(Location loc | loc = re.getLocation() | + loc.hasLocationInfo(filepath, startline, _, _, _) and + startcolumn = loc.getStartColumn() + start + 1 and + endline = startline and + endcolumn = loc.getStartColumn() + end // end is exclusive, so -1+1 = 0 offset + ) + } + + /** Gets the file this term is in. */ + File getFile() { result = re.getFile() } + + /** Gets the raw source text of this term. */ + string getRawValue() { result = this.toString() } + + /** Gets the string literal enclosing this term. */ + RegExpLiteral getLiteral() { result = TRegExpLiteral(re) } + + /** Gets the term matched (textually) immediately before this one, if any. */ + RegExpTerm getPredecessor() { + exists(RegExpTerm parent | parent = this.getParent() | + result = parent.(RegExpSequence).previousElement(this) + or + not exists(parent.(RegExpSequence).previousElement(this)) and + not parent instanceof RegExpSubPattern and + result = parent.getPredecessor() + ) + } + + /** Gets the term matched (textually) immediately after this one, if any. */ + RegExpTerm getSuccessor() { + exists(RegExpTerm parent | parent = this.getParent() | + result = parent.(RegExpSequence).nextElement(this) + or + not exists(parent.(RegExpSequence).nextElement(this)) and + not parent instanceof RegExpSubPattern and + result = parent.getSuccessor() + ) + } + + /** Gets the primary QL class for this term. */ + override string getAPrimaryQlClass() { result = "RegExpTerm" } + } + + // ------------------------------------------------------------------------- + // Quantifiers + // ------------------------------------------------------------------------- + /** + * A quantified regular expression term (`a*`, `a+`, `a?`, `a{n,m}`, etc.). + */ + class RegExpQuantifier extends RegExpTerm, TRegExpQuantifier { + int part_end; + boolean may_repeat_forever; + + RegExpQuantifier() { + this = TRegExpQuantifier(re, start, end) and + re.qualifiedPart(start, part_end, end, _, may_repeat_forever) + } + + override RegExpTerm getChild(int i) { + i = 0 and + result.getRegex() = re and + result.getStart() = start and + result.getEnd() = part_end + } + + /** Holds if this quantifier may match an unbounded number of times. */ + predicate mayRepeatForever() { may_repeat_forever = true } + + /** Gets the textual qualifier (e.g., `*`, `+`, `{2,5}`). */ + string getQualifier() { result = re.getText().substring(part_end, end) } + + override string getAPrimaryQlClass() { result = "RegExpQuantifier" } + } + + /** + * A quantifier that permits unlimited repetitions (`*`, `+`, or `{n,}`). + */ + class InfiniteRepetitionQuantifier extends RegExpQuantifier { + InfiniteRepetitionQuantifier() { this.mayRepeatForever() } + } + + /** A star-quantified term (`a*`). */ + class RegExpStar extends InfiniteRepetitionQuantifier { + RegExpStar() { this.getQualifier().charAt(0) = "*" } + + override string getAPrimaryQlClass() { result = "RegExpStar" } + } + + /** A plus-quantified term (`a+`). */ + class RegExpPlus extends InfiniteRepetitionQuantifier { + RegExpPlus() { this.getQualifier().charAt(0) = "+" } + + override string getAPrimaryQlClass() { result = "RegExpPlus" } + } + + /** An optional term (`a?`). */ + class RegExpOpt extends RegExpQuantifier { + RegExpOpt() { this.getQualifier().charAt(0) = "?" } + + override string getAPrimaryQlClass() { result = "RegExpOpt" } + } + + /** + * A range-quantified term (`a{2}`, `a{2,5}`, `a{2,}`). + */ + class RegExpRange extends RegExpQuantifier { + string upper; + string lower; + + RegExpRange() { re.multiples(part_end, end, lower, upper) } + + /** Gets the lower bound of this range. */ + int getLowerBound() { result = lower.toInt() } + + /** + * Gets the upper bound of this range, if any. + * + * For `{n}`, both lower and upper are `n`. + * For `{n,}`, there is no upper bound (this predicate has no result). + */ + int getUpperBound() { result = upper.toInt() } + + override string getAPrimaryQlClass() { result = "RegExpRange" } + } + + // ------------------------------------------------------------------------- + // Sequences and alternations + // ------------------------------------------------------------------------- + /** + * A sequence term - two or more items in a row. + * + * Example: `ab` is a sequence of `a` and `b`. + */ + class RegExpSequence extends RegExpTerm, TRegExpSequence { + RegExpSequence() { this = TRegExpSequence(re, start, end) } + + override RegExpTerm getChild(int i) { result = seqChild(re, start, end, i) } + + /** Gets the element preceding `element` in this sequence. */ + RegExpTerm previousElement(RegExpTerm element) { element = this.nextElement(result) } + + /** Gets the element following `element` in this sequence. */ + RegExpTerm nextElement(RegExpTerm element) { + exists(int i | + element = this.getChild(i) and + result = this.getChild(i + 1) + ) + } + + override string getAPrimaryQlClass() { result = "RegExpSequence" } + } + + /** + * An alternation term (`a|b`). + */ + class RegExpAlt extends RegExpTerm, TRegExpAlt { + RegExpAlt() { this = TRegExpAlt(re, start, end) } + + override RegExpTerm getChild(int i) { + i = 0 and + result.getRegex() = re and + result.getStart() = start and + exists(int part_end | + re.alternationOption(start, end, start, part_end) and + result.getEnd() = part_end + ) + or + i > 0 and + result.getRegex() = re and + exists(int part_start | + part_start = this.getChild(i - 1).getEnd() + 1 // skip the `|` + | + result.getStart() = part_start and + re.alternationOption(start, end, part_start, result.getEnd()) + ) + } + + override string getAPrimaryQlClass() { result = "RegExpAlt" } + } + + // ------------------------------------------------------------------------- + // Character escapes and normal characters + // ------------------------------------------------------------------------- + /** + * A normal character in a regular expression (including escape sequences). + */ + additional class RegExpNormalChar extends RegExpTerm, TRegExpNormalChar { + RegExpNormalChar() { this = TRegExpNormalChar(re, start, end) } + + /** Holds if this is a valid Unicode character (always true here). */ + predicate isCharacter() { any() } + + /** Gets the string representation of this character. */ + string getValue() { result = re.getText().substring(start, end) } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpNormalChar" } + } + + /** + * A character escape in a regular expression (`\.`, `\n`, etc.). + * This is a type alias; `RegExpCharEscape` == `RegExpEscape`. + */ + class RegExpCharEscape = RegExpEscape; + + /** + * An escaped regular expression term - starts with `\` and is not a + * back-reference. + */ + class RegExpEscape extends RegExpNormalChar { + RegExpEscape() { re.escapedCharacter(start, end) } + + /** + * Gets the name of the escaped character; for example, `w` for `\w` and + * `n` for `\n`. + */ + override string getValue() { + this.isIdentityEscape() and result = this.getUnescaped() + or + this.getUnescaped() = "n" and result = "\n" + or + this.getUnescaped() = "r" and result = "\r" + or + this.getUnescaped() = "t" and result = "\t" + or + this.getUnescaped() = "f" and result = 12.toUnicode() + or + this.getUnescaped() = "v" and result = 11.toUnicode() + } + + /** Holds if this escape's name is the character following the backslash. */ + predicate isIdentityEscape() { not this.getUnescaped() in ["n", "r", "t", "f", "v"] } + + override string getAPrimaryQlClass() { result = "RegExpEscape" } + + /** Gets the part of the term following the backslash (e.g., `w` for `\w`). */ + string getUnescaped() { result = re.getText().substring(start + 1, end) } + } + + /** + * A character-class escape - an escape that denotes a set of characters. + * + * Examples: `\d`, `\D`, `\w`, `\W`, `\s`, `\S`. + */ + class RegExpCharacterClassEscape extends RegExpEscape { + RegExpCharacterClassEscape() { this.getValue() in ["d", "D", "s", "S", "w", "W"] } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpCharacterClassEscape" } + } + + // ------------------------------------------------------------------------- + // Character classes [...] + // ------------------------------------------------------------------------- + /** + * A character class in a regular expression (`[a-z]`, `[^0-9]`, etc.). + */ + class RegExpCharacterClass extends RegExpTerm, TRegExpCharacterClass { + RegExpCharacterClass() { this = TRegExpCharacterClass(re, start, end) } + + /** Holds if this character class is inverted (`[^...]`). */ + predicate isInverted() { re.getChar(start + 1) = "^" } + + /** + * Holds if this character class matches any character. + * + * That is, `[^]` (empty inverted class) or `[\w\W]`/`[\d\D]`/`[\s\S]` + * (complementary class escapes). + */ + predicate isUniversalClass() { + // [^] - empty inverted class + this.isInverted() and not exists(this.getAChild()) + or + // [\w\W] and similar - two complementary class escapes + not this.isInverted() and + exists(string cce1, string cce2 | + cce1 = this.getAChild().(RegExpCharacterClassEscape).getValue() and + cce2 = this.getAChild().(RegExpCharacterClassEscape).getValue() + | + cce1 != cce2 and cce1.toLowerCase() = cce2.toLowerCase() + ) + } + + override RegExpTerm getChild(int i) { + i = 0 and + result.getRegex() = re and + exists(int itemStart, int itemEnd | + result.getStart() = itemStart and + re.char_set_start(start, itemStart) and + re.char_set_child(start, itemStart, itemEnd) and + result.getEnd() = itemEnd + ) + or + i > 0 and + result.getRegex() = re and + exists(int itemStart | itemStart = this.getChild(i - 1).getEnd() | + result.getStart() = itemStart and + re.char_set_child(start, itemStart, result.getEnd()) + ) + } + + override string getAPrimaryQlClass() { result = "RegExpCharacterClass" } + } + + /** + * A character range inside a character class (`a-z`). + */ + class RegExpCharacterRange extends RegExpTerm, TRegExpCharacterRange { + int lower_end; + int upper_start; + + RegExpCharacterRange() { + this = TRegExpCharacterRange(re, start, end) and + re.charRange(_, start, lower_end, upper_start, end) + } + + /** Holds if this range spans from `lo` to `hi`. */ + predicate isRange(string lo, string hi) { + lo = re.getText().substring(start, lower_end) and + hi = re.getText().substring(upper_start, end) + } + + override RegExpTerm getChild(int i) { + i = 0 and + result.getRegex() = re and + result.getStart() = start and + result.getEnd() = lower_end + or + i = 1 and + result.getRegex() = re and + result.getStart() = upper_start and + result.getEnd() = end + } + + override string getAPrimaryQlClass() { result = "RegExpCharacterRange" } + } + + /** + * A POSIX bracket sub-expression inside a character class. + * + * C++'s ECMAScript-mode `std::regex` grammar (per `[re.grammar]`) + * additionally supports three POSIX bracket forms inside a `[...]` + * character class, none of which are part of ECMA-262 JavaScript: + * + * - POSIX character class: `[:alpha:]`, `[:digit:]`, `[:space:]`, ... + * - Collating symbol: `[.a.]`, `[.tilde.]`, ... + * - Equivalence class: `[=a=]`, `[=A=]`, ... + * + * Each is treated as a single character-matching atom (a class member). + * + * For POSIX character classes with a clean Perl-escape equivalent + * (`digit`, `space`, `word`) - or a *subset* of one (`alpha`, `alnum`, + * `upper`, `lower`, `xdigit`, `blank`) - we map them onto `\d`, `\s`, `\w` + * via `isEscapeClass`, so the shared ReDoS engine can reason about their + * match sets. + * + * Other POSIX classes (`punct`, `cntrl`, `print`, `graph`) as well as + * collating and equivalence classes do NOT fit any of `\d`/`\s`/`\w`: + * - `[:punct:]` matches punctuation, disjoint from `\w`. + * - `[:cntrl:]` matches control characters, disjoint from `\w`. + * - `[:print:]` / `[:graph:]` include punctuation (and, for `print`, + * space), overlapping but not contained in `\w`. + * Mapping them onto `\w` would tell the shared engine that two such atoms + * "overlap" via `\w`, which is unsound (produces both false positives and + * false negatives in ambiguity reasoning). We therefore leave them opaque: + * they are single character-consuming class members, but `isEscapeClass` + * does not hold for them and the shared engine treats them as an unknown + * character set (the same treatment as collating/equivalence classes). + * + * Collating and equivalence classes semantically may match multi-character + * sequences (e.g. `[[.ll.]]`); we model them as single atoms, which is a + * documented approximation. + */ + additional class RegExpPosixBracket extends RegExpTerm, TRegExpPosixBracket { + string kind; + + RegExpPosixBracket() { this = TRegExpPosixBracket(re, start, end, kind) } + + /** + * Gets the kind of this POSIX bracket sub-expression: one of `"class"`, + * `"collating"`, or `"equivalence"`. + */ + string getKind() { result = kind } + + /** + * Gets the inner name of this bracket, without the delimiters. For + * `[:alpha:]` this is `"alpha"`; for `[.tilde.]` this is `"tilde"`; for + * `[=a=]` this is `"a"`. + */ + string getName() { result = re.getText().substring(start + 2, end - 2) } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpPosixBracket" } + } + + // ------------------------------------------------------------------------- + // Special characters (`.`, `^`, `$`, `\b`, `\B`) + // ------------------------------------------------------------------------- + /** + * A special (meta) character in a regular expression. + * + * In ECMAScript: `.`, `^`, `$`, `\b`, `\B`. + */ + additional class RegExpSpecialChar extends RegExpTerm, TRegExpSpecialChar { + string char; + + RegExpSpecialChar() { + this = TRegExpSpecialChar(re, start, end) and + re.specialCharacter(start, end, char) + } + + /** Holds if this constant is a valid Unicode character (always true). */ + predicate isCharacter() { any() } + + /** Gets the character (e.g., `"."`, `"^"`, `"\\b"`). */ + string getChar() { result = char } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpSpecialChar" } + } + + /** A dot (`.`) that matches any character (except possibly newlines). */ + class RegExpDot extends RegExpSpecialChar { + RegExpDot() { this.getChar() = "." } + + override string getAPrimaryQlClass() { result = "RegExpDot" } + } + + /** An anchor term (`^`, `$`). */ + class RegExpAnchor extends RegExpSpecialChar { + RegExpAnchor() { char = ["^", "$"] } + } + + /** + * A dollar assertion `$`, matching the end of the input (or end of a line in + * multiline mode). + * + * NOTE: `std::regex_constants::multiline` is detected by + * `RegexFlowConfigs.qll` (`hasMultilineFlag`) and exposed by + * `RegExpLiteral.isMultiline()`, but the shared `RegexTreeViewSig` + * signature has no hook to parameterize anchor semantics on multiline + * mode. We therefore always model `$` as the end-of-string anchor - + * mirroring the conservative choice made by the JavaScript and Ruby + * ReDoS analyses. Under `multiline`, `$` can additionally match at `\n`, + * which affects rejecting-suffix reasoning for `^`/`$`-anchored + * patterns; this may cause minor false positives/negatives for + * multiline-flagged regexes. Precise modeling would require extending + * `RegexTreeViewSig`, which is out of scope for the C++ port. + */ + class RegExpDollar extends RegExpAnchor { + RegExpDollar() { char = "$" } + + override string getAPrimaryQlClass() { result = "RegExpDollar" } + } + + /** + * A caret assertion `^`, matching the start of the input (or start of a line + * in multiline mode). + * + * See the note on `RegExpDollar` regarding the (currently unmodeled) + * `std::regex_constants::multiline` flag. + */ + class RegExpCaret extends RegExpAnchor { + RegExpCaret() { char = "^" } + + override string getAPrimaryQlClass() { result = "RegExpCaret" } + } + + /** A word-boundary assertion `\b`. */ + class RegExpWordBoundary extends RegExpSpecialChar { + RegExpWordBoundary() { this.getChar() = "\\b" } + + override string getAPrimaryQlClass() { result = "RegExpWordBoundary" } + } + + /** A non-word-boundary assertion `\B`. */ + class RegExpNonWordBoundary extends RegExpSpecialChar { + RegExpNonWordBoundary() { this.getChar() = "\\B" } + + override string getAPrimaryQlClass() { result = "RegExpNonWordBoundary" } + } + + // ------------------------------------------------------------------------- + // Groups (capturing, non-capturing, named, lookahead/lookbehind) + // ------------------------------------------------------------------------- + /** + * A grouped regular expression: `(...)`, `(?:...)`, `(?...)`, or an + * assertion group `(?=...)`, etc. + */ + class RegExpGroup extends RegExpTerm, TRegExpGroup { + RegExpGroup() { this = TRegExpGroup(re, start, end) } + + /** + * Gets the index of this capture group within the enclosing regex literal. + * + * For example, in `((a?).)(?:b)`: + * - `((a?).)` has index 1 + * - `(a?)` has index 2 + * - `(?:b)` has no index (non-capturing) + */ + int getNumber() { result = re.getGroupNumber(start, end) } + + /** Holds if this is a capture group (has an index). */ + predicate isCapture() { exists(this.getNumber()) } + + /** Holds if this is a named capture group. */ + predicate isNamed() { exists(this.getName()) } + + /** Gets the name of this named capture group, if any. */ + string getName() { result = re.getGroupName(start, end) } + + override RegExpTerm getChild(int i) { + result.getRegex() = re and + i = 0 and + re.groupContents(start, end, result.getStart(), result.getEnd()) + } + + override string getAPrimaryQlClass() { result = "RegExpGroup" } + } + + // ------------------------------------------------------------------------- + // Zero-width matches and sub-patterns (lookahead/lookbehind) + // ------------------------------------------------------------------------- + /** + * A zero-width match: an empty group or an assertion. + */ + additional class RegExpZeroWidthMatch extends RegExpGroup { + RegExpZeroWidthMatch() { re.zeroWidthMatch(start, end) } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpZeroWidthMatch" } + } + + /** + * A non-empty zero-width assertion (lookahead or lookbehind). + */ + class RegExpSubPattern extends RegExpZeroWidthMatch { + RegExpSubPattern() { not re.emptyGroup(start, end) } + + /** Gets the operand of this assertion. */ + RegExpTerm getOperand() { + exists(int in_start, int in_end | re.groupContents(start, end, in_start, in_end) | + result.getRegex() = re and + result.getStart() = in_start and + result.getEnd() = in_end + ) + } + } + + /** A lookahead assertion (`(?=...)` or `(?!...)`). */ + abstract class RegExpLookahead extends RegExpSubPattern { } + + /** A positive lookahead assertion (`(?=...)`). */ + class RegExpPositiveLookahead extends RegExpLookahead { + RegExpPositiveLookahead() { re.positiveLookaheadAssertionGroup(start, end, _, _) } + + override string getAPrimaryQlClass() { result = "RegExpPositiveLookahead" } + } + + /** A negative lookahead assertion (`(?!...)`). */ + additional class RegExpNegativeLookahead extends RegExpLookahead { + RegExpNegativeLookahead() { re.negativeLookaheadAssertionGroup(start, end, _, _) } + + override string getAPrimaryQlClass() { result = "RegExpNegativeLookahead" } + } + + /** A lookbehind assertion (`(?<=...)` or `(?`. + */ + class RegExpBackRef extends RegExpTerm, TRegExpBackRef { + RegExpBackRef() { this = TRegExpBackRef(re, start, end) } + + /** Gets the group number this back-reference refers to, if any. */ + int getNumber() { result = re.getBackrefNumber(start, end) } + + /** Gets the group name this back-reference refers to, if any. */ + string getName() { result = re.getBackrefName(start, end) } + + /** Gets the group that this back-reference refers to. */ + RegExpGroup getGroup() { + this.hasLiteralAndNumber(result.getLiteral(), result.getNumber()) or + this.hasLiteralAndName(result.getLiteral(), result.getName()) + } + + pragma[nomagic] + private predicate hasLiteralAndNumber(RegExpLiteral literal, int number) { + literal = this.getLiteral() and + number = this.getNumber() + } + + pragma[nomagic] + private predicate hasLiteralAndName(RegExpLiteral literal, string name) { + literal = this.getLiteral() and + name = this.getName() + } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpBackRef" } + } + + // ------------------------------------------------------------------------- + // RegExpConstant + // ------------------------------------------------------------------------- + /** + * A constant regular expression term - a sequence of characters that matches + * a fixed string. Currently this is always a single character (or escape + * sequence). + */ + class RegExpConstant extends RegExpTerm { + string value; + + RegExpConstant() { + this = TRegExpNormalChar(re, start, end) and + not this instanceof RegExpCharacterClassEscape and + not exists(int qstart, int qend | re.qualifiedPart(_, qstart, qend, _, _) | + qstart <= start and end <= qend + ) and + value = this.(RegExpNormalChar).getValue() + } + + /** Holds if this constant is a valid Unicode character (always true). */ + predicate isCharacter() { any() } + + /** Gets the string matched by this constant term. */ + string getValue() { result = value } + + override RegExpTerm getChild(int i) { none() } + + override string getAPrimaryQlClass() { result = "RegExpConstant" } + } + + // ------------------------------------------------------------------------- + // Top + // ------------------------------------------------------------------------- + /** The common supertype of all regex-related elements. */ + class Top = RegExpParent; + + // ------------------------------------------------------------------------- + // Signature predicates + // ------------------------------------------------------------------------- + /** + * Holds if `term` is an escape class (e.g., `\d`), and `clazz` is the + * character identifying the class (e.g., `"d"`). + */ + predicate isEscapeClass(RegExpTerm term, string clazz) { + exists(RegExpCharacterClassEscape escape | term = escape | escape.getValue() = clazz) + or + // Map POSIX bracket sub-expressions to the shared engine's escape-class + // signature. Only POSIX character classes (`[:name:]`) whose match set + // is `\d`, `\s`, or `\w` - or a subset of one - are mapped; other + // POSIX classes (`punct`, `cntrl`, `print`, `graph`) and the + // collating/equivalence forms have no meaningful escape-class equivalent + // and are left opaque (they still parse as single character-class atoms, + // but the shared engine treats their character set as unknown). + exists(RegExpPosixBracket posix, string name | + term = posix and + posix.getKind() = "class" and + name = posix.getName() + | + // `[:digit:]` = `\d`; `[:xdigit:]` is a superset, treated as `\d` + // (under-approximation - sound for the shared engine's overlap + // reasoning, since it will only ever detect fewer intersections, not + // invent them). + name = ["digit", "xdigit"] and clazz = "d" + or + // `[:space:]` = `\s`; `[:blank:]` is a subset (space + tab). + name = ["space", "blank"] and clazz = "s" + or + // `[:word:]` = `\w`; `[:alpha:]`, `[:alnum:]`, `[:upper:]`, `[:lower:]` + // are all subsets of `\w`. + name = ["word", "alpha", "alnum", "upper", "lower"] and clazz = "w" + ) + } + + /** + * ECMAScript (`std::regex`) does not support possessive quantifiers, + * so this never holds. + */ + predicate isPossessive(RegExpQuantifier term) { none() } + + /** + * Holds if the regex that `term` is part of is used in a way that ignores + * any leading prefix of the input it is matched against. + * + * Conservative over-approximation: always holds, matching the JavaScript + * implementation's placeholder. TODO (Phase 2): refine using dataflow. + */ + predicate matchesAnyPrefix(RegExpTerm term) { any() } + + /** + * Holds if the regex that `term` is part of is used in a way that ignores + * any trailing suffix of the input it is matched against. + * + * Conservative over-approximation: always holds. TODO (Phase 2): refine. + */ + predicate matchesAnySuffix(RegExpTerm term) { any() } + + /** + * Holds if the regular expression should not be considered by the shared + * analysis. + * + * We exclude: + * - Regexes in files without a relative path (i.e., external/library code). + * - Regexes with an excessive number of `.*` sub-expressions (performance). + */ + predicate isExcluded(RegExpParent parent) { + not exists(parent.getRegex().getFile().getRelativePath()) + or + count(int i | exists(parent.getRegex().getText().regexpFind("\\.\\*", i, _)) | i) > 10 + } + + /** + * Holds if `root` has the `i` flag for case-insensitive matching, i.e. the + * underlying `std::basic_regex` was constructed with + * `std::regex_constants::icase`. + */ + predicate isIgnoreCase(RegExpTerm root) { + root.isRootTerm() and + root.getLiteral().isIgnoreCase() + } + + /** + * Holds if `root` has the `s` (dot-all) flag, making `.` match newlines. + * + * ECMAScript `std::regex` has no dot-all flag, so this predicate never + * holds for C++ regexes. It is retained to satisfy the shared + * `RegexTreeViewSig` signature. + */ + predicate isDotAll(RegExpTerm root) { + root.isRootTerm() and + root.getLiteral().isDotAll() + } +} diff --git a/cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll b/cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll new file mode 100644 index 000000000000..371a65d99334 --- /dev/null +++ b/cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll @@ -0,0 +1,1779 @@ +/** + * Library for parsing C++ regular expressions. + * + * The parser is structured as a grammar-agnostic **core** (this module's + * `RegExp` abstract class) that expresses the shared, structural layer of + * a regex (sequences, alternations, groups, quantified items, character + * classes, POSIX bracket sub-expressions, character/normal-character + * tokenization, and failure-to-parse reporting) purely in terms of a small + * set of **dialect hook** abstract predicates. Concrete grammar dialects + * supply the raw lexical decisions - "is this position an escape + * backslash?", "is this a group open?", "is this a quantifier?", etc. - by + * overriding those hooks. + * + * The grammar dialects modeled today are: + * - ECMAScript (`EcmaRegExp`), the default used by `std::regex` (i.e. + * `std::regex_constants::ECMAScript`); + * - POSIX Extended Regular Expressions (`EreRegExp`), selected via the + * `extended`, `egrep`, and `awk` flags; and + * - POSIX Basic Regular Expressions (`BreRegExp`), selected via the + * `basic` and `grep` flags. + * + * Every grammar the standard defines now has a concrete parser subclass, + * so nothing is grammar-excluded from analysis. Whether a regex is + * ReDoS-eligible is a separate axis, handled by + * `RegexFlowConfigs::isBacktrackingEngine`. + * + * The single-grammar-per-literal assumption is still baked into this + * module: because each parsed literal's grammar is uniquely determined by + * `regexGrammar` (a functional classifier over the construction-site flag + * argument), a literal is exactly one of `EcmaRegExp`, `EreRegExp`, or + * `BreRegExp`. The "same literal used under two different grammars" case + * is not yet handled and is deferred to the phase that actually introduces + * overlap; at that point this file will need to revisit how + * `TRegExpParent` identity relates to grammar. + */ + +import cpp +private import semmle.code.cpp.regex.RegexFlowConfigs as RFC + +// =========================================================================== +// Core: grammar-agnostic parser +// =========================================================================== +/** + * A string literal used as a regular expression. + * + * A `StringLiteral` is treated as a regex only when dataflow indicates it + * flows to a `std::basic_regex` construction/assignment or to a + * `regex_match`/`regex_search`/`regex_replace`/iterator call. Every + * `std::regex` grammar has a concrete parser subclass + * (`EcmaRegExp`/`EreRegExp`/`BreRegExp`), so no regex is excluded from + * analysis on grammar grounds. + * + * This class is abstract: its structural predicates are expressed in terms + * of dialect hooks (see below), and concrete grammar subclasses supply the + * dialect-specific token-recognition behavior. + */ +abstract class RegExp extends StringLiteral { + RegExp() { + RFC::usedAsRegex(this) and + RFC::hasConcreteGrammar(RFC::regexGrammar(this)) + } + + /** Gets the `i`th character of this regex string. */ + string getChar(int i) { result = this.getValue().charAt(i) } + + /** Gets the text of this regex (the string value of the literal). */ + string getText() { result = this.getValue() } + + /** Gets the grammar dialect of this regex. */ + abstract RFC::TRegexGrammar getGrammar(); + + // --------------------------------------------------------------------------- + // Dialect hooks + // + // These predicates capture every place where the parser branches on raw + // metacharacters. Grammar dialects override them; the shared structural + // predicates below MUST NOT re-derive these facts (in particular, must not + // re-inspect backslashes to compute escaping) - they call the hooks. + // --------------------------------------------------------------------------- + /** + * Dialect hook - overridden per grammar. + * + * Holds if the character at position `pos` is a backslash that escapes the + * next character. + */ + abstract predicate escapingChar(int pos); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if an escaped character sequence spans `[start, end)` (a hex/unicode + * escape, a legacy octal escape, or a simple `\X` single-character escape), + * but not a back-reference. + */ + abstract predicate escapedCharacter(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if the character at `[start, end)` is a special (metacharacter) + * position-assertion or wildcard, with `char` giving its canonical + * representation (e.g. `.`, `^`, `$`, `\b`, `\B` in ECMAScript). + */ + abstract predicate specialCharacter(int start, int end, string char); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if position `i` is an alternation divider (e.g. `|` in ECMAScript). + */ + abstract predicate isOptionDivider(int i); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if position `i` opens a group (e.g. an unescaped `(` in ECMAScript). + */ + abstract predicate isGroupStart(int i); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if position `i` closes a group (e.g. an unescaped `)` in ECMAScript, + * or the leading `\` of a `\)` in BRE). The end-delimiter span itself is + * given by `group_end`. + */ + abstract predicate isGroupEnd(int i); + + /** + * Dialect hook - overridden per grammar. + * + * Matches the closing delimiter of a group, yielding `[start, end)` where + * `start` is the first position of the closing delimiter (i.e. the same + * position `isGroupEnd` reports) and `end` is the first position past the + * whole group. Single-character in ECMAScript/ERE (the `)`); two-character + * in BRE (the whole `\)` pair). + */ + abstract predicate group_end(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Matches the start of any group construct, yielding `[start, end)` where + * `end` is the first position of the group content. + */ + abstract predicate group_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Simple capturing group open (e.g. `(` in ECMAScript, `\(` in BRE). + */ + abstract predicate simple_group_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Non-capturing group open (`(?:` in ECMAScript). Grammars without a + * non-capturing form leave this empty. + */ + abstract predicate non_capturing_group_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * ECMAScript named group open (`(?`). Grammars without named groups + * leave this empty. + */ + abstract predicate ecma_named_group_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Positive lookahead open (`(?=` in ECMAScript). Grammars without + * lookaround leave this empty. + */ + abstract predicate lookahead_assertion_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Negative lookahead open (`(?!` in ECMAScript). Grammars without + * lookaround leave this empty. + */ + abstract predicate negative_lookahead_assertion_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Positive lookbehind open (`(?<=` in ECMAScript). Grammars without + * lookaround leave this empty. + */ + abstract predicate lookbehind_assertion_start(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Negative lookbehind open (`(?` in ECMAScript) spanning `[start, end)`. + */ + abstract predicate named_backreference(int start, int end, string name); + + /** + * Dialect hook - overridden per grammar. + * + * Holds if a back-reference spans `[start, end)`. + */ + abstract predicate backreference(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Gets the name of the named group at `[start, end)`, if any. + */ + abstract string getGroupName(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Gets the 1-based capture index of the group at `[start, end)`, if it is a + * capturing group under this grammar's numbering rules. + */ + abstract int getGroupNumber(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Gets the name of the back-reference at `[start, end)`, if any. + */ + abstract string getBackrefName(int start, int end); + + /** + * Dialect hook - overridden per grammar. + * + * Gets the number of the back-reference at `[start, end)`, if any. + */ + abstract int getBackrefNumber(int start, int end); + + // --------------------------------------------------------------------------- + // Shared helpers + // --------------------------------------------------------------------------- + /** + * Shared helper: holds if position `index` is a decimal digit. Digit + * recognition is grammar-independent. + */ + pragma[inline] + predicate isDecimalDigit(int index) { + this.getChar(index) = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"] + } + + // --------------------------------------------------------------------------- + // POSIX bracket sub-expressions (shared structural layer) + // + // The POSIX bracket forms `[:class:]`, `[.a.]`, `[=a=]` are shared by all + // C++ std::regex grammars (they live inside `[...]` character classes), so + // this whole layer is grammar-agnostic and lives in the core. It only + // consumes the `escapingChar` hook. + // --------------------------------------------------------------------------- + /** + * Shared structural predicate. + * + * Holds if `[start, end)` looks lexically like a POSIX bracket + * sub-expression: an unescaped `[` at `start` followed by a mark + * (`:`, `.`, or `=`) and terminated by the earliest matching `mark]`. + * + * This is a purely lexical recognition - it does NOT check whether the + * span is actually nested inside an outer `[...]` character class. The + * nesting requirement is applied on top of this in `posixBracketExpression`. + * + * Separating the lexical recognition from the nesting gate keeps the + * predicate definitions cleanly stratified (no self-recursion under + * negation). + */ + private predicate posixBracketCandidate(int start, int end, string kind) { + exists(string mark | + this.getChar(start) = "[" and + not this.escapingChar(start) and + ( + this.getChar(start + 1) = ":" and kind = "class" and mark = ":" + or + this.getChar(start + 1) = "." and kind = "collating" and mark = "." + or + this.getChar(start + 1) = "=" and kind = "equivalence" and mark = "=" + ) + | + end - 2 = min(int i | i > start + 1 and this.getChar(i) = mark and this.getChar(i + 1) = "]") + ) + } + + /** + * Shared structural predicate. + * + * Holds if `pos` is an unescaped `[` or `]` that acts as a genuine + * character-class delimiter - i.e. it is NOT part of any POSIX bracket + * candidate (neither the opening `[`, the closing `]`, nor any interior + * character of one). + * + * Used to decide, purely lexically, whether a POSIX bracket candidate is + * nested inside an outer, still-open `[...]` character class. + */ + private predicate structuralBracket(int pos) { + (this.nonEscapedCharAt(pos) = "[" or this.nonEscapedCharAt(pos) = "]") and + not exists(int s, int e | this.posixBracketCandidate(s, e, _) and s <= pos and pos < e) + } + + /** + * Shared structural predicate. + * + * Holds if `[start, end)` is a POSIX bracket sub-expression of the given + * `kind`, appearing nested inside another `[...]` character class. + * + * C++'s ECMAScript-mode `std::regex` (per [re.grammar]) additionally supports + * three POSIX bracket forms inside a character class, which plain ECMA-262 + * JavaScript does not: + * - character classes: `[:alpha:]`, `[:digit:]`, `[:space:]`, ... + * - collating symbols: `[.a.]`, `[.tilde.]`, ... + * - equivalence classes: `[=a=]` + * + * `kind` is one of `"class"`, `"collating"`, or `"equivalence"`. `start` + * points at the inner `[` and `end` is one past the outer `]`. + * + * A POSIX bracket sub-expression is only recognized when it is nested + * inside another `[...]` character class, since the standard only gives + * these forms meaning in that context. + * + * Implementation note: the "inside an outer, still-open [...]" gate is + * expressed as an ordinary universal quantification over `structuralBracket` + * positions (which only depend on the purely-lexical + * `posixBracketCandidate` predicate). There is no self-recursion under + * negation, so this predicate is trivially stratified. + */ + predicate posixBracketExpression(int start, int end, string kind) { + this.posixBracketCandidate(start, end, kind) and + // Gate: some structural `[` at position `q < start` must not have been + // closed by any structural `]` at position `r` with `q < r < start`. + exists(int q | + q < start and + this.structuralBracket(q) and + this.nonEscapedCharAt(q) = "[" and + not exists(int r | + q < r and + r < start and + this.structuralBracket(r) and + this.nonEscapedCharAt(r) = "]" + ) + ) + } + + /** + * Shared structural predicate. + * + * Holds if position `p` is inside a POSIX bracket sub-expression (i.e. + * anywhere from its opening `[` at `s` up to and including its closing + * `]` at `e-1`). Used to hide interior positions from the character-class + * scanner and the character tokenizer. + */ + predicate insidePosixBracket(int p) { + exists(this.getChar(p)) and + exists(int s, int e | this.posixBracketExpression(s, e, _) and s <= p and p < e) + } + + /** + * Shared structural predicate. + * + * Holds if position `pos` is a non-escaped `[` or `]` that acts as a + * character-class delimiter - i.e. it is NOT one of the outer brackets of + * a POSIX bracket sub-expression. This is the delimiter set used by + * `char_set_delimiter` and the class-end computation, so that + * `[[:alpha:]]` correctly closes at the final `]` rather than at the + * inner `]` of `:alpha:]`. + */ + private predicate classDelimiterAt(int pos) { + (this.nonEscapedCharAt(pos) = "[" or this.nonEscapedCharAt(pos) = "]") and + not exists(int s, int e | this.posixBracketExpression(s, e, _) | pos = s or pos = e - 1) + } + + // --------------------------------------------------------------------------- + // Character classes (shared structural layer) + // --------------------------------------------------------------------------- + /** + * Shared structural predicate. + * + * Holds if the (non-escaped) character at position `pos` is the `index`-th + * bracket (`[` or `]`) in the string. + * Result is `true` for `[` and `false` for `]`. + */ + private boolean char_set_delimiter(int index, int pos) { + pos = rank[index](int p | this.classDelimiterAt(p)) and + ( + this.nonEscapedCharAt(pos) = "[" and result = true + or + this.nonEscapedCharAt(pos) = "]" and result = false + ) + } + + /** + * Shared structural predicate. Helper for `char_set_start/1`. + * Returns `true` if position `pos` is the start of a character class. + */ + boolean char_set_start(int pos) { + exists(int index | + this.char_set_delimiter(index, pos) = true and + ( + index = 1 and result = true + or + index > 1 and + not this.char_set_delimiter(index - 1, _) = false and + result = false + or + exists(int prev_closing_bracket_pos | + this.char_set_delimiter(index - 1, prev_closing_bracket_pos) = false and + if + exists(int pos_before_prev | + if this.getChar(prev_closing_bracket_pos - 1) = "^" + then pos_before_prev = prev_closing_bracket_pos - 2 + else pos_before_prev = prev_closing_bracket_pos - 1 + | + this.char_set_delimiter(index - 2, pos_before_prev) = true + ) + then + exists(int pos_before_prev | + this.char_set_delimiter(index - 2, pos_before_prev) = true + | + result = this.char_set_start(pos_before_prev).booleanNot() + ) + else result = true + ) + ) + ) + } + + /** + * Shared structural predicate. + * + * Holds if a character class starts at position `start` with content + * starting at `end` (accounting for optional `^`). + */ + predicate char_set_start(int start, int end) { + this.char_set_start(start) = true and + ( + this.getChar(start + 1) = "^" and end = start + 2 + or + not this.getChar(start + 1) = "^" and end = start + 1 + ) + } + + /** Shared structural predicate. Holds if a character class spans `[start, end)`. */ + predicate charSet(int start, int end) { + exists(int inner_start | + this.char_set_start(start, inner_start) and + not this.char_set_start(_, start) + | + // Terminate at the first non-escaped `]` that is NOT the inner `]` of a + // POSIX bracket sub-expression (so `[[:alpha:]]` closes at the outer + // `]`, not at the `]` of `:alpha:]`). + end - 1 = + min(int i | this.classDelimiterAt(i) and this.nonEscapedCharAt(i) = "]" and inner_start < i) + ) + } + + /** Shared structural predicate. An indexed version of `char_set_token`. */ + private predicate char_set_token(int charset_start, int index, int token_start, int token_end) { + token_start = + rank[index](int start, int end | this.char_set_token(charset_start, start, end) | start) and + this.char_set_token(charset_start, token_start, token_end) + } + + /** + * Shared structural predicate. + * A single token (character or escape) inside a character class. + */ + private predicate char_set_token(int charset_start, int start, int end) { + this.char_set_start(charset_start, start) and + ( + this.escapedCharacter(start, end) + or + // A whole POSIX bracket sub-expression is a single class-member token. + this.posixBracketExpression(start, end, _) + or + exists(this.nonEscapedCharAt(start)) and + end = start + 1 and + not this.insidePosixBracket(start) + ) + or + this.char_set_token(charset_start, _, start) and + ( + this.escapedCharacter(start, end) + or + this.posixBracketExpression(start, end, _) + or + exists(this.nonEscapedCharAt(start)) and + end = start + 1 and + not this.getChar(start) = "]" and + not this.insidePosixBracket(start) + ) + } + + /** + * Shared structural predicate. + * + * Holds if the character class starting at `charset_start` contains a child + * (either a single character or a range) spanning `[start, end)`. + */ + predicate char_set_child(int charset_start, int start, int end) { + this.char_set_token(charset_start, start, end) and + not exists(int range_start, int range_end | + this.charRange(charset_start, range_start, _, _, range_end) and + range_start <= start and + range_end >= end + ) + or + this.charRange(charset_start, start, _, _, end) + } + + /** + * Shared structural predicate. + * + * Holds if the character class at `charset_start` contains a character range + * from `[start, lower_end)` to `[upper_start, end)`. + */ + predicate charRange(int charset_start, int start, int lower_end, int upper_start, int end) { + exists(int index | + this.charRangeEnd(charset_start, index) = true and + this.char_set_token(charset_start, index - 2, start, lower_end) and + this.char_set_token(charset_start, index, upper_start, end) + ) + } + + /** + * Shared structural predicate. Helper for `charRange`. + * Returns `true` if the `index`-th token in the character class is the upper + * bound of a range. + */ + private boolean charRangeEnd(int charset_start, int index) { + this.char_set_token(charset_start, index, _, _) and + ( + index in [1, 2] and result = false + or + index > 2 and + exists(int connector_start | + this.char_set_token(charset_start, index - 1, connector_start, _) and + this.nonEscapedCharAt(connector_start) = "-" and + result = + this.charRangeEnd(charset_start, index - 2) + .booleanNot() + .booleanAnd(this.charRangeEnd(charset_start, index - 1).booleanNot()) + ) + or + not exists(int connector_start | + this.char_set_token(charset_start, index - 1, connector_start, _) and + this.nonEscapedCharAt(connector_start) = "-" + ) and + result = false + ) + } + + // --------------------------------------------------------------------------- + // Characters and tokenization (shared structural layer) + // --------------------------------------------------------------------------- + /** Shared structural predicate. Gets the non-escaped character at position `i`, if any. */ + string nonEscapedCharAt(int i) { + result = this.getText().charAt(i) and + not exists(int x, int y | this.escapedCharacter(x, y) and i in [x .. y - 1]) + } + + /** Shared structural predicate. Holds if `index` is inside a character class. */ + predicate inCharSet(int index) { + exists(int x, int y | this.charSet(x, y) and index in [x + 1 .. y - 2]) + } + + /** + * Shared structural predicate. + * + * A 'simple' character is one that does not affect parsing of the regex + * (i.e., it is not a metacharacter). + */ + private predicate simpleCharacter(int start, int end) { + end = start + 1 and + not this.charSet(start, _) and + not this.charSet(_, start + 1) and + // A character inside a POSIX bracket sub-expression is not an + // independent simple character - the whole sub-expression is a single + // class-member atom (handled via `posixBracketExpression`). + not this.insidePosixBracket(start) and + exists(string c | c = this.getChar(start) | + exists(int x, int y, int z | + this.charSet(x, z) and + this.char_set_start(x, y) + | + start = y + or + start = z - 2 + or + start > y and start < z - 2 and not this.charRange(_, _, start, end, _) + ) + or + not this.inCharSet(start) and + not this.isGroupStart(start) and + not this.isGroupEnd(start) and + not this.isOptionDivider(start) and + not c = "[" and + not this.qualifier(start, _, _, _) + ) + } + + /** Shared structural predicate. Holds if a simple or escaped character spans `[start, end)`. */ + predicate character(int start, int end) { + ( + this.simpleCharacter(start, end) and + not exists(int x, int y | this.escapedCharacter(x, y) and x <= start and y >= end) + or + this.escapedCharacter(start, end) + ) and + not exists(int x, int y | this.group_start(x, y) and x <= start and y >= end) and + not exists(int x, int y | this.group_end(x, y) and x <= start and y >= end) and + not exists(int x, int y | this.backreference(x, y) and x <= start and y >= end) + } + + /** Shared structural predicate. Holds if a normal (non-special) character spans `[start, end)`. */ + predicate normalCharacter(int start, int end) { + end = start + 1 and + this.character(start, end) and + not this.specialCharacter(start, end, _) + } + + /** + * Shared structural predicate. + * Holds if `[start, end)` is a maximal run of normal characters (a "constant"). + */ + predicate normalCharacterSequence(int start, int end) { + // A single normal character inside a character class stands alone + this.normalCharacter(start, end) and + this.inCharSet(start) + or + // A maximal run of normal characters outside a character class + exists(int s, int e | + e = max(int i | this.normalCharacterRun(s, i)) and + not this.inCharSet(s) + | + if this.qualifier(e, _, _, _) + then + end = e and start = e - 1 + or + end = e - 1 and start = s and start < end + else ( + end = e and + start = s + ) + ) + } + + private predicate normalCharacterRun(int start, int end) { + ( + this.normalCharacterRun(start, end - 1) + or + start = end - 1 and not this.normalCharacter(start - 1, start) + ) and + this.normalCharacter(end - 1, end) + } + + private predicate characterItem(int start, int end) { + this.normalCharacterSequence(start, end) or + this.escapedCharacter(start, end) or + this.specialCharacter(start, end, _) + } + + // --------------------------------------------------------------------------- + // Qualified items, groups, sequences and alternations (shared structural layer) + // --------------------------------------------------------------------------- + /** + * Shared structural predicate. + * Holds if `[start, end)` is a qualified item (base item + quantifier). + */ + predicate qualifiedItem(int start, int end, boolean maybe_empty, boolean may_repeat_forever) { + this.qualifiedPart(start, _, end, maybe_empty, may_repeat_forever) + } + + /** + * Shared structural predicate. + * Holds if the base item spans `[start, part_end)` and the qualifier spans + * `[part_end, end)`. + */ + predicate qualifiedPart( + int start, int part_end, int end, boolean maybe_empty, boolean may_repeat_forever + ) { + this.baseItem(start, part_end) and + this.qualifier(part_end, end, maybe_empty, may_repeat_forever) + } + + /** Shared structural predicate. Holds if `[start, end)` is a single regex item (possibly qualified). */ + predicate item(int start, int end) { + this.qualifiedItem(start, end, _, _) + or + this.baseItem(start, end) and not this.qualifier(end, _, _, _) + } + + /** Shared structural predicate. Holds if a group spans `[start, end)`. */ + predicate group(int start, int end) { + this.groupContents(start, end, _, _) + or + this.emptyGroup(start, end) + } + + /** Shared structural predicate. Holds if an empty group spans `[start, end)`. */ + predicate emptyGroup(int start, int end) { + exists(int in_end | + this.group_start(start, in_end) and + this.group_end(in_end, end) + ) + } + + /** Shared structural predicate. Holds if the group at `[start, end)` has content at `[in_start, in_end)`. */ + predicate groupContents(int start, int end, int in_start, int in_end) { + this.group_start(start, in_start) and + this.group_end(in_end, end) and + this.top_level(in_start, in_end) + } + + /** Shared structural predicate. Holds if a zero-width match group is at `[start, end)`. */ + predicate zeroWidthMatch(int start, int end) { + this.emptyGroup(start, end) + or + this.negativeLookaheadAssertionGroup(start, end, _, _) + or + this.positiveLookaheadAssertionGroup(start, end, _, _) + or + this.positiveLookbehindAssertionGroup(start, end, _, _) + or + this.negativeLookbehindAssertionGroup(start, end, _, _) + } + + /** Shared structural predicate. Holds if a positive lookahead is at `[start, end)`. */ + predicate positiveLookaheadAssertionGroup(int start, int end, int in_start, int in_end) { + this.lookahead_assertion_start(start, in_start) and + this.groupContents(start, end, in_start, in_end) + } + + /** Shared structural predicate. Holds if a negative lookahead is at `[start, end)`. */ + predicate negativeLookaheadAssertionGroup(int start, int end, int in_start, int in_end) { + this.negative_lookahead_assertion_start(start, in_start) and + this.groupContents(start, end, in_start, in_end) + } + + /** Shared structural predicate. Holds if a positive lookbehind is at `[start, end)`. */ + predicate positiveLookbehindAssertionGroup(int start, int end, int in_start, int in_end) { + this.lookbehind_assertion_start(start, in_start) and + this.groupContents(start, end, in_start, in_end) + } + + /** Shared structural predicate. Holds if a negative lookbehind is at `[start, end)`. */ + predicate negativeLookbehindAssertionGroup(int start, int end, int in_start, int in_end) { + this.negative_lookbehind_assertion_start(start, in_start) and + this.groupContents(start, end, in_start, in_end) + } + + private predicate baseItem(int start, int end) { + this.characterItem(start, end) and + not exists(int x, int y | this.charSet(x, y) and x <= start and y >= end) + or + this.group(start, end) + or + this.charSet(start, end) + or + this.backreference(start, end) + } + + private predicate subsequence(int start, int end) { + ( + start = 0 or + this.group_start(_, start) or + this.isOptionDivider(start - 1) + ) and + this.item(start, end) + or + exists(int mid | + this.subsequence(start, mid) and + this.item(mid, end) + ) + } + + /** + * Shared structural predicate. + * Holds if `[start, end)` is a sequence of two or more items. + */ + predicate sequence(int start, int end) { + this.sequenceOrQualified(start, end) and + not this.qualifiedItem(start, end, _, _) + } + + private predicate sequenceOrQualified(int start, int end) { + this.subsequence(start, end) and + not this.item_start(end) + } + + private predicate item_start(int start) { + this.characterItem(start, _) or + this.isGroupStart(start) or + this.charSet(start, _) or + this.backreference(start, _) + } + + private predicate item_end(int end) { + this.characterItem(_, end) + or + this.group_end(_, end) + or + this.charSet(_, end) + or + this.qualifier(_, end, _, _) + } + + private predicate top_level(int start, int end) { + this.subalternation(start, end, _) and + not this.isOptionDivider(end) + } + + private predicate subalternation(int start, int end, int item_start) { + this.sequenceOrQualified(start, end) and + not this.isOptionDivider(start - 1) and + item_start = start + or + start = end and + not this.item_end(start) and + this.isOptionDivider(end) and + item_start = start + or + exists(int mid | + this.subalternation(start, mid, _) and + this.isOptionDivider(mid) and + item_start = mid + 1 + | + this.sequenceOrQualified(item_start, end) + or + not this.item_start(end) and end = item_start + ) + } + + /** Shared structural predicate. Holds if `[start, end)` is an alternation (two or more options separated by `|`). */ + predicate alternation(int start, int end) { + this.top_level(start, end) and + exists(int less | this.subalternation(start, less, _) and less < end) + } + + /** + * Shared structural predicate. + * Holds if `[start, end)` is an alternation, and `[part_start, part_end)` is + * one of its options. + */ + predicate alternationOption(int start, int end, int part_start, int part_end) { + this.alternation(start, end) and + this.subalternation(start, part_end, part_start) + } + + // --------------------------------------------------------------------------- + // Failure to parse + // --------------------------------------------------------------------------- + /** + * Shared structural predicate. + * Holds if the character at position `i` could not be parsed as part of any + * top-level regex construct. + */ + predicate failedToParse(int i) { + exists(this.getChar(i)) and + not exists(int start, int end | + this.top_level(start, end) and + start <= i and + end > i + ) + } +} + +// =========================================================================== +// ECMAScript dialect +// =========================================================================== +/** + * The ECMAScript-grammar concrete `RegExp` implementation. Supplies all + * dialect hooks with the ECMAScript token-recognition behavior. + * + * Selected for regex literals whose construction-site flag argument does + * not specify a POSIX grammar (i.e. anything not tagged as + * `basic`/`grep`/`extended`/`egrep`/`awk`), matching the default `std::regex` + * grammar. `EcmaRegExp`, `EreRegExp`, and `BreRegExp` are the three + * concrete subclasses of `RegExp`; the grammar of a given literal is + * determined uniquely by `regexGrammar`. + */ +class EcmaRegExp extends RegExp { + EcmaRegExp() { RFC::regexGrammar(this) = RFC::Ecma() } + + override RFC::TRegexGrammar getGrammar() { result = RFC::Ecma() } + + // --------------------------------------------------------------------------- + // Escaping + // --------------------------------------------------------------------------- + /** + * Helper predicate for `escapingChar`. + * Returns `true` if the character at position `pos` is an active backslash + * (i.e., it escapes the next character). Uses a boolean to avoid negation in + * recursive calls. + */ + private boolean escaping(int pos) { + pos = -1 and result = false + or + this.getChar(pos) = "\\" and result = this.escaping(pos - 1).booleanNot() + or + this.getChar(pos) != "\\" and result = false + } + + override predicate escapingChar(int pos) { this.escaping(pos) = true } + + // --------------------------------------------------------------------------- + // Escaped characters + // --------------------------------------------------------------------------- + override predicate escapedCharacter(int start, int end) { + this.escapingChar(start) and + not this.numbered_backreference(start, _, _) and + ( + // Hex escape: \xhh + this.getChar(start + 1) = "x" and end = start + 4 + or + // Unicode escape: \uhhhh (ECMAScript) + this.getChar(start + 1) = "u" and + not this.getChar(start + 2) = "{" and + end = start + 6 + or + // Unicode escape with braces: \u{...} (ECMAScript 2015+) + this.getChar(start + 1) = "u" and + this.getChar(start + 2) = "{" and + end - 1 = min(int i | start + 3 <= i and this.getChar(i) = "}") + or + // Octal (legacy): \0 (NUL), \ooo + this.getChar(start + 1) = "0" and + not this.isDecimalDigit(start + 2) and + end = start + 2 + or + // Simple escape: \n, \r, \t, \f, \v, \b (inside char class), \\, \., etc. + not this.getChar(start + 1) in ["x", "u", "0"] and + not this.isDecimalDigit(start + 1) and + not this.getChar(start + 1) = "k" and + // \k is a named backreference, handled separately + end = start + 2 + ) + } + + // --------------------------------------------------------------------------- + // Special (meta) characters + // --------------------------------------------------------------------------- + /** + * Holds if the character at `[start, end)` is a special (metacharacter) in + * ECMAScript regex syntax. `char` is the canonical representation. + * + * Special characters are: `.`, `^`, `$`, `\b`, `\B`. + * (Note: `\A`, `\Z`, `\G` are Ruby/Python-specific and not part of ECMAScript.) + */ + override predicate specialCharacter(int start, int end, string char) { + not this.inCharSet(start) and + this.character(start, end) and + ( + end = start + 1 and + char = this.getChar(start) and + (char = "$" or char = "^" or char = ".") + or + end = start + 2 and + this.escapingChar(start) and + char = this.getText().substring(start, end) and + (char = "\\b" or char = "\\B") + ) + } + + // --------------------------------------------------------------------------- + // Quantifiers + // --------------------------------------------------------------------------- + /** + * Holds if a repetition quantifier spans `[start, end)`, with `maybe_empty` + * indicating whether zero repetitions are possible, and `may_repeat_forever` + * indicating whether the count is unbounded. + * + * In ECMAScript, lazy quantifiers (`*?`, `+?`, `??`, `{n,m}?`) are treated + * as having the same `maybe_empty`/`may_repeat_forever` as their greedy + * counterparts from the perspective of ReDoS analysis. + */ + override predicate qualifier(int start, int end, boolean maybe_empty, boolean may_repeat_forever) { + this.short_qualifier(start, end, maybe_empty, may_repeat_forever) and + not this.getChar(end) = "?" + or + exists(int short_end | this.short_qualifier(start, short_end, maybe_empty, may_repeat_forever) | + if this.getChar(short_end) = "?" then end = short_end + 1 else end = short_end + ) + } + + /** + * Holds if `[start, end)` is a greedy (non-lazy) quantifier. The `maybe_empty` + * and `may_repeat_forever` booleans characterize the quantifier type. + */ + override predicate short_qualifier( + int start, int end, boolean maybe_empty, boolean may_repeat_forever + ) { + ( + this.getChar(start) = "+" and maybe_empty = false and may_repeat_forever = true + or + this.getChar(start) = "*" and maybe_empty = true and may_repeat_forever = true + or + this.getChar(start) = "?" and maybe_empty = true and may_repeat_forever = false + ) and + end = start + 1 + or + exists(string lower, string upper | + this.multiples(start, end, lower, upper) and + (if lower = "" or lower.toInt() = 0 then maybe_empty = true else maybe_empty = false) and + if upper = "" then may_repeat_forever = true else may_repeat_forever = false + ) + } + + /** + * Holds if `[start, end)` is a `{n}`, `{n,m}`, or `{n,}` quantifier. + * `lower` and `upper` are the textual lower and upper bounds; an empty + * `upper` means "no upper bound". + */ + override predicate multiples(int start, int end, string lower, string upper) { + exists(string text, string match, string inner | + text = this.getText() and + end = start + match.length() and + inner = match.substring(1, match.length() - 1) + | + match = text.regexpFind("\\{[0-9]+\\}", _, start) and + lower = inner and + upper = lower + or + match = text.regexpFind("\\{[0-9]*,[0-9]*\\}", _, start) and + exists(int commaIndex | + commaIndex = inner.indexOf(",") and + lower = inner.prefix(commaIndex) and + upper = inner.suffix(commaIndex + 1) + ) + ) + } + + // --------------------------------------------------------------------------- + // Groups + // --------------------------------------------------------------------------- + override predicate isOptionDivider(int i) { this.nonEscapedCharAt(i) = "|" } + + override predicate isGroupEnd(int i) { this.nonEscapedCharAt(i) = ")" and not this.inCharSet(i) } + + override predicate group_end(int start, int end) { this.isGroupEnd(start) and end = start + 1 } + + override predicate isGroupStart(int i) { + this.nonEscapedCharAt(i) = "(" and not this.inCharSet(i) + } + + override predicate group_start(int start, int end) { + this.non_capturing_group_start(start, end) + or + this.ecma_named_group_start(start, end) + or + this.lookahead_assertion_start(start, end) + or + this.negative_lookahead_assertion_start(start, end) + or + this.lookbehind_assertion_start(start, end) + or + this.negative_lookbehind_assertion_start(start, end) + or + this.simple_group_start(start, end) + } + + /** `(?:...)` - non-capturing group. */ + override predicate non_capturing_group_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) = "?" and + this.getChar(start + 2) = ":" and + end = start + 3 + } + + /** `(...)` - simple capturing group. */ + override predicate simple_group_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) != "?" and + end = start + 1 + } + + /** + * Holds if `[start, end)` is the opening delimiter of an ECMAScript named + * capturing group `(?...)`. The group name spans from `start+3` to + * the `>` character. + */ + override predicate ecma_named_group_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) = "?" and + this.getChar(start + 2) = "<" and + not this.getChar(start + 3) = "=" and + not this.getChar(start + 3) = "!" and + exists(int name_end | + name_end = min(int i | i > start + 3 and this.getChar(i) = ">") and + end = name_end + 1 + ) + } + + /** `(?=...)` - positive lookahead. */ + override predicate lookahead_assertion_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) = "?" and + this.getChar(start + 2) = "=" and + end = start + 3 + } + + /** `(?!...)` - negative lookahead. */ + override predicate negative_lookahead_assertion_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) = "?" and + this.getChar(start + 2) = "!" and + end = start + 3 + } + + /** `(?<=...)` - positive lookbehind. */ + override predicate lookbehind_assertion_start(int start, int end) { + this.isGroupStart(start) and + this.getChar(start + 1) = "?" and + this.getChar(start + 2) = "<" and + this.getChar(start + 3) = "=" and + end = start + 4 + } + + /** `(?...)` still get a + * number in ECMAScript, but `(?:...)` does not. + */ + override int getGroupNumber(int start, int end) { + this.group(start, end) and + not this.non_capturing_group_start(start, _) and + not this.lookahead_assertion_start(start, _) and + not this.negative_lookahead_assertion_start(start, _) and + not this.lookbehind_assertion_start(start, _) and + not this.negative_lookbehind_assertion_start(start, _) and + result = + count(int i | + this.group(i, _) and + i < start and + not this.non_capturing_group_start(i, _) and + not this.lookahead_assertion_start(i, _) and + not this.negative_lookahead_assertion_start(i, _) and + not this.lookbehind_assertion_start(i, _) and + not this.negative_lookbehind_assertion_start(i, _) + ) + 1 + } + + /** + * Gets the name of the ECMAScript named group at `[start, end)`, if any. + * Named groups have the form `(?...)`. + */ + override string getGroupName(int start, int end) { + this.group(start, end) and + exists(int name_end | + this.ecma_named_group_start(start, name_end) and + result = this.getText().substring(start + 3, name_end - 1) + ) + } + + // --------------------------------------------------------------------------- + // Back-references + // --------------------------------------------------------------------------- + /** + * Holds if a numbered back-reference `\1`..`\9` (or `\10` etc.) spans + * `[start, end)` with value `value`. + * + * In ECMAScript, `\0` is the NUL character (not a back-reference). + * `\1`..`\9` are always back-references. Higher numbers are back-references + * only when there are enough groups; we accept all here for simplicity. + */ + override predicate numbered_backreference(int start, int end, int value) { + this.escapingChar(start) and + // \0 is not a back-reference in ECMAScript + not this.getChar(start + 1) = "0" and + this.isDecimalDigit(start + 1) and + exists(string text, string svalue, int len | + end = start + len and + text = this.getText() and + len in [2 .. 3] + | + svalue = text.substring(start + 1, start + len) and + value = svalue.toInt() and + forall(int i | i in [start + 1 .. start + len - 1] | this.isDecimalDigit(i)) and + not ( + len = 2 and + exists(text.substring(start + 1, start + len + 1).toInt()) and + this.isDecimalDigit(start + len) + ) + ) + } + + /** + * Holds if an ECMAScript named back-reference `\k` spans `[start, end)` + * with the given `name`. + */ + override predicate named_backreference(int start, int end, string name) { + this.escapingChar(start) and + this.getChar(start + 1) = "k" and + this.getChar(start + 2) = "<" and + exists(int name_end | + name_end = min(int i | i > start + 3 and this.getChar(i) = ">") and + end = name_end + 1 and + name = this.getText().substring(start + 3, name_end) + ) + } + + override predicate backreference(int start, int end) { + this.numbered_backreference(start, end, _) + or + this.named_backreference(start, end, _) + } + + override int getBackrefNumber(int start, int end) { + this.numbered_backreference(start, end, result) + } + + override string getBackrefName(int start, int end) { + this.named_backreference(start, end, result) + } +} + +// =========================================================================== +// POSIX Extended Regular Expressions (ERE) dialect +// =========================================================================== +/** + * The POSIX Extended Regular Expressions concrete `RegExp` implementation. + * Supplies all dialect hooks with the ERE token-recognition behavior. + * + * ERE is selected via the `std::regex_constants` flags `extended`, `egrep`, + * and `awk` (all three parse the same grammar - they differ only in whether + * the matching engine is treated as backtracking; see + * `RegexFlowConfigs::isBacktrackingEngine`). + * + * ERE is largely subtractive relative to ECMAScript: + * + * - Same grouping (`(...)`), alternation (`|`), and quantifiers + * (`*`, `+`, `?`, `{n}`, `{n,}`, `{n,m}`). + * - Same character classes `[...]` (including the shared POSIX bracket + * sub-expressions handled entirely in the core: `[:class:]`, `[.a.]`, + * `[=a=]`). + * - Same anchors `^`, `$` and wildcard `.`. + * + * ERE has **no**: + * - Class-shorthand escapes `\d`, `\w`, `\s`, `\D`, `\W`, `\S`. + * - Word-boundary anchors `\b`, `\B`. + * - Numeric or hex/unicode/octal escapes (`\1`, `\xNN`, `\uNNNN`, `\0`). + * - Back-references (numbered or named). + * - Look-around (`(?=`, `(?!`, `(?<=`, `(?...)`). + * - Lazy quantifier suffix (`*?`, `+?`, `??`, `{n,m}?`). + * + * A backslash in front of any character produces a literal escaped + * character (two-character span): so `\.` matches a literal `.`, `\(` + * matches a literal `(`, and so on. This is the standard interpretation of + * `\` in ERE for metacharacters; behavior for `\` in front of an ordinary + * character is implementation-defined but is uniformly treated here as a + * literal escape of the following character so the tokenizer is total. + */ +class EreRegExp extends RegExp { + EreRegExp() { RFC::regexGrammar(this) = RFC::Ere() } + + override RFC::TRegexGrammar getGrammar() { result = RFC::Ere() } + + // --------------------------------------------------------------------------- + // Escaping + // --------------------------------------------------------------------------- + /** + * Helper predicate for `escapingChar`. + * Returns `true` if the character at position `pos` is an active backslash + * (i.e., it escapes the next character). Uses a boolean to avoid negation + * in recursive calls. + */ + private boolean escaping(int pos) { + pos = -1 and result = false + or + this.getChar(pos) = "\\" and result = this.escaping(pos - 1).booleanNot() + or + this.getChar(pos) != "\\" and result = false + } + + override predicate escapingChar(int pos) { this.escaping(pos) = true } + + // --------------------------------------------------------------------------- + // Escaped characters + // + // ERE has no numeric, hex, unicode, or octal escapes and no back-references. + // Every `\X` is a simple two-character escape yielding a literal X. + // --------------------------------------------------------------------------- + override predicate escapedCharacter(int start, int end) { + this.escapingChar(start) and + exists(this.getChar(start + 1)) and + end = start + 2 + } + + // --------------------------------------------------------------------------- + // Special (meta) characters + // + // ERE has only the position-assertion / wildcard specials `^`, `$`, `.`. + // There are no word-boundary escapes `\b` / `\B`. + // --------------------------------------------------------------------------- + override predicate specialCharacter(int start, int end, string char) { + not this.inCharSet(start) and + this.character(start, end) and + end = start + 1 and + char = this.getChar(start) and + (char = "$" or char = "^" or char = ".") + } + + // --------------------------------------------------------------------------- + // Quantifiers + // + // ERE has no lazy suffix, so `qualifier` and `short_qualifier` coincide. + // --------------------------------------------------------------------------- + override predicate qualifier(int start, int end, boolean maybe_empty, boolean may_repeat_forever) { + this.short_qualifier(start, end, maybe_empty, may_repeat_forever) + } + + override predicate short_qualifier( + int start, int end, boolean maybe_empty, boolean may_repeat_forever + ) { + ( + this.getChar(start) = "+" and maybe_empty = false and may_repeat_forever = true + or + this.getChar(start) = "*" and maybe_empty = true and may_repeat_forever = true + or + this.getChar(start) = "?" and maybe_empty = true and may_repeat_forever = false + ) and + end = start + 1 and + not this.escapingChar(start - 1) + or + exists(string lower, string upper | + this.multiples(start, end, lower, upper) and + (if lower = "" or lower.toInt() = 0 then maybe_empty = true else maybe_empty = false) and + if upper = "" then may_repeat_forever = true else may_repeat_forever = false + ) + } + + /** + * Holds if `[start, end)` is a `{n}`, `{n,m}`, or `{n,}` quantifier. + * + * In ERE, `{...}` is unconditionally a quantifier - there is no `\{...\}` + * literal-brace form (that would be BRE). A backslash-escaped `\{` is a + * literal `{`, so we must not treat it as a quantifier here. + */ + override predicate multiples(int start, int end, string lower, string upper) { + not this.escapingChar(start - 1) and + exists(string text, string match, string inner | + text = this.getText() and + end = start + match.length() and + inner = match.substring(1, match.length() - 1) + | + match = text.regexpFind("\\{[0-9]+\\}", _, start) and + lower = inner and + upper = lower + or + match = text.regexpFind("\\{[0-9]*,[0-9]*\\}", _, start) and + exists(int commaIndex | + commaIndex = inner.indexOf(",") and + lower = inner.prefix(commaIndex) and + upper = inner.suffix(commaIndex + 1) + ) + ) + } + + // --------------------------------------------------------------------------- + // Groups + // + // ERE has only simple capturing groups `(...)`. It has no `(?:...)`, + // no `(?...)`, and no look-around forms. + // --------------------------------------------------------------------------- + override predicate isOptionDivider(int i) { this.nonEscapedCharAt(i) = "|" } + + override predicate isGroupEnd(int i) { this.nonEscapedCharAt(i) = ")" and not this.inCharSet(i) } + + override predicate group_end(int start, int end) { this.isGroupEnd(start) and end = start + 1 } + + override predicate isGroupStart(int i) { + this.nonEscapedCharAt(i) = "(" and not this.inCharSet(i) + } + + override predicate group_start(int start, int end) { this.simple_group_start(start, end) } + + /** `(...)` - simple capturing group. */ + override predicate simple_group_start(int start, int end) { + this.isGroupStart(start) and end = start + 1 + } + + /** ERE has no non-capturing group form. */ + override predicate non_capturing_group_start(int start, int end) { none() } + + /** ERE has no named capturing group form. */ + override predicate ecma_named_group_start(int start, int end) { none() } + + /** ERE has no look-around. */ + override predicate lookahead_assertion_start(int start, int end) { none() } + + /** ERE has no look-around. */ + override predicate negative_lookahead_assertion_start(int start, int end) { none() } + + /** ERE has no look-around. */ + override predicate lookbehind_assertion_start(int start, int end) { none() } + + /** ERE has no look-around. */ + override predicate negative_lookbehind_assertion_start(int start, int end) { none() } + + /** + * Gets the 1-based index of the capture group at `[start, end)`. In ERE + * every `(...)` group is a capturing group, numbered by left-to-right + * position of the opening `(`. + */ + override int getGroupNumber(int start, int end) { + this.group(start, end) and + result = count(int i | this.group(i, _) and i < start) + 1 + } + + /** ERE has no named groups. */ + override string getGroupName(int start, int end) { none() } + + // --------------------------------------------------------------------------- + // Back-references + // + // ERE has no back-references (neither numbered nor named). + // --------------------------------------------------------------------------- + override predicate numbered_backreference(int start, int end, int value) { none() } + + override predicate named_backreference(int start, int end, string name) { none() } + + override predicate backreference(int start, int end) { none() } + + override int getBackrefNumber(int start, int end) { none() } + + override string getBackrefName(int start, int end) { none() } +} + +// =========================================================================== +// POSIX Basic Regular Expressions (BRE) dialect +// =========================================================================== +/** + * The POSIX Basic Regular Expressions concrete `RegExp` implementation. + * Supplies all dialect hooks with the BRE token-recognition behavior. + * + * BRE is selected via the `std::regex_constants` flags `basic` and `grep`. + * (`basic` is backtracking-eligible; `grep` is parsed but excluded from + * ReDoS analysis by `isBacktrackingEngine`.) + * + * BRE inverts the ERE / ECMAScript escaping convention for a handful of + * metacharacters: constructs that are special in ERE become literals in + * BRE unless prefixed with a backslash. Concretely, this dialect models: + * + * - **Groups**: `\(...\)` (backslash-prefixed) are capturing groups; bare + * `(` and `)` are literal characters. No non-capturing, named, or + * look-around forms exist. + * - **Interval quantifier**: `\{n\}`, `\{n,\}`, `\{n,m\}` are the range + * quantifiers; bare `{` and `}` are literals. + * - **Quantifier `*`**: unlimited-repetition quantifier - except when it + * is the first character of the regex, the first character after a + * leading `^`, or the first character of a subexpression (i.e., right + * after a `\(` group-open), in which cases it is a literal `*`. + * - **`+` and `?`**: literal characters in POSIX BRE (this implementation + * does not model the GNU-BRE extensions `\+` and `\?`; they are treated + * uniformly as escaped literals, matching a literal `+` or `?`). + * - **Alternation**: POSIX BRE has no alternation operator. Bare `|` is a + * literal `|`; `\|` is likewise a literal `|` (the GNU-BRE `\|` + * extension is not modeled). + * - **Anchors**: `^` is an anchor only at the start of the regex or of a + * subexpression (immediately after `\(`); `$` is an anchor only at the + * end of the regex or of a subexpression (immediately before `\)`). + * Otherwise both are literal characters. `.` is always the wildcard. + * There are no `\b` / `\B` word-boundary escapes. + * - **Back-references**: `\1` through `\9` are numbered back-references. + * There are no named back-references. + * - **Escaped characters**: `\` before an ordinary metacharacter renders + * that metacharacter literal (e.g. `\.` -> literal `.`, `\*` -> literal + * `*`, `\\` -> literal `\`). Backslash escapes that are consumed by the + * group / interval / back-reference hooks above (`\(`, `\)`, `\{`, `\}`, + * `\1..\9`) are structural and are *not* treated as escaped literals. + * - **POSIX bracket expressions**: identical to ERE / ECMAScript, + * including the shared POSIX-class / equivalence-class / collation + * layer (`[[:alpha:]]`, `[[=a=]]`, `[[.ch.]]`). + */ +class BreRegExp extends RegExp { + BreRegExp() { RFC::regexGrammar(this) = RFC::Bre() } + + override RFC::TRegexGrammar getGrammar() { result = RFC::Bre() } + + // --------------------------------------------------------------------------- + // Escaping + // --------------------------------------------------------------------------- + /** + * Helper predicate for `escapingChar`. Boolean-valued to avoid negation + * in recursive calls; mirrors the ERE / ECMAScript backslash-parity rule. + */ + private boolean escaping(int pos) { + pos = -1 and result = false + or + this.getChar(pos) = "\\" and result = this.escaping(pos - 1).booleanNot() + or + this.getChar(pos) != "\\" and result = false + } + + override predicate escapingChar(int pos) { this.escaping(pos) = true } + + // --------------------------------------------------------------------------- + // Escaped characters + // + // In BRE, backslash is used both structurally (to make `(`, `)`, `{`, `}` + // and digits special) and to make ordinary metacharacters literal + // (`\.`, `\*`, `\\`, `\|`, `\+`, `\?`, ...). The `escapedCharacter` hook + // is only for the *literal-escape* case: it must exclude the structural + // sequences, otherwise `\(` would be misread as a literal `(` and the + // group would vanish. + // --------------------------------------------------------------------------- + override predicate escapedCharacter(int start, int end) { + this.escapingChar(start) and + exists(string next | next = this.getChar(start + 1) | + not next = "(" and + not next = ")" and + not next = "{" and + not next = "}" and + not this.isDecimalDigit(start + 1) + ) and + end = start + 2 + } + + // --------------------------------------------------------------------------- + // Special (meta) characters + // + // BRE has only the position-assertion / wildcard specials `^`, `$`, `.`. + // The anchors `^` and `$` are positional: `^` is an anchor only at the + // start of a subexpression, `$` only at the end. Elsewhere they are + // literal characters. + // --------------------------------------------------------------------------- + /** + * Holds if position `start` is the start of a subexpression: either the + * start of the whole regex, or immediately after a `\(` group-open + * (i.e., after the `(` of a `\(` pair, which is the content-start + * reported by `simple_group_start`). + */ + private predicate atSubexpressionStart(int start) { + start = 0 + or + this.group_start(_, start) + } + + /** + * Holds if position `start` is at the end of a subexpression: either the + * position of the trailing character in the whole regex, or immediately + * before a `\)` group-close (i.e., the position of the `\` of a `\)` + * pair - which is the position reported by `isGroupEnd` and by + * `group_end` as its start). + */ + private predicate atSubexpressionEnd(int start) { + start = this.getText().length() - 1 + or + this.group_end(start + 1, _) + } + + override predicate specialCharacter(int start, int end, string char) { + not this.inCharSet(start) and + this.character(start, end) and + end = start + 1 and + char = this.getChar(start) and + ( + char = "." + or + char = "^" and this.atSubexpressionStart(start) + or + char = "$" and this.atSubexpressionEnd(start) + ) + } + + // --------------------------------------------------------------------------- + // Quantifiers + // + // BRE has `*` (positional literal at start-of-subexpression) and the + // interval `\{n\}` / `\{n,\}` / `\{n,m\}` form. `+` and `?` are literal + // characters. There is no lazy suffix. + // --------------------------------------------------------------------------- + override predicate qualifier(int start, int end, boolean maybe_empty, boolean may_repeat_forever) { + this.short_qualifier(start, end, maybe_empty, may_repeat_forever) + } + + override predicate short_qualifier( + int start, int end, boolean maybe_empty, boolean may_repeat_forever + ) { + this.getChar(start) = "*" and + end = start + 1 and + maybe_empty = true and + may_repeat_forever = true and + not this.escapingChar(start - 1) and + // `*` is a literal at the start of a subexpression (start of regex, + // start of `^`-anchored regex, or immediately after `\(`). + not this.atSubexpressionStart(start) and + not (start = 1 and this.getChar(0) = "^") + or + exists(string lower, string upper | + this.multiples(start, end, lower, upper) and + (if lower = "" or lower.toInt() = 0 then maybe_empty = true else maybe_empty = false) and + if upper = "" then may_repeat_forever = true else may_repeat_forever = false + ) + } + + /** + * Holds if `[start, end)` is a `\{n\}`, `\{n,\}`, or `\{n,m\}` interval + * quantifier. `start` is the position of the leading backslash. + * + * In BRE the braces themselves must be backslash-prefixed. Bare `{...}` + * is literal text. + */ + override predicate multiples(int start, int end, string lower, string upper) { + this.escapingChar(start) and + exists(string text, string match, string inner | + text = this.getText() and + end = start + match.length() and + // Strip the leading `\{` and trailing `\}` (2 chars each side). + inner = match.substring(2, match.length() - 2) + | + match = text.regexpFind("\\\\\\{[0-9]+\\\\\\}", _, start) and + lower = inner and + upper = lower + or + match = text.regexpFind("\\\\\\{[0-9]*,[0-9]*\\\\\\}", _, start) and + exists(int commaIndex | + commaIndex = inner.indexOf(",") and + lower = inner.prefix(commaIndex) and + upper = inner.suffix(commaIndex + 1) + ) + ) + } + + // --------------------------------------------------------------------------- + // Groups + // + // BRE has only `\(...\)` capturing groups. There are no non-capturing, + // named, or look-around forms. No alternation operator either. + // --------------------------------------------------------------------------- + /** POSIX BRE has no alternation operator; bare `|` is a literal. */ + override predicate isOptionDivider(int i) { none() } + + /** + * Holds at the position of the `(` in a `\(` group-open. Note this is + * *not* the position of the leading backslash - the whole delimiter + * span is reported by `simple_group_start` / `group_start` starting at + * the backslash. + */ + override predicate isGroupStart(int i) { + this.getChar(i) = "(" and + this.escapingChar(i - 1) and + not this.inCharSet(i) + } + + /** + * Holds at the position of the leading `\` in a `\)` group-close. + * The whole two-character delimiter span is reported by `group_end`. + */ + override predicate isGroupEnd(int i) { + this.getChar(i) = "\\" and + this.escapingChar(i) and + this.getChar(i + 1) = ")" and + not this.inCharSet(i) + } + + override predicate group_end(int start, int end) { this.isGroupEnd(start) and end = start + 2 } + + override predicate group_start(int start, int end) { this.simple_group_start(start, end) } + + /** `\(...\)` - simple capturing group. `start` is the leading backslash. */ + override predicate simple_group_start(int start, int end) { + this.isGroupStart(start + 1) and end = start + 2 + } + + /** BRE has no non-capturing group form. */ + override predicate non_capturing_group_start(int start, int end) { none() } + + /** BRE has no named capturing group form. */ + override predicate ecma_named_group_start(int start, int end) { none() } + + /** BRE has no look-around. */ + override predicate lookahead_assertion_start(int start, int end) { none() } + + /** BRE has no look-around. */ + override predicate negative_lookahead_assertion_start(int start, int end) { none() } + + /** BRE has no look-around. */ + override predicate lookbehind_assertion_start(int start, int end) { none() } + + /** BRE has no look-around. */ + override predicate negative_lookbehind_assertion_start(int start, int end) { none() } + + /** + * Gets the 1-based index of the capture group at `[start, end)`. In BRE + * every `\(...\)` group is a capturing group, numbered by left-to-right + * position of the opening `\(`. + */ + override int getGroupNumber(int start, int end) { + this.group(start, end) and + result = count(int i | this.group(i, _) and i < start) + 1 + } + + /** BRE has no named groups. */ + override string getGroupName(int start, int end) { none() } + + // --------------------------------------------------------------------------- + // Back-references + // + // BRE supports numbered back-references `\1` .. `\9` (single digit only). + // There are no named back-references. + // --------------------------------------------------------------------------- + override predicate numbered_backreference(int start, int end, int value) { + this.escapingChar(start) and + this.isDecimalDigit(start + 1) and + not this.getChar(start + 1) = "0" and + end = start + 2 and + value = this.getChar(start + 1).toInt() + } + + override predicate named_backreference(int start, int end, string name) { none() } + + override predicate backreference(int start, int end) { + this.numbered_backreference(start, end, _) + } + + override int getBackrefNumber(int start, int end) { + this.numbered_backreference(start, end, result) + } + + override string getBackrefName(int start, int end) { none() } +} diff --git a/cpp/ql/lib/semmle/code/cpp/security/regexp/ExponentialReDoSQuery.qll b/cpp/ql/lib/semmle/code/cpp/security/regexp/ExponentialReDoSQuery.qll new file mode 100644 index 000000000000..f7934c46501c --- /dev/null +++ b/cpp/ql/lib/semmle/code/cpp/security/regexp/ExponentialReDoSQuery.qll @@ -0,0 +1,42 @@ +/** + * Provides classes and predicates for reasoning about exponential-time + * regular-expression denial-of-service (ReDoS) vulnerabilities in C++. + * + * The library mirrors the JavaScript `js/redos` query: it plugs the C++ + * regex parse-tree view (`semmle.code.cpp.regex.RegexTreeView`) into the + * shared `ExponentialBackTracking` analysis, and exposes the + * `hasReDoSResult` predicate that identifies regex terms whose worst-case + * matching is exponential in the input length. + * + * Unlike the polynomial ReDoS query, no data-flow path from a + * user-controlled source is required: the vulnerable regex term itself is + * the alert. The parse-tree view already restricts to string literals used + * as `std::regex` patterns (see Phase 1's `RegExp` class in + * `semmle.code.cpp.regex.internal.ParseRegExp`), so only regexes that are + * actually used with `std::regex` are considered. + */ + +import semmle.code.cpp.regex.RegexTreeView +private import semmle.code.cpp.regex.RegexTreeView::RegexTreeView as TreeView +private import codeql.regex.nfa.ExponentialBackTracking +private import semmle.code.cpp.regex.RegexFlowConfigs + +private module Impl = Make; + +/** A state of the NFA constructed for a regular expression. */ +class State = Impl::State; + +/** + * Holds if `t` is a regex term whose worst-case matching is exponential in + * the input length, with `pump` being an example pumping string that + * triggers the backtracking from state `s`, and `prefixMsg` describing any + * required prefix. + * + * The result is restricted to terms whose root regex satisfies + * `isBacktrackingEngine`, so that regexes constructed with a non-backtracking + * `std::regex` grammar flag (`awk`, `grep`, or `egrep`) are excluded. + */ +predicate hasReDoSResult(RegExpTerm t, string pump, State s, string prefixMsg) { + Impl::hasReDoSResult(t, pump, s, prefixMsg) and + isBacktrackingEngine(t.getRootTerm().getLiteral().getRegex()) +} diff --git a/cpp/ql/lib/semmle/code/cpp/security/regexp/PolynomialReDoSQuery.qll b/cpp/ql/lib/semmle/code/cpp/security/regexp/PolynomialReDoSQuery.qll new file mode 100644 index 000000000000..62a9fe09d2c9 --- /dev/null +++ b/cpp/ql/lib/semmle/code/cpp/security/regexp/PolynomialReDoSQuery.qll @@ -0,0 +1,110 @@ +/** + * Provides classes and predicates for reasoning about polynomial-time + * regular-expression denial-of-service (ReDoS) vulnerabilities in C++. + * + * The library mirrors the Java `PolynomialReDoSQuery` library: it plugs the + * C++ regex parse-tree view (`semmle.code.cpp.regex.RegexTreeView`) into the + * shared `SuperlinearBackTracking` analysis, and defines a data-flow + * configuration that tracks user-controlled data to a subject expression + * that is matched against a `std::regex` whose parse tree contains a + * polynomial-backtracking term. + */ + +private import cpp +private import semmle.code.cpp.regex.RegexTreeView::RegexTreeView as TreeView +import codeql.regex.nfa.SuperlinearBackTracking::Make as SuperlinearBackTracking +private import semmle.code.cpp.ir.dataflow.DataFlow +private import semmle.code.cpp.ir.dataflow.TaintTracking +private import semmle.code.cpp.regex.RegexFlowConfigs +private import semmle.code.cpp.security.FlowSources + +/** + * A sink for the polynomial ReDoS query: a subject expression that is + * matched against a `std::regex` whose pattern is a `TreeView::RegExpLiteral`. + */ +class PolynomialRedosSink extends DataFlow::Node { + TreeView::RegExpLiteral reg; + + PolynomialRedosSink() { + exists(Expr e | + regexMatchedAgainst(reg.getRegex(), e) and + isBacktrackingEngine(reg.getRegex()) and + (this.asExpr() = e or this.asIndirectExpr() = e) + ) + } + + /** Gets a regex term (a child of the matched literal) associated with this sink. */ + TreeView::RegExpTerm getRegExp() { result.getParent() = reg } +} + +/** + * A function whose result typically has a limited length, such as HTTP + * headers, cookies, request URIs, or their C++ analogues. Values derived + * from calls to such functions are treated as length-restricted and act as + * barriers for the polynomial ReDoS analysis. + * + * This is a conservative, name-based heuristic: it matches functions whose + * unqualified name (or declaring class name for member functions) suggests + * that the returned string is bounded in length in practice. + */ +private class LengthRestrictedFunction extends Function { + LengthRestrictedFunction() { + exists(string n | n = this.getName().toLowerCase() | + n.matches(["%header%", "%cookie%", "%requesturi%", "%requesturl%", "%useragent%"]) + ) + or + exists(MemberFunction mf, string cls, string n | + mf = this and + cls = mf.getDeclaringType().getName().toLowerCase() and + n = mf.getName().toLowerCase() + | + cls.matches("%cookie%") and n.matches("get%") + or + cls.matches("%request%") and n.matches(["%get%path%", "get%user%", "%querystring%"]) + ) + } +} + +/** + * Holds if `node` is a value whose static type has a small, fixed size, so + * that it is treated as length-restricted for the polynomial ReDoS + * analysis. This includes integral and floating-point values, which cannot + * usefully be matched against a regex. + */ +private predicate isSmallFixedSizeType(DataFlow::Node node) { + node.asExpr().getUnspecifiedType() instanceof IntegralType + or + node.asExpr().getUnspecifiedType() instanceof FloatingPointType +} + +/** A configuration for the polynomial ReDoS query. */ +module PolynomialRedosConfig implements DataFlow::ConfigSig { + predicate isSource(DataFlow::Node source) { source instanceof FlowSource } + + predicate isSink(DataFlow::Node sink) { + exists(SuperlinearBackTracking::PolynomialBackTrackingTerm regexp | + regexp.getRootTerm() = sink.(PolynomialRedosSink).getRegExp() + ) + } + + predicate isBarrier(DataFlow::Node node) { + isSmallFixedSizeType(node) + or + node.asExpr().(Call).getTarget() instanceof LengthRestrictedFunction + } + + predicate observeDiffInformedIncrementalMode() { any() } + + Location getASelectedSinkLocation(DataFlow::Node sink) { + exists(SuperlinearBackTracking::PolynomialBackTrackingTerm regexp | + regexp.getRootTerm() = sink.(PolynomialRedosSink).getRegExp() + | + result = sink.getLocation() + or + result = regexp.getLocation() + ) + } +} + +/** Taint-tracking flow from user input to a polynomial-backtracking regex match. */ +module PolynomialRedosFlow = TaintTracking::Global; diff --git a/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.qhelp b/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.qhelp new file mode 100644 index 000000000000..2449a417b253 --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.qhelp @@ -0,0 +1,117 @@ + + + + + + +

+ Consider this use of a regular expression, which removes all leading + and trailing whitespace in a string: +

+ + +std::string trim(const std::string &text) { + static const std::regex re("^\\s+|\\s+$"); + return std::regex_replace(text, re, ""); // BAD +} + + +

+ The sub-expression "\\s+$" will match the whitespace + characters in text from left to right, but it can start + matching anywhere within a whitespace sequence. This is problematic + for strings that do not end with a whitespace + character. Such a string will force the regular expression engine to + process each whitespace sequence once per whitespace character in the + sequence. +

+ +

+ This ultimately means that the time cost of trimming a string is + quadratic in the length of the string. So a string like "a b" + will take milliseconds to process, but a similar string with a + million spaces instead of just one will take several minutes. +

+ +

+ Avoid this problem by rewriting the regular expression to not contain + the ambiguity about when to start matching whitespace sequences. For + instance, by using a negative look-behind + ("^\\s+|(?<!\\s)\\s+$"), or by trimming the string + using non-regex means. +

+ +

+ Note that the sub-expression "^\\s+" is + not problematic as the ^ anchor + restricts when that sub-expression can start matching, and as the + regular expression engine matches from left to right. +

+ + + +

+ As a similar, but slightly subtler problem, consider the regular + expression that matches lines with numbers, possibly written using + scientific notation: +

+ + +static const std::regex numRe("^0\\.\\d+E?\\d+$"); +if (std::regex_search(input, numRe)) { /* ... */ } // BAD + + +

+ The problem with this regular expression is in the sub-expression + \d+E?\d+ because the second \d+ can start + matching digits anywhere after the first match of the first + \d+ if there is no E in the input string. +

+ +

+ This is problematic for strings that do not end with + a digit. Such a string will force the regular expression engine to + process each digit sequence once per digit in the sequence, again + leading to a quadratic time complexity. +

+ +

+ To make the processing faster, the regular expression should be + rewritten such that the two \d+ sub-expressions do not + have overlapping matches: "^0\\.\\d+(E\\d+)?$". +

+ + + +

+ Sometimes it is unclear how a regular expression can be rewritten to + avoid the problem. In such cases, it often suffices to limit the + length of the input string. For instance, the following regular + expression is used to match numbers, and on some non-number inputs it + can have quadratic time complexity: +

+ + +static const std::regex numRe( + "^(\\+|-)?(\\d+|(\\d*\\.\\d*))?(E|e)?([-+])?(\\d+)?$"); +if (std::regex_match(input, numRe)) { /* ... */ } + + +

+ It is not immediately obvious how to rewrite this regular expression + to avoid the problem. However, you can mitigate performance issues by + limiting the length of the input to a small constant, which will + always finish in a reasonable amount of time: +

+ + +if (input.size() > 1000) { + throw std::invalid_argument("Input too long"); +} +if (std::regex_match(input, numRe)) { /* ... */ } + + + + + + diff --git a/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql b/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql new file mode 100644 index 000000000000..b6dc720c8f86 --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/PolynomialReDoS.ql @@ -0,0 +1,29 @@ +/** + * @name Polynomial regular expression used on uncontrolled data + * @description A regular expression that can require polynomial time + * to match may be vulnerable to denial-of-service attacks. + * @kind path-problem + * @problem.severity warning + * @security-severity 7.5 + * @precision high + * @id cpp/polynomial-redos + * @tags security + * external/cwe/cwe-1333 + * external/cwe/cwe-730 + * external/cwe/cwe-400 + */ + +import cpp +import semmle.code.cpp.security.regexp.PolynomialReDoSQuery +import PolynomialRedosFlow::PathGraph + +from + PolynomialRedosFlow::PathNode source, PolynomialRedosFlow::PathNode sink, + SuperlinearBackTracking::PolynomialBackTrackingTerm regexp +where + PolynomialRedosFlow::flowPath(source, sink) and + regexp.getRootTerm() = sink.getNode().(PolynomialRedosSink).getRegExp() +select sink, source, sink, + "This $@ that depends on a $@ may run slow on strings " + regexp.getPrefixMessage() + + "with many repetitions of '" + regexp.getPumpString() + "'.", regexp, "regular expression", + source.getNode(), "user-provided value" diff --git a/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.qhelp b/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.qhelp new file mode 100644 index 000000000000..014e604ec320 --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.qhelp @@ -0,0 +1,64 @@ + + + + + + +

+ Consider this regular expression: +

+ + +static const std::regex re("^(a+)+$"); +if (std::regex_match(input, re)) { /* ... */ } // BAD + + +

+ Its sub-expression "(a+)+" can match the string + "aa" either by a single iteration of the outer group + matching two as in the inner a+, or by two + iterations of the outer group each matching a single a. + Thus, an input consisting of many as followed by a + non-matching character will cause the regular expression engine to run + for an exponential amount of time before rejecting the input. +

+ +

+ This problem can be avoided by rewriting the regular expression so + that the repetition of the outer group cannot overlap with the + repetition of the inner group. For example, by simply using a single + repetition: +

+ + +static const std::regex re("^a+$"); +if (std::regex_match(input, re)) { /* ... */ } + + + + +

+ As a slightly subtler example, consider the regular expression + "^(a|a)*$". The two alternatives inside the group both + match the same character, so the regular expression engine can match + each character in the input in two different ways. The number of ways + of matching an input consisting of n as + followed by a non-matching character grows as 2n, + leading to exponential worst-case behavior: +

+ + +static const std::regex re("^(a|a)*$"); +if (std::regex_match(input, re)) { /* ... */ } // BAD + + +

+ This can be fixed by removing the ambiguity between the two branches + of the alternative, for instance by rewriting the pattern as + "^a*$". +

+ + + + + diff --git a/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql b/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql new file mode 100644 index 000000000000..80ba68734359 --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/ReDoS.ql @@ -0,0 +1,24 @@ +/** + * @name Inefficient regular expression + * @description A regular expression that requires exponential time to match certain inputs + * can be a performance bottleneck, and may be vulnerable to denial-of-service + * attacks. + * @kind problem + * @problem.severity error + * @security-severity 7.5 + * @precision high + * @id cpp/redos + * @tags security + * external/cwe/cwe-1333 + * external/cwe/cwe-730 + * external/cwe/cwe-400 + */ + +import cpp +import semmle.code.cpp.security.regexp.ExponentialReDoSQuery + +from RegExpTerm t, string pump, State s, string prefixMsg +where hasReDoSResult(t, pump, s, prefixMsg) +select t, + "This part of the regular expression may cause exponential backtracking on strings " + prefixMsg + + "containing many repetitions of '" + pump + "'." diff --git a/cpp/ql/src/Security/CWE/CWE-1333/ReDoSIntroduction.inc.qhelp b/cpp/ql/src/Security/CWE/CWE-1333/ReDoSIntroduction.inc.qhelp new file mode 100644 index 000000000000..f9b645a9070d --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/ReDoSIntroduction.inc.qhelp @@ -0,0 +1,39 @@ + + + +

+ Some regular expressions take a long time to match certain input strings + to the point where the time it takes to match a string of length n + is proportional to nk or even 2n. + Such regular expressions can negatively affect performance, and potentially allow + a malicious user to perform a Denial of Service ("DoS") attack by crafting + an expensive input string for the regular expression to match. +

+

+ The C++ standard library regular expression engine (std::regex, + std::regex_match, std::regex_search, + std::regex_replace) uses a backtracking non-deterministic + finite automaton to implement regular expression matching. While this + approach is space-efficient and allows supporting advanced features like + capture groups and back-references, it is not time-efficient in general. + The worst-case time complexity of such an automaton can be polynomial or + even exponential, meaning that for strings of a certain shape, + increasing the input length by ten characters may make the automaton + about 1000 times slower. +

+

+ Typically, a regular expression is affected by this problem if it + contains a repetition of the form r* or r+ + where the sub-expression r is ambiguous in the sense that + it can match some string in multiple ways. More information about the + precise circumstances can be found in the references. +

+ + +

+ Modify the regular expression to remove the ambiguity, or ensure that + the strings matched with the regular expression are short enough that + the time complexity does not matter. +

+ + diff --git a/cpp/ql/src/Security/CWE/CWE-1333/ReDoSReferences.inc.qhelp b/cpp/ql/src/Security/CWE/CWE-1333/ReDoSReferences.inc.qhelp new file mode 100644 index 000000000000..9fea5c47e365 --- /dev/null +++ b/cpp/ql/src/Security/CWE/CWE-1333/ReDoSReferences.inc.qhelp @@ -0,0 +1,14 @@ + + + +
  • + OWASP: + Regular expression Denial of Service - ReDoS. +
    • +
  • Wikipedia: ReDoS.
    • +
  • Wikipedia: Time complexity.
    • +
  • James Kirrage, Asiri Rathnayake, Hayo Thielecke: + Static Analysis for Regular Expression Denial-of-Service Attack. +
    • +     +     diff --git a/cpp/ql/src/change-notes/2026-07-16-polynomial-redos.md b/cpp/ql/src/change-notes/2026-07-16-polynomial-redos.md new file mode 100644 index 000000000000..bc29950d02a0 --- /dev/null +++ b/cpp/ql/src/change-notes/2026-07-16-polynomial-redos.md @@ -0,0 +1,4 @@ +--- +category: newQuery +--- +* Added a new query, `cpp/polynomial-redos`, for detecting polynomial-time regular expression denial-of-service (ReDoS) vulnerabilities where user-controlled data is matched against a `std::regex` whose pattern can exhibit super-linear worst-case matching behavior. This is the C++ analogue of `java/polynomial-redos` and is based on the shared `SuperlinearBackTracking` analysis wired to the C++ regex parse tree (`semmle.code.cpp.regex.RegexTreeView`) and the C++ regex flow modeling (`semmle.code.cpp.regex.RegexFlowConfigs`). diff --git a/cpp/ql/src/change-notes/2026-07-16-redos.md b/cpp/ql/src/change-notes/2026-07-16-redos.md new file mode 100644 index 000000000000..3e87de1adc69 --- /dev/null +++ b/cpp/ql/src/change-notes/2026-07-16-redos.md @@ -0,0 +1,4 @@ +--- +category: newQuery +--- +* Added a new query, `cpp/redos`, for detecting regular expressions that can exhibit exponential worst-case matching behavior when used with `std::regex`. Unlike `cpp/polynomial-redos`, this query does not require a dataflow path from a user-controlled source; the vulnerable regex term itself is the alert, mirroring the design of `js/redos`. The query is based on the shared `ExponentialBackTracking` analysis wired to the C++ regex parse tree (`semmle.code.cpp.regex.RegexTreeView`). diff --git a/cpp/ql/test/library-tests/regex/Consistency.expected b/cpp/ql/test/library-tests/regex/Consistency.expected new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/cpp/ql/test/library-tests/regex/Consistency.ql b/cpp/ql/test/library-tests/regex/Consistency.ql new file mode 100644 index 000000000000..1380871c3134 --- /dev/null +++ b/cpp/ql/test/library-tests/regex/Consistency.ql @@ -0,0 +1,13 @@ +/** + * Flags positions in regular expressions that failed to parse. + * An empty expected output confirms that all regex constructs in the test + * file were parsed successfully. + */ + +import cpp +import semmle.code.cpp.regex.RegexTreeView::RegexTreeView as RTV +import semmle.code.cpp.regex.internal.ParseRegExp + +from RegExp re, int i +where re.failedToParse(i) +select re, i, "Failed to parse character at offset " + i + " in " + re.getValue() diff --git a/cpp/ql/test/library-tests/regex/RegexFlow.expected b/cpp/ql/test/library-tests/regex/RegexFlow.expected new file mode 100644 index 000000000000..212ebdd77a9e --- /dev/null +++ b/cpp/ql/test/library-tests/regex/RegexFlow.expected @@ -0,0 +1,129 @@ +usedAsRegex_ +| test.cpp:133:19:133:27 | a*b+c?d | +| test.cpp:134:19:134:26 | a{4,8} | +| test.cpp:135:19:135:25 | a{3,} | +| test.cpp:140:19:140:28 | foo\|bar+ | +| test.cpp:144:19:144:26 | [abc]+ | +| test.cpp:145:19:145:33 | [a-fA-F0-9_]+ | +| test.cpp:146:19:146:26 | [\\w]+ | +| test.cpp:147:19:147:27 | [^A-Z]* | +| test.cpp:151:19:151:22 | .* | +| test.cpp:152:19:152:27 | \\w+\\W | +| test.cpp:153:19:153:27 | \\s+\\S | +| test.cpp:154:19:154:27 | \\d+\\D | +| test.cpp:158:19:158:26 | ^a+bc$ | +| test.cpp:159:19:159:30 | \\ba+bc\\B | +| test.cpp:163:19:163:29 | (foo)*bar | +| test.cpp:164:19:164:30 | fo(o\|b+)ar | +| test.cpp:165:19:165:30 | (a\|b\|cd)e+ | +| test.cpp:166:19:166:29 | (?::+)\\w | +| test.cpp:170:19:170:31 | (?\\w+) | +| test.cpp:171:19:171:53 | (?[a-z]+)(?[0-9]+) | +| test.cpp:175:19:175:29 | (a+)b+\\1 | +| test.cpp:176:19:176:43 | (?q+)\\s+\\k+ | +| test.cpp:180:19:180:31 | (?=\\w+)abc | +| test.cpp:181:19:181:31 | (?!\\d+)abc | +| test.cpp:182:19:182:32 | a+bc(?<=\\w) | +| test.cpp:183:19:183:32 | a+bc(? [RegExpConstant, RegExpNormalChar] a + +# 133| [RegExpSequence] a*b+c?d +#-----| 0 -> [RegExpStar] a* +#-----| 1 -> [RegExpPlus] b+ +#-----| 2 -> [RegExpOpt] c? +#-----| 3 -> [RegExpConstant, RegExpNormalChar] d + +# 133| [RegExpConstant, RegExpNormalChar] b + +# 133| [RegExpPlus] b+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] b + +# 133| [RegExpConstant, RegExpNormalChar] c + +# 133| [RegExpOpt] c? +#-----| 0 -> [RegExpConstant, RegExpNormalChar] c + +# 133| [RegExpConstant, RegExpNormalChar] d + +# 134| [RegExpConstant, RegExpNormalChar] a + +# 134| [RegExpRange] a{4,8} +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 134| [RegExpNormalChar] 4,8} + +# 135| [RegExpConstant, RegExpNormalChar] a + +# 135| [RegExpRange] a{3,} +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 135| [RegExpNormalChar] 3,} + +# 140| [RegExpConstant, RegExpNormalChar] foo + +# 140| [RegExpAlt] foo|bar+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] foo +#-----| 1 -> [RegExpSequence] bar+ + +# 140| [RegExpConstant, RegExpNormalChar] ba + +# 140| [RegExpSequence] bar+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ba +#-----| 1 -> [RegExpPlus] r+ + +# 140| [RegExpConstant, RegExpNormalChar] r + +# 140| [RegExpPlus] r+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] r + +# 144| [RegExpCharacterClass] [abc] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] b +#-----| 2 -> [RegExpConstant, RegExpNormalChar] c + +# 144| [RegExpPlus] [abc]+ +#-----| 0 -> [RegExpCharacterClass] [abc] + +# 144| [RegExpConstant, RegExpNormalChar] a + +# 144| [RegExpConstant, RegExpNormalChar] b + +# 144| [RegExpConstant, RegExpNormalChar] c + +# 145| [RegExpCharacterClass] [a-fA-F0-9_] +#-----| 0 -> [RegExpCharacterRange] a-f +#-----| 1 -> [RegExpCharacterRange] A-F +#-----| 2 -> [RegExpCharacterRange] 0-9 +#-----| 3 -> [RegExpConstant, RegExpNormalChar] _ + +# 145| [RegExpPlus] [a-fA-F0-9_]+ +#-----| 0 -> [RegExpCharacterClass] [a-fA-F0-9_] + +# 145| [RegExpConstant, RegExpNormalChar] a + +# 145| [RegExpCharacterRange] a-f +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] f + +# 145| [RegExpConstant, RegExpNormalChar] f + +# 145| [RegExpConstant, RegExpNormalChar] A + +# 145| [RegExpCharacterRange] A-F +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] F + +# 145| [RegExpConstant, RegExpNormalChar] F + +# 145| [RegExpConstant, RegExpNormalChar] 0 + +# 145| [RegExpCharacterRange] 0-9 +#-----| 0 -> [RegExpConstant, RegExpNormalChar] 0 +#-----| 1 -> [RegExpConstant, RegExpNormalChar] 9 + +# 145| [RegExpConstant, RegExpNormalChar] 9 + +# 145| [RegExpConstant, RegExpNormalChar] _ + +# 146| [RegExpCharacterClass] [\w] +#-----| 0 -> [RegExpCharacterClassEscape] \w + +# 146| [RegExpPlus] [\w]+ +#-----| 0 -> [RegExpCharacterClass] [\w] + +# 146| [RegExpCharacterClassEscape] \w + +# 147| [RegExpCharacterClass] [^A-Z] +#-----| 0 -> [RegExpCharacterRange] A-Z + +# 147| [RegExpStar] [^A-Z]* +#-----| 0 -> [RegExpCharacterClass] [^A-Z] + +# 147| [RegExpConstant, RegExpNormalChar] A + +# 147| [RegExpCharacterRange] A-Z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] Z + +# 147| [RegExpConstant, RegExpNormalChar] Z + +# 151| [RegExpDot] . + +# 151| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 152| [RegExpCharacterClassEscape] \w + +# 152| [RegExpPlus] \w+ +#-----| 0 -> [RegExpCharacterClassEscape] \w + +# 152| [RegExpSequence] \w+\W +#-----| 0 -> [RegExpPlus] \w+ +#-----| 1 -> [RegExpCharacterClassEscape] \W + +# 152| [RegExpCharacterClassEscape] \W + +# 153| [RegExpCharacterClassEscape] \s + +# 153| [RegExpPlus] \s+ +#-----| 0 -> [RegExpCharacterClassEscape] \s + +# 153| [RegExpSequence] \s+\S +#-----| 0 -> [RegExpPlus] \s+ +#-----| 1 -> [RegExpCharacterClassEscape] \S + +# 153| [RegExpCharacterClassEscape] \S + +# 154| [RegExpCharacterClassEscape] \d + +# 154| [RegExpPlus] \d+ +#-----| 0 -> [RegExpCharacterClassEscape] \d + +# 154| [RegExpSequence] \d+\D +#-----| 0 -> [RegExpPlus] \d+ +#-----| 1 -> [RegExpCharacterClassEscape] \D + +# 154| [RegExpCharacterClassEscape] \D + +# 158| [RegExpCaret] ^ + +# 158| [RegExpSequence] ^a+bc$ +#-----| 0 -> [RegExpCaret] ^ +#-----| 1 -> [RegExpPlus] a+ +#-----| 2 -> [RegExpConstant, RegExpNormalChar] bc +#-----| 3 -> [RegExpDollar] $ + +# 158| [RegExpConstant, RegExpNormalChar] a + +# 158| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 158| [RegExpConstant, RegExpNormalChar] bc + +# 158| [RegExpDollar] $ + +# 159| [RegExpWordBoundary] \b + +# 159| [RegExpSequence] \ba+bc\B +#-----| 0 -> [RegExpWordBoundary] \b +#-----| 1 -> [RegExpPlus] a+ +#-----| 2 -> [RegExpConstant, RegExpNormalChar] bc +#-----| 3 -> [RegExpNonWordBoundary] \B + +# 159| [RegExpConstant, RegExpNormalChar] a + +# 159| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 159| [RegExpConstant, RegExpNormalChar] bc + +# 159| [RegExpNonWordBoundary] \B + +# 163| [RegExpGroup] (foo) +#-----| 0 -> [RegExpConstant, RegExpNormalChar] foo + +# 163| [RegExpStar] (foo)* +#-----| 0 -> [RegExpGroup] (foo) + +# 163| [RegExpSequence] (foo)*bar +#-----| 0 -> [RegExpStar] (foo)* +#-----| 1 -> [RegExpConstant, RegExpNormalChar] bar + +# 163| [RegExpConstant, RegExpNormalChar] foo + +# 163| [RegExpConstant, RegExpNormalChar] bar + +# 164| [RegExpConstant, RegExpNormalChar] fo + +# 164| [RegExpSequence] fo(o|b+)ar +#-----| 0 -> [RegExpConstant, RegExpNormalChar] fo +#-----| 1 -> [RegExpGroup] (o|b+) +#-----| 2 -> [RegExpConstant, RegExpNormalChar] ar + +# 164| [RegExpGroup] (o|b+) +#-----| 0 -> [RegExpAlt] o|b+ + +# 164| [RegExpConstant, RegExpNormalChar] o + +# 164| [RegExpAlt] o|b+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] o +#-----| 1 -> [RegExpPlus] b+ + +# 164| [RegExpConstant, RegExpNormalChar] b + +# 164| [RegExpPlus] b+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] b + +# 164| [RegExpConstant, RegExpNormalChar] ar + +# 165| [RegExpGroup] (a|b|cd) +#-----| 0 -> [RegExpAlt] a|b|cd + +# 165| [RegExpSequence] (a|b|cd)e+ +#-----| 0 -> [RegExpGroup] (a|b|cd) +#-----| 1 -> [RegExpPlus] e+ + +# 165| [RegExpConstant, RegExpNormalChar] a + +# 165| [RegExpAlt] a|b|cd +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] b +#-----| 2 -> [RegExpConstant, RegExpNormalChar] cd + +# 165| [RegExpConstant, RegExpNormalChar] b + +# 165| [RegExpConstant, RegExpNormalChar] cd + +# 165| [RegExpConstant, RegExpNormalChar] e + +# 165| [RegExpPlus] e+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] e + +# 166| [RegExpGroup] (?::+) +#-----| 0 -> [RegExpPlus] :+ + +# 166| [RegExpSequence] (?::+)\w +#-----| 0 -> [RegExpGroup] (?::+) +#-----| 1 -> [RegExpCharacterClassEscape] \w + +# 166| [RegExpConstant, RegExpNormalChar] : + +# 166| [RegExpPlus] :+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] : + +# 166| [RegExpCharacterClassEscape] \w + +# 170| [RegExpGroup] (?\w+) +#-----| 0 -> [RegExpPlus] \w+ + +# 170| [RegExpCharacterClassEscape] \w + +# 170| [RegExpPlus] \w+ +#-----| 0 -> [RegExpCharacterClassEscape] \w + +# 171| [RegExpGroup] (?[a-z]+) +#-----| 0 -> [RegExpPlus] [a-z]+ + +# 171| [RegExpSequence] (?[a-z]+)(?[0-9]+) +#-----| 0 -> [RegExpGroup] (?[a-z]+) +#-----| 1 -> [RegExpGroup] (?[0-9]+) + +# 171| [RegExpCharacterClass] [a-z] +#-----| 0 -> [RegExpCharacterRange] a-z + +# 171| [RegExpPlus] [a-z]+ +#-----| 0 -> [RegExpCharacterClass] [a-z] + +# 171| [RegExpConstant, RegExpNormalChar] a + +# 171| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 171| [RegExpConstant, RegExpNormalChar] z + +# 171| [RegExpGroup] (?[0-9]+) +#-----| 0 -> [RegExpPlus] [0-9]+ + +# 171| [RegExpCharacterClass] [0-9] +#-----| 0 -> [RegExpCharacterRange] 0-9 + +# 171| [RegExpPlus] [0-9]+ +#-----| 0 -> [RegExpCharacterClass] [0-9] + +# 171| [RegExpConstant, RegExpNormalChar] 0 + +# 171| [RegExpCharacterRange] 0-9 +#-----| 0 -> [RegExpConstant, RegExpNormalChar] 0 +#-----| 1 -> [RegExpConstant, RegExpNormalChar] 9 + +# 171| [RegExpConstant, RegExpNormalChar] 9 + +# 175| [RegExpGroup] (a+) +#-----| 0 -> [RegExpPlus] a+ + +# 175| [RegExpSequence] (a+)b+\1 +#-----| 0 -> [RegExpGroup] (a+) +#-----| 1 -> [RegExpPlus] b+ +#-----| 2 -> [RegExpBackRef] \1 + +# 175| [RegExpConstant, RegExpNormalChar] a + +# 175| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 175| [RegExpConstant, RegExpNormalChar] b + +# 175| [RegExpPlus] b+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] b + +# 175| [RegExpBackRef] \1 + +# 176| [RegExpGroup] (?q+) +#-----| 0 -> [RegExpPlus] q+ + +# 176| [RegExpSequence] (?q+)\s+\k+ +#-----| 0 -> [RegExpGroup] (?q+) +#-----| 1 -> [RegExpPlus] \s+ +#-----| 2 -> [RegExpPlus] \k+ + +# 176| [RegExpConstant, RegExpNormalChar] q + +# 176| [RegExpPlus] q+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] q + +# 176| [RegExpCharacterClassEscape] \s + +# 176| [RegExpPlus] \s+ +#-----| 0 -> [RegExpCharacterClassEscape] \s + +# 176| [RegExpBackRef] \k + +# 176| [RegExpPlus] \k+ +#-----| 0 -> [RegExpBackRef] \k + +# 180| [RegExpPositiveLookahead] (?=\w+) + +# 180| [RegExpSequence] (?=\w+)abc +#-----| 0 -> [RegExpPositiveLookahead] (?=\w+) +#-----| 1 -> [RegExpConstant, RegExpNormalChar] abc + +# 180| [RegExpCharacterClassEscape] \w + +# 180| [RegExpPlus] \w+ +#-----| 0 -> [RegExpCharacterClassEscape] \w + +# 180| [RegExpConstant, RegExpNormalChar] abc + +# 181| [RegExpNegativeLookahead] (?!\d+) + +# 181| [RegExpSequence] (?!\d+)abc +#-----| 0 -> [RegExpNegativeLookahead] (?!\d+) +#-----| 1 -> [RegExpConstant, RegExpNormalChar] abc + +# 181| [RegExpCharacterClassEscape] \d + +# 181| [RegExpPlus] \d+ +#-----| 0 -> [RegExpCharacterClassEscape] \d + +# 181| [RegExpConstant, RegExpNormalChar] abc + +# 182| [RegExpConstant, RegExpNormalChar] a + +# 182| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 182| [RegExpSequence] a+bc(?<=\w) +#-----| 0 -> [RegExpPlus] a+ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] bc +#-----| 2 -> [RegExpPositiveLookbehind] (?<=\w) + +# 182| [RegExpConstant, RegExpNormalChar] bc + +# 182| [RegExpPositiveLookbehind] (?<=\w) + +# 182| [RegExpCharacterClassEscape] \w + +# 183| [RegExpConstant, RegExpNormalChar] a + +# 183| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 183| [RegExpSequence] a+bc(? [RegExpPlus] a+ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] bc +#-----| 2 -> [RegExpNegativeLookbehind] (? [RegExpPlus] a+ + +# 187| [RegExpPlus] (a+)+ +#-----| 0 -> [RegExpGroup] (a+) + +# 187| [RegExpConstant, RegExpNormalChar] a + +# 187| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 188| [RegExpGroup] (a*) +#-----| 0 -> [RegExpStar] a* + +# 188| [RegExpStar] (a*)* +#-----| 0 -> [RegExpGroup] (a*) + +# 188| [RegExpSequence] (a*)*b +#-----| 0 -> [RegExpStar] (a*)* +#-----| 1 -> [RegExpConstant, RegExpNormalChar] b + +# 188| [RegExpConstant, RegExpNormalChar] a + +# 188| [RegExpStar] a* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 188| [RegExpConstant, RegExpNormalChar] b + +# 189| [RegExpGroup] ([a-zA-Z]+) +#-----| 0 -> [RegExpPlus] [a-zA-Z]+ + +# 189| [RegExpStar] ([a-zA-Z]+)* +#-----| 0 -> [RegExpGroup] ([a-zA-Z]+) + +# 189| [RegExpCharacterClass] [a-zA-Z] +#-----| 0 -> [RegExpCharacterRange] a-z +#-----| 1 -> [RegExpCharacterRange] A-Z + +# 189| [RegExpPlus] [a-zA-Z]+ +#-----| 0 -> [RegExpCharacterClass] [a-zA-Z] + +# 189| [RegExpConstant, RegExpNormalChar] a + +# 189| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 189| [RegExpConstant, RegExpNormalChar] z + +# 189| [RegExpConstant, RegExpNormalChar] A + +# 189| [RegExpCharacterRange] A-Z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] Z + +# 189| [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpCharacterClass] [a-zA-Z0-9._%+-] +#-----| 0 -> [RegExpCharacterRange] a-z +#-----| 1 -> [RegExpCharacterRange] A-Z +#-----| 2 -> [RegExpCharacterRange] 0-9 +#-----| 3 -> [RegExpConstant, RegExpNormalChar] . +#-----| 4 -> [RegExpConstant, RegExpNormalChar] _ +#-----| 5 -> [RegExpConstant, RegExpNormalChar] % +#-----| 6 -> [RegExpConstant, RegExpNormalChar] + +#-----| 7 -> [RegExpConstant, RegExpNormalChar] - + +# 193| [RegExpPlus] [a-zA-Z0-9._%+-]+ +#-----| 0 -> [RegExpCharacterClass] [a-zA-Z0-9._%+-] + +# 193| [RegExpSequence] [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,} +#-----| 0 -> [RegExpPlus] [a-zA-Z0-9._%+-]+ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] @ +#-----| 2 -> [RegExpPlus] [a-zA-Z0-9.-]+ +#-----| 3 -> [RegExpConstant, RegExpEscape] \. +#-----| 4 -> [RegExpRange] [a-zA-Z]{2,} + +# 193| [RegExpConstant, RegExpNormalChar] a + +# 193| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] A + +# 193| [RegExpCharacterRange] A-Z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpConstant, RegExpNormalChar] 0 + +# 193| [RegExpCharacterRange] 0-9 +#-----| 0 -> [RegExpConstant, RegExpNormalChar] 0 +#-----| 1 -> [RegExpConstant, RegExpNormalChar] 9 + +# 193| [RegExpConstant, RegExpNormalChar] 9 + +# 193| [RegExpConstant, RegExpNormalChar] . + +# 193| [RegExpConstant, RegExpNormalChar] _ + +# 193| [RegExpConstant, RegExpNormalChar] % + +# 193| [RegExpConstant, RegExpNormalChar] + + +# 193| [RegExpConstant, RegExpNormalChar] - + +# 193| [RegExpConstant, RegExpNormalChar] @ + +# 193| [RegExpCharacterClass] [a-zA-Z0-9.-] +#-----| 0 -> [RegExpCharacterRange] a-z +#-----| 1 -> [RegExpCharacterRange] A-Z +#-----| 2 -> [RegExpCharacterRange] 0-9 +#-----| 3 -> [RegExpConstant, RegExpNormalChar] . +#-----| 4 -> [RegExpConstant, RegExpNormalChar] - + +# 193| [RegExpPlus] [a-zA-Z0-9.-]+ +#-----| 0 -> [RegExpCharacterClass] [a-zA-Z0-9.-] + +# 193| [RegExpConstant, RegExpNormalChar] a + +# 193| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] A + +# 193| [RegExpCharacterRange] A-Z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpConstant, RegExpNormalChar] 0 + +# 193| [RegExpCharacterRange] 0-9 +#-----| 0 -> [RegExpConstant, RegExpNormalChar] 0 +#-----| 1 -> [RegExpConstant, RegExpNormalChar] 9 + +# 193| [RegExpConstant, RegExpNormalChar] 9 + +# 193| [RegExpConstant, RegExpNormalChar] . + +# 193| [RegExpConstant, RegExpNormalChar] - + +# 193| [RegExpConstant, RegExpEscape] \. + +# 193| [RegExpCharacterClass] [a-zA-Z] +#-----| 0 -> [RegExpCharacterRange] a-z +#-----| 1 -> [RegExpCharacterRange] A-Z + +# 193| [RegExpRange] [a-zA-Z]{2,} +#-----| 0 -> [RegExpCharacterClass] [a-zA-Z] + +# 193| [RegExpConstant, RegExpNormalChar] a + +# 193| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] z + +# 193| [RegExpConstant, RegExpNormalChar] A + +# 193| [RegExpCharacterRange] A-Z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] A +#-----| 1 -> [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpConstant, RegExpNormalChar] Z + +# 193| [RegExpNormalChar] 2,} + +# 194| [RegExpGroup] (https?|ftp) +#-----| 0 -> [RegExpAlt] https?|ftp + +# 194| [RegExpSequence] (https?|ftp)://[^\s/$.?#].[^\s]* +#-----| 0 -> [RegExpGroup] (https?|ftp) +#-----| 1 -> [RegExpConstant, RegExpNormalChar] :// +#-----| 2 -> [RegExpCharacterClass] [^\s/$.?#] +#-----| 3 -> [RegExpDot] . +#-----| 4 -> [RegExpStar] [^\s]* + +# 194| [RegExpConstant, RegExpNormalChar] http + +# 194| [RegExpSequence] https? +#-----| 0 -> [RegExpConstant, RegExpNormalChar] http +#-----| 1 -> [RegExpOpt] s? + +# 194| [RegExpAlt] https?|ftp +#-----| 0 -> [RegExpSequence] https? +#-----| 1 -> [RegExpConstant, RegExpNormalChar] ftp + +# 194| [RegExpConstant, RegExpNormalChar] s + +# 194| [RegExpOpt] s? +#-----| 0 -> [RegExpConstant, RegExpNormalChar] s + +# 194| [RegExpConstant, RegExpNormalChar] ftp + +# 194| [RegExpConstant, RegExpNormalChar] :// + +# 194| [RegExpCharacterClass] [^\s/$.?#] +#-----| 0 -> [RegExpCharacterClassEscape] \s +#-----| 1 -> [RegExpConstant, RegExpNormalChar] / +#-----| 2 -> [RegExpConstant, RegExpNormalChar] $ +#-----| 3 -> [RegExpConstant, RegExpNormalChar] . +#-----| 4 -> [RegExpConstant, RegExpNormalChar] ? +#-----| 5 -> [RegExpConstant, RegExpNormalChar] # + +# 194| [RegExpCharacterClassEscape] \s + +# 194| [RegExpConstant, RegExpNormalChar] / + +# 194| [RegExpConstant, RegExpNormalChar] $ + +# 194| [RegExpConstant, RegExpNormalChar] . + +# 194| [RegExpConstant, RegExpNormalChar] ? + +# 194| [RegExpConstant, RegExpNormalChar] # + +# 194| [RegExpDot] . + +# 194| [RegExpCharacterClass] [^\s] +#-----| 0 -> [RegExpCharacterClassEscape] \s + +# 194| [RegExpStar] [^\s]* +#-----| 0 -> [RegExpCharacterClass] [^\s] + +# 194| [RegExpCharacterClassEscape] \s + +# 209| [RegExpGroup] (a+) +#-----| 0 -> [RegExpPlus] a+ + +# 209| [RegExpPlus] (a+)+ +#-----| 0 -> [RegExpGroup] (a+) + +# 209| [RegExpSequence] (a+)+b +#-----| 0 -> [RegExpPlus] (a+)+ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] b + +# 209| [RegExpConstant, RegExpNormalChar] a + +# 209| [RegExpPlus] a+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 209| [RegExpConstant, RegExpNormalChar] b + +# 218| [RegExpConstant, RegExpNormalChar] fo + +# 218| [RegExpSequence] foo+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] fo +#-----| 1 -> [RegExpPlus] o+ + +# 218| [RegExpConstant, RegExpNormalChar] o + +# 218| [RegExpPlus] o+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] o + +# 225| [RegExpConstant, RegExpNormalChar] ba + +# 225| [RegExpSequence] bar+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ba +#-----| 1 -> [RegExpPlus] r+ + +# 225| [RegExpConstant, RegExpNormalChar] r + +# 225| [RegExpPlus] r+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] r + +# 231| [RegExpConstant, RegExpNormalChar] ab + +# 231| [RegExpSequence] abc+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ab +#-----| 1 -> [RegExpPlus] c+ + +# 231| [RegExpConstant, RegExpNormalChar] c + +# 231| [RegExpPlus] c+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] c + +# 242| [RegExpConstant, RegExpNormalChar] abc+ + +# 245| [RegExpCaret] ^ + +# 245| [RegExpSequence] ^gr+ep +#-----| 0 -> [RegExpCaret] ^ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] gr+ep + +# 245| [RegExpConstant, RegExpNormalChar] gr+ep + +# 249| [RegExpGroup] \(ab\) +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ab + +# 249| [RegExpStar] \(ab\)* +#-----| 0 -> [RegExpGroup] \(ab\) + +# 249| [RegExpSequence] \(ab\)*c.* +#-----| 0 -> [RegExpStar] \(ab\)* +#-----| 1 -> [RegExpConstant, RegExpNormalChar] c +#-----| 2 -> [RegExpStar] .* + +# 249| [RegExpConstant, RegExpNormalChar] ab + +# 249| [RegExpConstant, RegExpNormalChar] c + +# 249| [RegExpDot] . + +# 249| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 250| [RegExpConstant, RegExpNormalChar] a(b)c + +# 250| [RegExpSequence] a(b)c.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a(b)c +#-----| 1 -> [RegExpStar] .* + +# 250| [RegExpDot] . + +# 250| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 252| [RegExpConstant, RegExpNormalChar] a + +# 252| [RegExpRange] a\{2,5\} +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 252| [RegExpSequence] a\{2,5\}.* +#-----| 0 -> [RegExpRange] a\{2,5\} +#-----| 1 -> [RegExpStar] .* + +# 252| [RegExpNormalChar] {2,5\} + +# 252| [RegExpDot] . + +# 252| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 253| [RegExpConstant, RegExpNormalChar] a{2,5} + +# 253| [RegExpSequence] a{2,5}.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a{2,5} +#-----| 1 -> [RegExpStar] .* + +# 253| [RegExpDot] . + +# 253| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 255| [RegExpConstant, RegExpNormalChar] a+b? + +# 255| [RegExpSequence] a+b?.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a+b? +#-----| 1 -> [RegExpStar] .* + +# 255| [RegExpDot] . + +# 255| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 258| [RegExpConstant, RegExpNormalChar] *abc + +# 258| [RegExpSequence] *abc.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] *abc +#-----| 1 -> [RegExpStar] .* + +# 258| [RegExpDot] . + +# 258| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 259| [RegExpConstant, RegExpNormalChar] a + +# 259| [RegExpStar] a* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 259| [RegExpSequence] a*.* +#-----| 0 -> [RegExpStar] a* +#-----| 1 -> [RegExpStar] .* + +# 259| [RegExpDot] . + +# 259| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 263| [RegExpConstant, RegExpNormalChar] a^b$c + +# 263| [RegExpSequence] a^b$c.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a^b$c +#-----| 1 -> [RegExpStar] .* + +# 263| [RegExpDot] . + +# 263| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 265| [RegExpGroup] \(a\) +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 265| [RegExpSequence] \(a\)\1.* +#-----| 0 -> [RegExpGroup] \(a\) +#-----| 1 -> [RegExpBackRef] \1 +#-----| 2 -> [RegExpStar] .* + +# 265| [RegExpConstant, RegExpNormalChar] a + +# 265| [RegExpBackRef] \1 + +# 265| [RegExpDot] . + +# 265| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 268| [RegExpConstant, RegExpEscape] \. + +# 268| [RegExpSequence] \.\*\\.* +#-----| 0 -> [RegExpConstant, RegExpEscape] \. +#-----| 1 -> [RegExpConstant, RegExpEscape] \* +#-----| 2 -> [RegExpConstant, RegExpEscape] \\ +#-----| 3 -> [RegExpStar] .* + +# 268| [RegExpConstant, RegExpEscape] \* + +# 268| [RegExpConstant, RegExpEscape] \\ + +# 268| [RegExpDot] . + +# 268| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 271| [RegExpGroup] \([[:alpha:]]\) +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]] + +# 271| [RegExpStar] \([[:alpha:]]\)* +#-----| 0 -> [RegExpGroup] \([[:alpha:]]\) + +# 271| [RegExpSequence] \([[:alpha:]]\)*.* +#-----| 0 -> [RegExpStar] \([[:alpha:]]\)* +#-----| 1 -> [RegExpStar] .* + +# 271| [RegExpCharacterClass] [[:alpha:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] + +# 271| [RegExpPosixBracket] [:alpha:] + +# 271| [RegExpDot] . + +# 271| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 275| [RegExpGroup] \(a*\) +#-----| 0 -> [RegExpStar] a* + +# 275| [RegExpStar] \(a*\)* +#-----| 0 -> [RegExpGroup] \(a*\) + +# 275| [RegExpConstant, RegExpNormalChar] a + +# 275| [RegExpStar] a* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a + +# 290| [RegExpConstant, RegExpNormalChar] ab + +# 290| [RegExpSequence] abc+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ab +#-----| 1 -> [RegExpPlus] c+ + +# 290| [RegExpConstant, RegExpNormalChar] c + +# 290| [RegExpPlus] c+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] c + +# 293| [RegExpCharacterClass] [[:alpha:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] + +# 293| [RegExpPlus] [[:alpha:]]+ +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]] + +# 293| [RegExpPosixBracket] [:alpha:] + +# 297| [RegExpConstant, RegExpNormalChar] a + +# 297| [RegExpSequence] a\.b\(c\).* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpEscape] \. +#-----| 2 -> [RegExpConstant, RegExpNormalChar] b +#-----| 3 -> [RegExpConstant, RegExpEscape] \( +#-----| 4 -> [RegExpConstant, RegExpNormalChar] c +#-----| 5 -> [RegExpConstant, RegExpEscape] \) +#-----| 6 -> [RegExpStar] .* + +# 297| [RegExpConstant, RegExpEscape] \. + +# 297| [RegExpConstant, RegExpNormalChar] b + +# 297| [RegExpConstant, RegExpEscape] \( + +# 297| [RegExpConstant, RegExpNormalChar] c + +# 297| [RegExpConstant, RegExpEscape] \) + +# 297| [RegExpDot] . + +# 297| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 299| [RegExpGroup] (foo|bar) +#-----| 0 -> [RegExpAlt] foo|bar + +# 299| [RegExpRange] (foo|bar){2,3} +#-----| 0 -> [RegExpGroup] (foo|bar) + +# 299| [RegExpConstant, RegExpNormalChar] foo + +# 299| [RegExpAlt] foo|bar +#-----| 0 -> [RegExpConstant, RegExpNormalChar] foo +#-----| 1 -> [RegExpConstant, RegExpNormalChar] bar + +# 299| [RegExpConstant, RegExpNormalChar] bar + +# 299| [RegExpNormalChar] 2,3} + +# 301| [RegExpConstant, RegExpNormalChar] aw + +# 301| [RegExpSequence] awk+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] aw +#-----| 1 -> [RegExpPlus] k+ + +# 301| [RegExpConstant, RegExpNormalChar] k + +# 301| [RegExpPlus] k+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] k + +# 303| [RegExpGroup] (x|y) +#-----| 0 -> [RegExpAlt] x|y + +# 303| [RegExpPlus] (x|y)+ +#-----| 0 -> [RegExpGroup] (x|y) + +# 303| [RegExpConstant, RegExpNormalChar] x + +# 303| [RegExpAlt] x|y +#-----| 0 -> [RegExpConstant, RegExpNormalChar] x +#-----| 1 -> [RegExpConstant, RegExpNormalChar] y + +# 303| [RegExpConstant, RegExpNormalChar] y + +# 305| [RegExpCaret] ^ + +# 305| [RegExpSequence] ^a.*b$ +#-----| 0 -> [RegExpCaret] ^ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] a +#-----| 2 -> [RegExpStar] .* +#-----| 3 -> [RegExpConstant, RegExpNormalChar] b +#-----| 4 -> [RegExpDollar] $ + +# 305| [RegExpConstant, RegExpNormalChar] a + +# 305| [RegExpDot] . + +# 305| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 305| [RegExpConstant, RegExpNormalChar] b + +# 305| [RegExpDollar] $ + +# 309| [RegExpGroup] ([a-z]+) +#-----| 0 -> [RegExpPlus] [a-z]+ + +# 309| [RegExpPlus] ([a-z]+)+ +#-----| 0 -> [RegExpGroup] ([a-z]+) + +# 309| [RegExpCharacterClass] [a-z] +#-----| 0 -> [RegExpCharacterRange] a-z + +# 309| [RegExpPlus] [a-z]+ +#-----| 0 -> [RegExpCharacterClass] [a-z] + +# 309| [RegExpConstant, RegExpNormalChar] a + +# 309| [RegExpCharacterRange] a-z +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 309| [RegExpConstant, RegExpNormalChar] z + +# 315| [RegExpConstant, RegExpNormalChar] ba + +# 315| [RegExpSequence] baz+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ba +#-----| 1 -> [RegExpPlus] z+ + +# 315| [RegExpConstant, RegExpNormalChar] z + +# 315| [RegExpPlus] z+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] z + +# 320| [RegExpConstant, RegExpNormalChar] pa + +# 320| [RegExpSequence] pat+ern +#-----| 0 -> [RegExpConstant, RegExpNormalChar] pa +#-----| 1 -> [RegExpPlus] t+ +#-----| 2 -> [RegExpConstant, RegExpNormalChar] ern + +# 320| [RegExpConstant, RegExpNormalChar] t + +# 320| [RegExpPlus] t+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] t + +# 320| [RegExpConstant, RegExpNormalChar] ern + +# 330| [RegExpConstant, RegExpNormalChar] he + +# 330| [RegExpSequence] hel+o +#-----| 0 -> [RegExpConstant, RegExpNormalChar] he +#-----| 1 -> [RegExpPlus] l+ +#-----| 2 -> [RegExpConstant, RegExpNormalChar] o + +# 330| [RegExpConstant, RegExpNormalChar] l + +# 330| [RegExpPlus] l+ +#-----| 0 -> [RegExpConstant, RegExpNormalChar] l + +# 330| [RegExpConstant, RegExpNormalChar] o + +# 346| [RegExpCharacterClass] [[:alpha:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] + +# 346| [RegExpSequence] [[:alpha:]].* +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]] +#-----| 1 -> [RegExpStar] .* + +# 346| [RegExpPosixBracket] [:alpha:] + +# 346| [RegExpDot] . + +# 346| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 348| [RegExpCharacterClass] [[:digit:]] +#-----| 0 -> [RegExpPosixBracket] [:digit:] + +# 348| [RegExpPlus] [[:digit:]]+ +#-----| 0 -> [RegExpCharacterClass] [[:digit:]] + +# 348| [RegExpPosixBracket] [:digit:] + +# 350| [RegExpCharacterClass] [a[:space:]] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpPosixBracket] [:space:] + +# 350| [RegExpSequence] [a[:space:]].* +#-----| 0 -> [RegExpCharacterClass] [a[:space:]] +#-----| 1 -> [RegExpStar] .* + +# 350| [RegExpConstant, RegExpNormalChar] a + +# 350| [RegExpPosixBracket] [:space:] + +# 350| [RegExpDot] . + +# 350| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 352| [RegExpCharacterClass] [[:alpha:][:digit:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] +#-----| 1 -> [RegExpPosixBracket] [:digit:] + +# 352| [RegExpSequence] [[:alpha:][:digit:]].* +#-----| 0 -> [RegExpCharacterClass] [[:alpha:][:digit:]] +#-----| 1 -> [RegExpStar] .* + +# 352| [RegExpPosixBracket] [:alpha:] + +# 352| [RegExpPosixBracket] [:digit:] + +# 352| [RegExpDot] . + +# 352| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 354| [RegExpCharacterClass] [^[:space:]] +#-----| 0 -> [RegExpPosixBracket] [:space:] + +# 354| [RegExpSequence] [^[:space:]].* +#-----| 0 -> [RegExpCharacterClass] [^[:space:]] +#-----| 1 -> [RegExpStar] .* + +# 354| [RegExpPosixBracket] [:space:] + +# 354| [RegExpDot] . + +# 354| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 356| [RegExpCharacterClass] [[.a.]] +#-----| 0 -> [RegExpPosixBracket] [.a.] + +# 356| [RegExpSequence] [[.a.]].* +#-----| 0 -> [RegExpCharacterClass] [[.a.]] +#-----| 1 -> [RegExpStar] .* + +# 356| [RegExpPosixBracket] [.a.] + +# 356| [RegExpDot] . + +# 356| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 358| [RegExpCharacterClass] [[=a=]] +#-----| 0 -> [RegExpPosixBracket] [=a=] + +# 358| [RegExpSequence] [[=a=]].* +#-----| 0 -> [RegExpCharacterClass] [[=a=]] +#-----| 1 -> [RegExpStar] .* + +# 358| [RegExpPosixBracket] [=a=] + +# 358| [RegExpDot] . + +# 358| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 361| [RegExpGroup] ([[:alpha:]]+) +#-----| 0 -> [RegExpPlus] [[:alpha:]]+ + +# 361| [RegExpPlus] ([[:alpha:]]+)+ +#-----| 0 -> [RegExpGroup] ([[:alpha:]]+) + +# 361| [RegExpCharacterClass] [[:alpha:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] + +# 361| [RegExpPlus] [[:alpha:]]+ +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]] + +# 361| [RegExpPosixBracket] [:alpha:] + +# 380| [RegExpCharacterClass] [:alpha:] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] : +#-----| 1 -> [RegExpConstant, RegExpNormalChar] a +#-----| 2 -> [RegExpConstant, RegExpNormalChar] l +#-----| 3 -> [RegExpConstant, RegExpNormalChar] p +#-----| 4 -> [RegExpConstant, RegExpNormalChar] h +#-----| 5 -> [RegExpConstant, RegExpNormalChar] a +#-----| 6 -> [RegExpConstant, RegExpNormalChar] : + +# 380| [RegExpSequence] [:alpha:].* +#-----| 0 -> [RegExpCharacterClass] [:alpha:] +#-----| 1 -> [RegExpStar] .* + +# 380| [RegExpConstant, RegExpNormalChar] : + +# 380| [RegExpConstant, RegExpNormalChar] a + +# 380| [RegExpConstant, RegExpNormalChar] l + +# 380| [RegExpConstant, RegExpNormalChar] p + +# 380| [RegExpConstant, RegExpNormalChar] h + +# 380| [RegExpConstant, RegExpNormalChar] a + +# 380| [RegExpConstant, RegExpNormalChar] : + +# 380| [RegExpDot] . + +# 380| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 386| [RegExpConstant, RegExpNormalChar] a + +# 386| [RegExpSequence] a[:b:]c.* +#-----| 0 -> [RegExpConstant, RegExpNormalChar] a +#-----| 1 -> [RegExpCharacterClass] [:b:] +#-----| 2 -> [RegExpConstant, RegExpNormalChar] c +#-----| 3 -> [RegExpStar] .* + +# 386| [RegExpCharacterClass] [:b:] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] : +#-----| 1 -> [RegExpConstant, RegExpNormalChar] b +#-----| 2 -> [RegExpConstant, RegExpNormalChar] : + +# 386| [RegExpConstant, RegExpNormalChar] : + +# 386| [RegExpConstant, RegExpNormalChar] b + +# 386| [RegExpConstant, RegExpNormalChar] : + +# 386| [RegExpConstant, RegExpNormalChar] c + +# 386| [RegExpDot] . + +# 386| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 389| [RegExpCharacterClass] [[:alpha:][:digit:][:space:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] +#-----| 1 -> [RegExpPosixBracket] [:digit:] +#-----| 2 -> [RegExpPosixBracket] [:space:] + +# 389| [RegExpSequence] [[:alpha:][:digit:][:space:]].* +#-----| 0 -> [RegExpCharacterClass] [[:alpha:][:digit:][:space:]] +#-----| 1 -> [RegExpStar] .* + +# 389| [RegExpPosixBracket] [:alpha:] + +# 389| [RegExpPosixBracket] [:digit:] + +# 389| [RegExpPosixBracket] [:space:] + +# 389| [RegExpDot] . + +# 389| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 393| [RegExpCharacterClass] []a[:alpha:] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] ] +#-----| 1 -> [RegExpConstant, RegExpNormalChar] a +#-----| 2 -> [RegExpConstant, RegExpNormalChar] [ +#-----| 3 -> [RegExpConstant, RegExpNormalChar] : +#-----| 4 -> [RegExpConstant, RegExpNormalChar] a +#-----| 5 -> [RegExpConstant, RegExpNormalChar] l +#-----| 6 -> [RegExpConstant, RegExpNormalChar] p +#-----| 7 -> [RegExpConstant, RegExpNormalChar] h +#-----| 8 -> [RegExpConstant, RegExpNormalChar] a +#-----| 9 -> [RegExpConstant, RegExpNormalChar] : + +# 393| [RegExpSequence] []a[:alpha:]].* +#-----| 0 -> [RegExpCharacterClass] []a[:alpha:] +#-----| 1 -> [RegExpConstant, RegExpNormalChar] ] +#-----| 2 -> [RegExpStar] .* + +# 393| [RegExpConstant, RegExpNormalChar] ] + +# 393| [RegExpConstant, RegExpNormalChar] a + +# 393| [RegExpConstant, RegExpNormalChar] [ + +# 393| [RegExpConstant, RegExpNormalChar] : + +# 393| [RegExpConstant, RegExpNormalChar] a + +# 393| [RegExpConstant, RegExpNormalChar] l + +# 393| [RegExpConstant, RegExpNormalChar] p + +# 393| [RegExpConstant, RegExpNormalChar] h + +# 393| [RegExpConstant, RegExpNormalChar] a + +# 393| [RegExpConstant, RegExpNormalChar] : + +# 393| [RegExpConstant, RegExpNormalChar] ] + +# 393| [RegExpDot] . + +# 393| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 400| [RegExpCharacterClass] [[:alpha:]-z] +#-----| 0 -> [RegExpCharacterRange] [:alpha:]-z + +# 400| [RegExpSequence] [[:alpha:]-z].* +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]-z] +#-----| 1 -> [RegExpStar] .* + +# 400| [RegExpPosixBracket] [:alpha:] + +# 400| [RegExpCharacterRange] [:alpha:]-z +#-----| 0 -> [RegExpPosixBracket] [:alpha:] +#-----| 1 -> [RegExpConstant, RegExpNormalChar] z + +# 400| [RegExpConstant, RegExpNormalChar] z + +# 400| [RegExpDot] . + +# 400| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 406| [RegExpCharacterClass] [[:alpha] +#-----| 0 -> [RegExpConstant, RegExpNormalChar] [ +#-----| 1 -> [RegExpConstant, RegExpNormalChar] : +#-----| 2 -> [RegExpConstant, RegExpNormalChar] a +#-----| 3 -> [RegExpConstant, RegExpNormalChar] l +#-----| 4 -> [RegExpConstant, RegExpNormalChar] p +#-----| 5 -> [RegExpConstant, RegExpNormalChar] h +#-----| 6 -> [RegExpConstant, RegExpNormalChar] a + +# 406| [RegExpSequence] [[:alpha].* +#-----| 0 -> [RegExpCharacterClass] [[:alpha] +#-----| 1 -> [RegExpStar] .* + +# 406| [RegExpConstant, RegExpNormalChar] [ + +# 406| [RegExpConstant, RegExpNormalChar] : + +# 406| [RegExpConstant, RegExpNormalChar] a + +# 406| [RegExpConstant, RegExpNormalChar] l + +# 406| [RegExpConstant, RegExpNormalChar] p + +# 406| [RegExpConstant, RegExpNormalChar] h + +# 406| [RegExpConstant, RegExpNormalChar] a + +# 406| [RegExpDot] . + +# 406| [RegExpStar] .* +#-----| 0 -> [RegExpDot] . + +# 415| [RegExpCaret] ^ + +# 415| [RegExpSequence] ^([[:alpha:]]+[[:digit:]]*[[:space:]]?)+([[:punct:]]|[[:xdigit:]])+$ +#-----| 0 -> [RegExpCaret] ^ +#-----| 1 -> [RegExpPlus] ([[:alpha:]]+[[:digit:]]*[[:space:]]?)+ +#-----| 2 -> [RegExpPlus] ([[:punct:]]|[[:xdigit:]])+ +#-----| 3 -> [RegExpDollar] $ + +# 415| [RegExpGroup] ([[:alpha:]]+[[:digit:]]*[[:space:]]?) +#-----| 0 -> [RegExpSequence] [[:alpha:]]+[[:digit:]]*[[:space:]]? + +# 415| [RegExpPlus] ([[:alpha:]]+[[:digit:]]*[[:space:]]?)+ +#-----| 0 -> [RegExpGroup] ([[:alpha:]]+[[:digit:]]*[[:space:]]?) + +# 415| [RegExpCharacterClass] [[:alpha:]] +#-----| 0 -> [RegExpPosixBracket] [:alpha:] + +# 415| [RegExpPlus] [[:alpha:]]+ +#-----| 0 -> [RegExpCharacterClass] [[:alpha:]] + +# 415| [RegExpSequence] [[:alpha:]]+[[:digit:]]*[[:space:]]? +#-----| 0 -> [RegExpPlus] [[:alpha:]]+ +#-----| 1 -> [RegExpStar] [[:digit:]]* +#-----| 2 -> [RegExpOpt] [[:space:]]? + +# 415| [RegExpPosixBracket] [:alpha:] + +# 415| [RegExpCharacterClass] [[:digit:]] +#-----| 0 -> [RegExpPosixBracket] [:digit:] + +# 415| [RegExpStar] [[:digit:]]* +#-----| 0 -> [RegExpCharacterClass] [[:digit:]] + +# 415| [RegExpPosixBracket] [:digit:] + +# 415| [RegExpCharacterClass] [[:space:]] +#-----| 0 -> [RegExpPosixBracket] [:space:] + +# 415| [RegExpOpt] [[:space:]]? +#-----| 0 -> [RegExpCharacterClass] [[:space:]] + +# 415| [RegExpPosixBracket] [:space:] + +# 415| [RegExpGroup] ([[:punct:]]|[[:xdigit:]]) +#-----| 0 -> [RegExpAlt] [[:punct:]]|[[:xdigit:]] + +# 415| [RegExpPlus] ([[:punct:]]|[[:xdigit:]])+ +#-----| 0 -> [RegExpGroup] ([[:punct:]]|[[:xdigit:]]) + +# 415| [RegExpCharacterClass] [[:punct:]] +#-----| 0 -> [RegExpPosixBracket] [:punct:] + +# 415| [RegExpAlt] [[:punct:]]|[[:xdigit:]] +#-----| 0 -> [RegExpCharacterClass] [[:punct:]] +#-----| 1 -> [RegExpCharacterClass] [[:xdigit:]] + +# 415| [RegExpPosixBracket] [:punct:] + +# 415| [RegExpCharacterClass] [[:xdigit:]] +#-----| 0 -> [RegExpPosixBracket] [:xdigit:] + +# 415| [RegExpPosixBracket] [:xdigit:] + +# 415| [RegExpDollar] $ diff --git a/cpp/ql/test/library-tests/regex/parse.ql b/cpp/ql/test/library-tests/regex/parse.ql new file mode 100644 index 000000000000..e65e0cd324a0 --- /dev/null +++ b/cpp/ql/test/library-tests/regex/parse.ql @@ -0,0 +1,27 @@ +/** + * @kind graph + */ + +import cpp +import semmle.code.cpp.regex.RegexTreeView + +query predicate nodes(RegExpTerm n, string attr, string val) { + attr = "semmle.label" and + val = "[" + concat(n.getAPrimaryQlClass(), ", ") + "] " + n.toString() + or + attr = "semmle.order" and + val = + any(int i | + n = + rank[i](RegExpTerm t, string fp, int sl, int sc, int el, int ec | + t.hasLocationInfo(fp, sl, sc, el, ec) + | + t order by fp, sl, sc, el, ec, t.toString() + ) + ).toString() +} + +query predicate edges(RegExpTerm pred, RegExpTerm succ, string attr, string val) { + attr in ["semmle.label", "semmle.order"] and + val = any(int i | succ = pred.getChild(i)).toString() +} diff --git a/cpp/ql/test/library-tests/regex/regexp.expected b/cpp/ql/test/library-tests/regex/regexp.expected new file mode 100644 index 000000000000..fcd0042c40d5 --- /dev/null +++ b/cpp/ql/test/library-tests/regex/regexp.expected @@ -0,0 +1,719 @@ +groupName +| test.cpp:170:20:170:29 | (?\\w+) | id | +| test.cpp:171:20:171:35 | (?[a-z]+) | first | +| test.cpp:171:36:171:52 | (?[0-9]+) | second | +| test.cpp:176:20:176:29 | (?q+) | qux | +groupNumber +| test.cpp:163:20:163:24 | (foo) | 1 | +| test.cpp:164:22:164:27 | (o\|b+) | 1 | +| test.cpp:165:20:165:27 | (a\|b\|cd) | 1 | +| test.cpp:170:20:170:29 | (?\\w+) | 1 | +| test.cpp:171:20:171:35 | (?[a-z]+) | 1 | +| test.cpp:171:36:171:52 | (?[0-9]+) | 2 | +| test.cpp:175:20:175:23 | (a+) | 1 | +| test.cpp:176:20:176:29 | (?q+) | 1 | +| test.cpp:187:20:187:23 | (a+) | 1 | +| test.cpp:188:20:188:23 | (a*) | 1 | +| test.cpp:189:20:189:30 | ([a-zA-Z]+) | 1 | +| test.cpp:194:20:194:31 | (https?\|ftp) | 1 | +| test.cpp:209:19:209:22 | (a+) | 1 | +| test.cpp:249:20:249:25 | \\(ab\\) | 1 | +| test.cpp:265:21:265:25 | \\(a\\) | 1 | +| test.cpp:271:21:271:35 | \\([[:alpha:]]\\) | 1 | +| test.cpp:275:21:275:26 | \\(a*\\) | 1 | +| test.cpp:299:20:299:28 | (foo\|bar) | 1 | +| test.cpp:303:20:303:24 | (x\|y) | 1 | +| test.cpp:309:20:309:27 | ([a-z]+) | 1 | +| test.cpp:361:19:361:32 | ([[:alpha:]]+) | 1 | +| test.cpp:415:21:415:58 | ([[:alpha:]]+[[:digit:]]*[[:space:]]?) | 1 | +| test.cpp:415:60:415:85 | ([[:punct:]]\|[[:xdigit:]]) | 2 | +term +| test.cpp:133:20:133:20 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:133:20:133:21 | a* | RegExpStar | +| test.cpp:133:20:133:26 | a*b+c?d | RegExpSequence | +| test.cpp:133:22:133:22 | b | RegExpConstant,RegExpNormalChar | +| test.cpp:133:22:133:23 | b+ | RegExpPlus | +| test.cpp:133:24:133:24 | c | RegExpConstant,RegExpNormalChar | +| test.cpp:133:24:133:25 | c? | RegExpOpt | +| test.cpp:133:26:133:26 | d | RegExpConstant,RegExpNormalChar | +| test.cpp:134:20:134:20 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:134:20:134:25 | a{4,8} | RegExpRange | +| test.cpp:134:22:134:25 | 4,8} | RegExpNormalChar | +| test.cpp:135:20:135:20 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:135:20:135:24 | a{3,} | RegExpRange | +| test.cpp:135:22:135:24 | 3,} | RegExpNormalChar | +| test.cpp:140:20:140:22 | foo | RegExpConstant,RegExpNormalChar | +| test.cpp:140:20:140:27 | foo\|bar+ | RegExpAlt | +| test.cpp:140:24:140:25 | ba | RegExpConstant,RegExpNormalChar | +| test.cpp:140:24:140:27 | bar+ | RegExpSequence | +| test.cpp:140:26:140:26 | r | RegExpConstant,RegExpNormalChar | +| test.cpp:140:26:140:27 | r+ | RegExpPlus | +| test.cpp:144:20:144:24 | [abc] | RegExpCharacterClass | +| test.cpp:144:20:144:25 | [abc]+ | RegExpPlus | +| test.cpp:144:21:144:21 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:144:22:144:22 | b | RegExpConstant,RegExpNormalChar | +| test.cpp:144:23:144:23 | c | RegExpConstant,RegExpNormalChar | +| test.cpp:145:20:145:31 | [a-fA-F0-9_] | RegExpCharacterClass | +| test.cpp:145:20:145:32 | [a-fA-F0-9_]+ | RegExpPlus | +| test.cpp:145:21:145:21 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:145:21:145:23 | a-f | RegExpCharacterRange | +| test.cpp:145:23:145:23 | f | RegExpConstant,RegExpNormalChar | +| test.cpp:145:24:145:24 | A | RegExpConstant,RegExpNormalChar | +| test.cpp:145:24:145:26 | A-F | RegExpCharacterRange | +| test.cpp:145:26:145:26 | F | RegExpConstant,RegExpNormalChar | +| test.cpp:145:27:145:27 | 0 | RegExpConstant,RegExpNormalChar | +| test.cpp:145:27:145:29 | 0-9 | RegExpCharacterRange | +| test.cpp:145:29:145:29 | 9 | RegExpConstant,RegExpNormalChar | +| test.cpp:145:30:145:30 | _ | RegExpConstant,RegExpNormalChar | +| test.cpp:146:20:146:23 | [\\w] | RegExpCharacterClass | +| test.cpp:146:20:146:24 | [\\w]+ | RegExpPlus | +| test.cpp:146:21:146:22 | \\w | RegExpCharacterClassEscape | +| test.cpp:147:20:147:25 | [^A-Z] | RegExpCharacterClass | +| test.cpp:147:20:147:26 | [^A-Z]* | RegExpStar | +| test.cpp:147:22:147:22 | A | RegExpConstant,RegExpNormalChar | +| test.cpp:147:22:147:24 | A-Z | RegExpCharacterRange | +| test.cpp:147:24:147:24 | Z | RegExpConstant,RegExpNormalChar | +| test.cpp:151:20:151:20 | . | RegExpDot | +| test.cpp:151:20:151:21 | .* | RegExpStar | +| test.cpp:152:20:152:21 | \\w | RegExpCharacterClassEscape | +| test.cpp:152:20:152:22 | \\w+ | RegExpPlus | +| test.cpp:152:20:152:24 | \\w+\\W | RegExpSequence | +| test.cpp:152:23:152:24 | \\W | RegExpCharacterClassEscape | +| test.cpp:153:20:153:21 | \\s | RegExpCharacterClassEscape | +| test.cpp:153:20:153:22 | \\s+ | RegExpPlus | +| test.cpp:153:20:153:24 | \\s+\\S | RegExpSequence | +| test.cpp:153:23:153:24 | \\S | RegExpCharacterClassEscape | +| test.cpp:154:20:154:21 | \\d | RegExpCharacterClassEscape | +| test.cpp:154:20:154:22 | \\d+ | RegExpPlus | +| test.cpp:154:20:154:24 | \\d+\\D | RegExpSequence | +| test.cpp:154:23:154:24 | \\D | RegExpCharacterClassEscape | +| test.cpp:158:20:158:20 | ^ | RegExpCaret | +| test.cpp:158:20:158:25 | ^a+bc$ | RegExpSequence | +| test.cpp:158:21:158:21 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:158:21:158:22 | a+ | RegExpPlus | +| test.cpp:158:23:158:24 | bc | RegExpConstant,RegExpNormalChar | +| test.cpp:158:25:158:25 | $ | RegExpDollar | +| test.cpp:159:20:159:21 | \\b | RegExpWordBoundary | +| test.cpp:159:20:159:27 | \\ba+bc\\B | RegExpSequence | +| test.cpp:159:22:159:22 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:159:22:159:23 | a+ | RegExpPlus | +| test.cpp:159:24:159:25 | bc | RegExpConstant,RegExpNormalChar | +| test.cpp:159:26:159:27 | \\B | RegExpNonWordBoundary | +| test.cpp:163:20:163:24 | (foo) | RegExpGroup | +| test.cpp:163:20:163:25 | (foo)* | RegExpStar | +| test.cpp:163:20:163:28 | (foo)*bar | RegExpSequence | +| test.cpp:163:21:163:23 | foo | RegExpConstant,RegExpNormalChar | +| test.cpp:163:26:163:28 | bar | RegExpConstant,RegExpNormalChar | +| test.cpp:164:20:164:21 | fo | RegExpConstant,RegExpNormalChar | +| test.cpp:164:20:164:29 | fo(o\|b+)ar | RegExpSequence | +| test.cpp:164:22:164:27 | (o\|b+) | RegExpGroup | +| test.cpp:164:23:164:23 | o | RegExpConstant,RegExpNormalChar | +| test.cpp:164:23:164:26 | o\|b+ | RegExpAlt | +| test.cpp:164:25:164:25 | b | RegExpConstant,RegExpNormalChar | +| test.cpp:164:25:164:26 | b+ | RegExpPlus | +| test.cpp:164:28:164:29 | ar | RegExpConstant,RegExpNormalChar | +| test.cpp:165:20:165:27 | (a\|b\|cd) | RegExpGroup | +| test.cpp:165:20:165:29 | (a\|b\|cd)e+ | RegExpSequence | +| test.cpp:165:21:165:21 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:165:21:165:26 | a\|b\|cd | RegExpAlt | +| test.cpp:165:23:165:23 | b | RegExpConstant,RegExpNormalChar | +| test.cpp:165:25:165:26 | cd | RegExpConstant,RegExpNormalChar | +| test.cpp:165:28:165:28 | e | RegExpConstant,RegExpNormalChar | +| test.cpp:165:28:165:29 | e+ | RegExpPlus | +| test.cpp:166:20:166:25 | (?::+) | RegExpGroup | +| test.cpp:166:20:166:27 | (?::+)\\w | RegExpSequence | +| test.cpp:166:23:166:23 | : | RegExpConstant,RegExpNormalChar | +| test.cpp:166:23:166:24 | :+ | RegExpPlus | +| test.cpp:166:26:166:27 | \\w | RegExpCharacterClassEscape | +| test.cpp:170:20:170:29 | (?\\w+) | RegExpGroup | +| test.cpp:170:26:170:27 | \\w | RegExpCharacterClassEscape | +| test.cpp:170:26:170:28 | \\w+ | RegExpPlus | +| test.cpp:171:20:171:35 | (?[a-z]+) | RegExpGroup | +| test.cpp:171:20:171:52 | (?[a-z]+)(?[0-9]+) | RegExpSequence | +| test.cpp:171:29:171:33 | [a-z] | RegExpCharacterClass | +| test.cpp:171:29:171:34 | [a-z]+ | RegExpPlus | +| test.cpp:171:30:171:30 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:171:30:171:32 | a-z | RegExpCharacterRange | +| test.cpp:171:32:171:32 | z | RegExpConstant,RegExpNormalChar | +| test.cpp:171:36:171:52 | (?[0-9]+) | RegExpGroup | +| test.cpp:171:46:171:50 | [0-9] | RegExpCharacterClass | +| test.cpp:171:46:171:51 | [0-9]+ | RegExpPlus | +| test.cpp:171:47:171:47 | 0 | RegExpConstant,RegExpNormalChar | +| test.cpp:171:47:171:49 | 0-9 | RegExpCharacterRange | +| test.cpp:171:49:171:49 | 9 | RegExpConstant,RegExpNormalChar | +| test.cpp:175:20:175:23 | (a+) | RegExpGroup | +| test.cpp:175:20:175:27 | (a+)b+\\1 | RegExpSequence | +| test.cpp:175:21:175:21 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:175:21:175:22 | a+ | RegExpPlus | +| test.cpp:175:24:175:24 | b | RegExpConstant,RegExpNormalChar | +| test.cpp:175:24:175:25 | b+ | RegExpPlus | +| test.cpp:175:26:175:27 | \\1 | RegExpBackRef | +| test.cpp:176:20:176:29 | (?q+) | RegExpGroup | +| test.cpp:176:20:176:40 | (?q+)\\s+\\k+ | RegExpSequence | +| test.cpp:176:27:176:27 | q | RegExpConstant,RegExpNormalChar | +| test.cpp:176:27:176:28 | q+ | RegExpPlus | +| test.cpp:176:30:176:31 | \\s | RegExpCharacterClassEscape | +| test.cpp:176:30:176:32 | \\s+ | RegExpPlus | +| test.cpp:176:33:176:39 | \\k | RegExpBackRef | +| test.cpp:176:33:176:40 | \\k+ | RegExpPlus | +| test.cpp:180:20:180:26 | (?=\\w+) | RegExpPositiveLookahead | +| test.cpp:180:20:180:29 | (?=\\w+)abc | RegExpSequence | +| test.cpp:180:23:180:24 | \\w | RegExpCharacterClassEscape | +| test.cpp:180:23:180:25 | \\w+ | RegExpPlus | +| test.cpp:180:27:180:29 | abc | RegExpConstant,RegExpNormalChar | +| test.cpp:181:20:181:26 | (?!\\d+) | RegExpNegativeLookahead | +| test.cpp:181:20:181:29 | (?!\\d+)abc | RegExpSequence | +| test.cpp:181:23:181:24 | \\d | RegExpCharacterClassEscape | +| test.cpp:181:23:181:25 | \\d+ | RegExpPlus | +| test.cpp:181:27:181:29 | abc | RegExpConstant,RegExpNormalChar | +| test.cpp:182:20:182:20 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:182:20:182:21 | a+ | RegExpPlus | +| test.cpp:182:20:182:30 | a+bc(?<=\\w) | RegExpSequence | +| test.cpp:182:22:182:23 | bc | RegExpConstant,RegExpNormalChar | +| test.cpp:182:24:182:30 | (?<=\\w) | RegExpPositiveLookbehind | +| test.cpp:182:28:182:29 | \\w | RegExpCharacterClassEscape | +| test.cpp:183:20:183:20 | a | RegExpConstant,RegExpNormalChar | +| test.cpp:183:20:183:21 | a+ | RegExpPlus | +| test.cpp:183:20:183:30 | a+bc(? +class basic_string { +public: + basic_string() {} + basic_string(const CharT*) {} + const CharT* c_str() const { return 0; } + const CharT* data() const { return 0; } + unsigned long size() const { return 0; } +}; + +typedef basic_string string; +typedef basic_string wstring; + +namespace regex_constants { + // syntax_option_type constants. + enum syntax_option_type { + ECMAScript = 1, + basic = 2, + extended = 4, + awk = 8, + grep = 16, + egrep = 32, + icase = 256, + nosubs = 512, + optimize = 1024, + collate = 2048, + multiline = 4096 + }; + // match_flag_type constants. + enum match_flag_type { + match_default = 0, + match_not_bol = 1, + match_not_eol = 2, + match_any = 16, + format_default = 0, + format_sed = 1, + format_no_copy = 4, + format_first_only = 8 + }; + inline syntax_option_type operator|(syntax_option_type a, syntax_option_type b) { + return (syntax_option_type)((int)a | (int)b); + } + inline match_flag_type operator|(match_flag_type a, match_flag_type b) { + return (match_flag_type)((int)a | (int)b); + } +} // namespace regex_constants + +template +class basic_regex { +public: + typedef regex_constants::syntax_option_type flag_type; + basic_regex() {} + basic_regex(const CharT* p) {} + basic_regex(const CharT* p, flag_type f) {} + basic_regex(const basic_string& p) {} + basic_regex(const basic_string& p, flag_type f) {} + basic_regex& assign(const CharT* p) { return *this; } + basic_regex& assign(const CharT* p, flag_type f) { return *this; } + basic_regex& assign(const basic_string& p) { return *this; } + basic_regex& assign(const basic_string& p, flag_type f) { return *this; } +}; + +typedef basic_regex regex; +typedef basic_regex wregex; + +// smatch is a match_results<...> stub. +template class match_results { public: match_results() {} }; +typedef match_results cmatch; +typedef match_results smatch; + +// regex_match / regex_search overloads (subject then regex). +template +bool regex_match(const CharT* s, const basic_regex& re) { return false; } +template +bool regex_match(const basic_string& s, const basic_regex& re) { return false; } +template +bool regex_match(const basic_string& s, const basic_regex& re, + regex_constants::match_flag_type) { return false; } + +template +bool regex_search(const CharT* s, const basic_regex& re) { return false; } +template +bool regex_search(const basic_string& s, const basic_regex& re) { return false; } +template +bool regex_search(const basic_string& s, const basic_regex& re, + regex_constants::match_flag_type) { return false; } + +// regex_replace: (subject, regex, replacement). +template +basic_string +regex_replace(const basic_string& s, const basic_regex& re, + const basic_string& fmt) { return basic_string(); } +template +basic_string +regex_replace(const CharT* s, const basic_regex& re, const CharT* fmt) { + return basic_string(); +} + +// Iterator constructors: (begin, end, regex). +template +class regex_iterator { +public: + regex_iterator() {} + regex_iterator(Iter b, Iter e, const basic_regex& re) {} +}; + +template +class regex_token_iterator { +public: + regex_token_iterator() {} + regex_token_iterator(Iter b, Iter e, const basic_regex& re) {} +}; + +} // namespace std + +// ----------------------------------------------------------------------------- +// Phase 1 tests: syntactic constructs that the parser should recognize as +// regex terms once the strings are used as regexes. These literals now +// flow to a `std::regex` construction site. +// ----------------------------------------------------------------------------- + +void basic_sequence() { + std::regex r1("abc"); // NOT a `RegExp` (no +, *, {n,}) +} + +void repetition() { + std::regex r1("a*b+c?d"); // RegExp + std::regex r2("a{4,8}"); // NOT a RegExp (bounded) + std::regex r3("a{3,}"); // RegExp (unbounded {n,}) + std::regex r4("a{7}"); // NOT a RegExp +} + +void alternation() { + std::regex r1("foo|bar+"); // RegExp +} + +void character_classes() { + std::regex r1("[abc]+"); // RegExp + std::regex r2("[a-fA-F0-9_]+"); // RegExp + std::regex r3("[\\w]+"); // RegExp + std::regex r4("[^A-Z]*"); // RegExp +} + +void meta_classes() { + std::regex r1(".*"); // RegExp + std::regex r2("\\w+\\W"); // RegExp + std::regex r3("\\s+\\S"); // RegExp + std::regex r4("\\d+\\D"); // RegExp +} + +void anchors() { + std::regex r1("^a+bc$"); // RegExp + std::regex r2("\\ba+bc\\B"); // RegExp +} + +void groups() { + std::regex r1("(foo)*bar"); // RegExp + std::regex r2("fo(o|b+)ar"); // RegExp + std::regex r3("(a|b|cd)e+"); // RegExp + std::regex r4("(?::+)\\w"); // RegExp +} + +void named_groups() { + std::regex r1("(?\\w+)"); // RegExp + std::regex r2("(?[a-z]+)(?[0-9]+)"); // RegExp +} + +void backreferences() { + std::regex r1("(a+)b+\\1"); // RegExp + std::regex r2("(?q+)\\s+\\k+"); // RegExp +} + +void lookahead_lookbehind() { + std::regex r1("(?=\\w+)abc"); // RegExp + std::regex r2("(?!\\d+)abc"); // RegExp + std::regex r3("a+bc(?<=\\w)"); // RegExp + std::regex r4("a+bc(?a]+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '='. | +| test.cpp:157:23:157:24 | a+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | +| test.cpp:162:23:162:25 | \\n+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '\\n'. | +| test.cpp:167:28:167:28 | b | This part of the regular expression may cause exponential backtracking on strings starting with 'W' and containing many repetitions of 'bW'. | +| test.cpp:176:24:176:25 | ab | This part of the regular expression may cause exponential backtracking on strings starting with 'a' and containing many repetitions of 'ab'. | +| test.cpp:229:24:229:29 | [a-z]+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | +| test.cpp:241:24:241:25 | a+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | +| test.cpp:265:24:265:35 | [[:alpha:]]+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | +| test.cpp:267:24:267:35 | [[:alpha:]]+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '0'. | +| test.cpp:331:24:331:29 | [a-z]+ | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | +| test.cpp:375:25:375:26 | a* | This part of the regular expression may cause exponential backtracking on strings containing many repetitions of 'a'. | diff --git a/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/ReDoS.qlref b/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/ReDoS.qlref new file mode 100644 index 000000000000..7f6d41509c46 --- /dev/null +++ b/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/ReDoS.qlref @@ -0,0 +1,2 @@ +query: Security/CWE/CWE-1333/ReDoS.ql +postprocess: utils/test/InlineExpectationsTestQuery.ql diff --git a/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/test.cpp b/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/test.cpp new file mode 100644 index 000000000000..de849f27e1b6 --- /dev/null +++ b/cpp/ql/test/query-tests/Security/CWE/CWE-1333-ReDoS/test.cpp @@ -0,0 +1,388 @@ +// Minimal std::regex/std::basic_string stubs, mirroring the ones in +// cpp/ql/test/library-tests/regex/test.cpp. Real headers are not available +// in the extractor sandbox. + +namespace std { + +template +class basic_string { +public: + basic_string() {} + basic_string(const CharT*) {} + const CharT* c_str() const { return 0; } + const CharT* data() const { return 0; } + unsigned long size() const { return 0; } +}; + +typedef basic_string string; + +namespace regex_constants { + enum syntax_option_type { + ECMAScript = 1, basic = 2, extended = 4, awk = 8, grep = 16, egrep = 32, + icase = 256, nosubs = 512, optimize = 1024, collate = 2048, multiline = 4096 + }; + enum match_flag_type { match_default = 0 }; +} + +template +class basic_regex { +public: + typedef regex_constants::syntax_option_type flag_type; + basic_regex(const CharT* p) {} + basic_regex(const CharT* p, flag_type f) {} + basic_regex(const basic_string& p) {} +}; + +typedef basic_regex regex; + +template +bool regex_match(const basic_string& s, const basic_regex& re) { return false; } +template +bool regex_search(const basic_string& s, const basic_regex& re) { return false; } +template +basic_string regex_replace(const basic_string& s, const basic_regex& re, + const basic_string& fmt) { return basic_string(); } + +} // namespace std + +// ----------------------------------------------------------------------------- +// Tests for cpp/redos (exponential ReDoS). +// +// Ported from ruby/ql/test/query-tests/security/cwe-1333-exponential-redos/tst.rb. +// Each pattern is constructed as a `std::regex re("...")` and then used in +// `std::regex_match`, which is enough for Phase 1's parser to consider the +// literal a `RegExp`. +// +// Cases are annotated as: +// BAD - a cpp/redos alert is expected on the exponential-backtracking term +// GOOD - no cpp/redos alert is expected +// Any C++-specific divergence from Ruby/JS is noted inline. +// ----------------------------------------------------------------------------- + +void run(const std::regex& re, const std::string& s) { + (void)std::regex_match(s, re); +} + +int main(int argc, char** argv) { + std::string input(argv[1]); + + // ------------------------------------------------------------------------- + // 1. Classic nested quantifiers (Ruby bad79..bad82). + // ------------------------------------------------------------------------- + + // BAD: (a*)*b + { std::regex re("(a*)*b"); run(re, input); } // $ Alert + // BAD: (a+)*b + { std::regex re("(a+)*b"); run(re, input); } // $ Alert + // BAD: (a*)+b + { std::regex re("(a*)+b"); run(re, input); } // $ Alert + // BAD: (a+)+b + { std::regex re("(a+)+b"); run(re, input); } // $ Alert + // BAD: (a*)+ + { std::regex re("^(a*)+$"); run(re, input); } // $ Alert + // BAD: ((a+a?)*)+b+ (Ruby bad48) + { std::regex re("(((a+a?)*)+b+)"); run(re, input); } // $ Alert + // BAD: (a?a?)*b (Ruby bad71) + { std::regex re("(a?a?)*b"); run(re, input); } // $ Alert + // BAD: (a?)+b via non-capturing alt (Ruby bad73) + { std::regex re("(?:a|a?)+b"); run(re, input); } // $ Alert + // BAD: foo([\w-]*)+bar (Ruby bad69) + { std::regex re("foo([\\w-]*)+bar"); run(re, input); } // $ Alert + // BAD: ((ab)*)+c (Ruby bad70) + { std::regex re("((ab)*)+c"); run(re, input); } // $ Alert + + // ------------------------------------------------------------------------- + // 2. Anchored nested-quantifier patterns (Ruby bad7..bad10, bad77, bad78). + // ------------------------------------------------------------------------- + + // BAD: ^([a-z]+)+$ + { std::regex re("^([a-z]+)+$"); run(re, input); } // $ Alert + // BAD: ^([a-z]*)*$ + { std::regex re("^([a-z]*)*$"); run(re, input); } // $ Alert + // BAD: e-mail-like regex + { std::regex re("^([a-zA-Z0-9])(([\\.-]|[_]+)?([a-zA-Z0-9]+))*(@){1}[a-z0-9]+[.]{1}(([a-z]{2,3})|([a-z]{2,3}[.]{1}[a-z]{2,3}))$"); // $ Alert + run(re, input); } + // BAD: ^(([a-z])+.)+[A-Z]([a-z])+$ + { std::regex re("^(([a-z])+.)+[A-Z]([a-z])+$"); run(re, input); } // $ Alert + // BAD: ^((a)+\w)+$ + { std::regex re("^((a)+\\w)+$"); run(re, input); } // $ Alert + // BAD: ^(b+.)+$ + { std::regex re("^(b+.)+$"); run(re, input); } // $ Alert + // BAD: ^ab(c+)+$ (Ruby bad66) + { std::regex re("^ab(c+)+$"); run(re, input); } // $ Alert + + // ------------------------------------------------------------------------- + // 3. Alternations with overlapping / identical branches inside a repetition. + // ------------------------------------------------------------------------- + + // BAD: (a|a)* + { std::regex re("^(a|a)*$"); run(re, input); } // $ Alert + // BAD: (a|aa?)*b (Ruby bad15) + { std::regex re("(a|aa?)*b"); run(re, input); } // $ Alert + // BAD: (b|a?b)*c (Ruby bad13) + { std::regex re("(b|a?b)*c"); run(re, input); } // $ Alert + // BAD: (a+|b+|c+)*c (Ruby bad47) + { std::regex re("(a+|b+|c+)*c"); run(re, input); } // $ Alert + + // ------------------------------------------------------------------------- + // 4. Character-class / complement ambiguity (Ruby bad52, bad53, bad18, + // bad20, bad21, bad22, bad23, bad43). + // ------------------------------------------------------------------------- + + // BAD: ([^X]+)*$ + { std::regex re("([^X]+)*$"); run(re, input); } // $ Alert + // BAD: (([^X]b)+)*$ + { std::regex re("(([^X]b)+)*$"); run(re, input); } // $ Alert + // BAD: (([\S\s]|[^a])*)" + { std::regex re("(([\\S\\s]|[^a])*)\""); run(re, input); } // $ Alert + // BAD: ((.|[^a])*)" + { std::regex re("((.|[^a])*)\""); run(re, input); } // $ Alert + // BAD: ((b|[^a])*)" + { std::regex re("((b|[^a])*)\""); run(re, input); } // $ Alert + // BAD: (([0-9]|[^a])*)" + { std::regex re("(([0-9]|[^a])*)\""); run(re, input); } // $ Alert + // BAD: ^([^>a]+)*(>|$) + { std::regex re("^([^>a]+)*(>|$)"); run(re, input); } // $ Alert + + // ------------------------------------------------------------------------- + // 5. Anchored-vs-unanchored GOOD/BAD pairs (Ruby good16/bad50, + // good17/bad51, good18/bad54, good20..good22/bad55). These are the + // high-value FP-sensitivity cases: an "accept any" tail should make the + // otherwise-vulnerable body safe. + // ------------------------------------------------------------------------- + + // GOOD: (a+)+aaaaa*a+ -- Ruby good16 (no rejecting suffix) + { std::regex re("(a+)+aaaaa*a+"); run(re, input); } + // BAD: (a+)+aaaaa$ -- Ruby bad50 + { std::regex re("(a+)+aaaaa$"); run(re, input); } // $ Alert + + // GOOD: (\n+)+\n\n -- Ruby good17 + { std::regex re("(\\n+)+\\n\\n"); run(re, input); } + // BAD: (\n+)+\n\n$ -- Ruby bad51 + { std::regex re("(\\n+)+\\n\\n$"); run(re, input); } // $ Alert + + // GOOD: (([^X]b)+)*($|[^X]b) -- Ruby good18 + { std::regex re("(([^X]b)+)*($|[^X]b)"); run(re, input); } + // BAD: (([^X]b)+)*($|[^X]c) -- Ruby bad54 + { std::regex re("(([^X]b)+)*($|[^X]c)"); run(re, input); } // $ Alert + + // GOOD: ((ab)+)*ababab -- Ruby good20 + { std::regex re("((ab)+)*ababab"); run(re, input); } + // GOOD: ((ab)+)*abab(ab)*(ab)+ -- Ruby good21 + { std::regex re("((ab)+)*abab(ab)*(ab)+"); run(re, input); } + // GOOD: ((ab)+)* -- Ruby good22 + { std::regex re("((ab)+)*"); run(re, input); } + // BAD: ((ab)+)*$ -- Ruby bad55 + { std::regex re("((ab)+)*$"); run(re, input); } // $ Alert + + // ------------------------------------------------------------------------- + // 6. GOOD patterns that specifically guard against false positives. + // ------------------------------------------------------------------------- + + // GOOD: (.*,)+.+ -- Ruby good2 (no witness at the end) + { std::regex re("(.*,)+.+"); run(re, input); } + // GOOD: (a|.)* -- Ruby good6 + { std::regex re("(a|.)*"); run(re, input); } + // GOOD: (.|\n)*! -- Ruby good7 + { std::regex re("(.|\\n)*!"); run(re, input); } + // GOOD: ([\w.]+)* -- Ruby good8 + { std::regex re("([\\w.]+)*"); run(re, input); } + // GOOD: ([^\"']+)* -- Ruby good10 + { std::regex re("([^\"']+)*"); run(re, input); } + // GOOD: ((a|[^a])*)" -- Ruby good10 (later) + { std::regex re("((a|[^a])*)\""); run(re, input); } + // GOOD: ((\s|\d)*)" -- Ruby good11 + { std::regex re("((\\s|\\d)*)\""); run(re, input); } + // GOOD: (\d+(X\d+)?)+ -- Ruby good12 + { std::regex re("(\\d+(X\\d+)?)+"); run(re, input); } + // GOOD: ^([^>]+)*(>|$) -- Ruby good15 + { std::regex re("^([^>]+)*(>|$)"); run(re, input); } + // GOOD: (A*A*X)* -- Ruby good24 (every witness passes through the accept state) + { std::regex re("(A*A*X)*"); run(re, input); } + // GOOD: ([^\\\]]+)* -- Ruby good26 + { std::regex re("([^\\\\\\]]+)*"); run(re, input); } + // GOOD: (a?)*b -- Ruby good38 + { std::regex re("(a?)*b"); run(re, input); } + // GOOD: a*b -- Ruby good39 (linear) + { std::regex re("a*b"); run(re, input); } + // GOOD: (a|b)+ -- Ruby good40 + { std::regex re("(a|b)+"); run(re, input); } + + // ------------------------------------------------------------------------- + // 7. Cases that are polynomial (not exponential) - must NOT be reported by + // cpp/redos; the polynomial-exclusion heuristic in the shared engine + // should suppress them. + // ------------------------------------------------------------------------- + + // GOOD (for cpp/redos): quadratic trim (a cpp/polynomial-redos alert instead). + { std::regex re("^\\s+|\\s+$"); run(re, input); } + // GOOD (for cpp/redos): quadratic \d+E?\d+. + { std::regex re("^0\\.\\d+E?\\d+$"); run(re, input); } + // GOOD (for cpp/redos): (a|ab)* - actually polynomial, not exponential. + { std::regex re("^(a|ab)*$"); run(re, input); } + + // ------------------------------------------------------------------------- + // 8. Flag / dialect gating. + // ------------------------------------------------------------------------- + + // BAD: exponential regex with icase - case-insensitivity does not suppress the alert. + { std::regex re("^([a-z]+)+$", std::regex_constants::icase); run(re, input); } // $ Alert + // BAD: BRE grammar (`basic`) is modeled by the parser (`BreRegExp`, + // Phase D) and is backtracking-eligible, so the nested-quantifier is + // reported as exponential. Note that `(...)` in BRE is *literal* - the + // `\(...\)` form used in Section 12 below is the actual group syntax. + // Here `(a+)+` parses in BRE as: literal `(`, literal `a`, literal `+`, + // literal `)`, literal `+` - no backtracking risk, so it is *not* + // flagged (the group-and-quantifier structure disappears). + { std::regex re("^(a+)+$", std::regex_constants::basic); run(re, input); } + // BAD: extended selects the ERE grammar (modeled by `EreRegExp` since + // Phase C) and is backtracking-eligible, so the shared engine reports + // the nested-quantifier as exponential - same as the ECMAScript case. + { std::regex re("^(a+)+$", std::regex_constants::extended); run(re, input); } // $ Alert + // GOOD: awk parses as ERE (same as `extended`) but is excluded from the + // ReDoS queries by `isBacktrackingEngine` (POSIX tool-style engines are + // treated as linear-time), *not* by any parsing/grammar difference. + { std::regex re("^(a+)+$", std::regex_constants::awk); run(re, input); } + // GOOD: grep selects BRE (`BreRegExp`, Phase D) - the pattern is + // parsed, but `grep` is excluded from ReDoS analysis by + // `isBacktrackingEngine`. (Regardless of the parse: as noted above, + // `(a+)+` under BRE is all literals anyway.) + { std::regex re("^(a+)+$", std::regex_constants::grep); run(re, input); } + // GOOD: egrep parses as ERE (same as `extended`) but is excluded from + // the ReDoS queries by `isBacktrackingEngine`, *not* by any + // parsing/grammar difference. + { std::regex re("^(a+)+$", std::regex_constants::egrep); run(re, input); } + + // ------------------------------------------------------------------------- + // 9. POSIX bracket sub-expressions (C++ ECMAScript-mode extension). + // + // std::regex under ECMAScript additionally supports POSIX bracket forms + // (`[:name:]`, `[.sym.]`, `[=sym=]`) inside character classes. These + // atoms must flow through the ReDoS engine correctly. + // ------------------------------------------------------------------------- + + // BAD: nested-quantifier exponential regex using a POSIX character class. + { std::regex re("^([[:alpha:]]+)+$"); run(re, input); } // $ Alert + // BAD: same pattern under icase - case-folding must not suppress the alert. + { std::regex re("^([[:alpha:]]+)+$", std::regex_constants::icase); run(re, input); } // $ Alert + // Opaque POSIX character class: `[:punct:]` has no clean `\d`/`\s`/`\w` + // equivalent (see the discussion in RegexTreeView.qll::isEscapeClass). + // It is therefore left as an opaque atom in the shared engine's view; + // this pattern verifies that the parser does not crash on such inputs + // and that (any) alerts remain deterministic. The atom flowing through + // the engine as opaque means the shared ambiguity reasoning will not + // detect exponential behaviour here - a documented limitation of the + // current shared `RegexTreeViewSig`, which has no hook for + // "opaque single-atom character class"; see the acceptance-criteria + // discussion in the accompanying PR. + { std::regex re("^([[:punct:]]+)+$"); run(re, input); } + + // ------------------------------------------------------------------------- + // 10. Non-backtracking POSIX tool-style grammars (`awk`, `grep`, `egrep`) + // combined with other flags via bitwise-OR. + // + // Regression guard for the `isBacktrackingEngine` gate: `awk`, + // `egrep`, and `grep` are all parsed (`awk`/`egrep` as ERE via + // `EreRegExp` in Phase C; `grep` as BRE via `BreRegExp` in Phase D) + // - the same trees the `extended`/`basic` positive cases would be + // analysed with - so suppression here is exclusively due to + // `isBacktrackingEngine`, not any parsing/grammar difference. + // + // The equivalent default-grammar (ECMAScript) patterns in section 2 + // above fire, showing that the gate - not some unrelated exclusion - + // suppresses these cases. + // ------------------------------------------------------------------------- + + // GOOD: awk grammar - non-backtracking. + { std::regex re("^([a-z]+)+$", std::regex_constants::awk); run(re, input); } + // GOOD: grep grammar combined with icase - non-backtracking. + { std::regex re("^([a-z]+)+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::grep | std::regex_constants::icase)); + run(re, input); } + // GOOD: egrep grammar combined with icase - non-backtracking. + { std::regex re("^([a-z]+)+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::egrep | std::regex_constants::icase)); + run(re, input); } + + // ------------------------------------------------------------------------- + // 11. End-to-end ERE-grammar coverage for cpp/redos. + // + // These cases exercise the ERE parser (Phase C, `EreRegExp`) end-to-end + // through the shared backtracking engine. All three flags below + // (`extended`, `egrep`, `awk`) select the *same* grammar (ERE) and + // therefore produce structurally-identical parse trees for the same + // pattern string - the only axis they differ on is + // `isBacktrackingEngine`: + // + // * `extended` -> ERE + backtracking-eligible -> flagged. + // * `egrep` / `awk` -> ERE + non-backtracking -> NOT flagged. + // + // The pattern `^([a-z]+)+$` is chosen because it (a) uses only ERE-legal + // constructs (no `\d`/`\w`/backrefs/lookaround, which are literals in + // ERE) so the ERE parse tree is genuinely equivalent to the ECMAScript + // one, and (b) is a known-exponential nested-quantifier shape (same as + // the ECMAScript case at line 99 above) so we can be confident the + // shared engine reports it. + // ------------------------------------------------------------------------- + + // BAD: ERE grammar via `extended` - parsed as ERE and backtracking-eligible. + { std::regex re("^([a-z]+)+$", std::regex_constants::extended); run(re, input); } // $ Alert + // GOOD: same pattern under `egrep` - parses identically as ERE but + // excluded by `isBacktrackingEngine`. + { std::regex re("^([a-z]+)+$", std::regex_constants::egrep); run(re, input); } + // GOOD: same pattern under `awk` - parses identically as ERE but + // excluded by `isBacktrackingEngine`. + { std::regex re("^([a-z]+)+$", std::regex_constants::awk); run(re, input); } + // GOOD: `egrep | icase` bitwise-OR combination - still ERE-parsed and + // still excluded by `isBacktrackingEngine`; case-folding does not + // re-enable the backtracking-engine gate. + { std::regex re("^([a-z]+)+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::egrep | std::regex_constants::icase)); + run(re, input); } + + // ------------------------------------------------------------------------- + // 12. End-to-end BRE-grammar coverage for cpp/redos. + // + // These cases exercise the BRE parser (Phase D, `BreRegExp`) end-to-end + // through the shared backtracking engine. Both flags below (`basic`, + // `grep`) select the *same* grammar (BRE) and therefore produce + // structurally-identical parse trees for the same pattern string - the + // only axis they differ on is `isBacktrackingEngine`: + // + // * `basic` -> BRE + backtracking-eligible -> flagged. + // * `grep` -> BRE + non-backtracking -> NOT flagged. + // + // BRE inverts the group / metacharacter convention relative to + // ERE/ECMAScript: `\(...\)` are groups (bare `(...)` are literals), + // `\{n,\}` is the interval quantifier (bare `{...}` is literal). The + // pattern below is the BRE spelling of the ECMAScript exponential + // shape `([a-z]+)+`. It uses only BRE-legal constructs (`\(...\)` + // groups, `*` unlimited-repetition quantifier, a character class) so + // its BRE parse tree is structurally equivalent to the ECMAScript one. + // ------------------------------------------------------------------------- + + // BAD: BRE grammar via `basic` - parsed as BRE and backtracking-eligible. + // `\(a*\)*` is a classic nested-quantifier exponential shape under + // BRE: the outer `\(...\)` is the group syntax, and both `*`s are + // quantifiers on ambiguous content (the inner `a*` can consume any + // prefix of a run of `a`s and the outer `*` can partition the run in + // exponentially many ways). BRE has no `+`, so this is the BRE + // analogue of the ECMAScript `^(a*)*$` (or the more familiar + // `^(a+)+$`) shape. + { std::regex re("^\\(a*\\)*$", std::regex_constants::basic); run(re, input); } // $ Alert + // GOOD: same pattern under `grep` - parses identically as BRE but + // excluded by `isBacktrackingEngine`. + { std::regex re("^\\(a*\\)*$", std::regex_constants::grep); run(re, input); } + // GOOD: `grep | icase` bitwise-OR combination - still BRE-parsed and + // still excluded by `isBacktrackingEngine`; case-folding does not + // re-enable the backtracking-engine gate. + { std::regex re("^\\(a*\\)*$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::grep | std::regex_constants::icase)); + run(re, input); } + + return 0; +} diff --git a/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.expected b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.expected new file mode 100644 index 000000000000..ea07654cbb3c --- /dev/null +++ b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.expected @@ -0,0 +1,59 @@ +edges +| test.cpp:107:27:107:30 | **argv | test.cpp:117:28:117:32 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:132:27:132:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:169:27:169:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:179:26:179:30 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:185:27:185:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:192:28:192:32 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:279:27:279:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:308:28:308:32 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:328:28:328:32 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:360:27:360:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:367:27:367:31 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:439:26:439:30 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:502:28:502:32 | *input | provenance | TaintFunction | +| test.cpp:107:27:107:30 | **argv | test.cpp:550:28:550:32 | *input | provenance | TaintFunction | +| test.cpp:387:27:387:57 | *call to getenv | test.cpp:387:27:387:57 | *call to getenv | provenance | | +| test.cpp:387:27:387:57 | *call to getenv | test.cpp:390:28:390:28 | *s | provenance | TaintFunction | +| test.cpp:398:20:398:22 | fread output argument | test.cpp:401:28:401:28 | *s | provenance | TaintFunction | +nodes +| test.cpp:107:27:107:30 | **argv | semmle.label | **argv | +| test.cpp:117:28:117:32 | *input | semmle.label | *input | +| test.cpp:132:27:132:31 | *input | semmle.label | *input | +| test.cpp:169:27:169:31 | *input | semmle.label | *input | +| test.cpp:179:26:179:30 | *input | semmle.label | *input | +| test.cpp:185:27:185:31 | *input | semmle.label | *input | +| test.cpp:192:28:192:32 | *input | semmle.label | *input | +| test.cpp:279:27:279:31 | *input | semmle.label | *input | +| test.cpp:308:28:308:32 | *input | semmle.label | *input | +| test.cpp:328:28:328:32 | *input | semmle.label | *input | +| test.cpp:360:27:360:31 | *input | semmle.label | *input | +| test.cpp:367:27:367:31 | *input | semmle.label | *input | +| test.cpp:387:27:387:57 | *call to getenv | semmle.label | *call to getenv | +| test.cpp:387:27:387:57 | *call to getenv | semmle.label | *call to getenv | +| test.cpp:390:28:390:28 | *s | semmle.label | *s | +| test.cpp:398:20:398:22 | fread output argument | semmle.label | fread output argument | +| test.cpp:401:28:401:28 | *s | semmle.label | *s | +| test.cpp:439:26:439:30 | *input | semmle.label | *input | +| test.cpp:502:28:502:32 | *input | semmle.label | *input | +| test.cpp:550:28:550:32 | *input | semmle.label | *input | +subpaths +#select +| test.cpp:117:28:117:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:117:28:117:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:116:29:116:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:132:27:132:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:132:27:132:31 | *input | This $@ that depends on a $@ may run slow on strings starting with '0.9' and with many repetitions of '99'. | test.cpp:131:33:131:35 | \\d+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:169:27:169:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:169:27:169:31 | *input | This $@ that depends on a $@ may run slow on strings starting with ' ' and with many repetitions of ' '. | test.cpp:168:29:168:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:169:27:169:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:169:27:169:31 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:168:24:168:26 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:179:26:179:30 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:179:26:179:30 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:178:29:178:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:185:27:185:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:185:27:185:31 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:184:29:184:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:192:28:192:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:192:28:192:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:191:29:191:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:279:27:279:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:279:27:279:31 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:278:29:278:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:308:28:308:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:308:28:308:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:307:29:307:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:328:28:328:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:328:28:328:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:327:29:327:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:360:27:360:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:360:27:360:31 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of '9'. | test.cpp:359:25:359:27 | \\d+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:367:27:367:31 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:367:27:367:31 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of 'a'. | test.cpp:366:26:366:27 | .* | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:390:28:390:28 | *s | test.cpp:387:27:387:57 | *call to getenv | test.cpp:390:28:390:28 | *s | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:389:29:389:31 | \\s+ | regular expression | test.cpp:387:27:387:57 | *call to getenv | user-provided value | +| test.cpp:401:28:401:28 | *s | test.cpp:398:20:398:22 | fread output argument | test.cpp:401:28:401:28 | *s | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:400:29:400:31 | \\s+ | regular expression | test.cpp:398:20:398:22 | fread output argument | user-provided value | +| test.cpp:439:26:439:30 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:439:26:439:30 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of 'a'. | test.cpp:438:25:438:26 | a+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:502:28:502:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:502:28:502:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:501:29:501:31 | \\s+ | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:550:28:550:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:550:28:550:32 | *input | This $@ that depends on a $@ may run slow on strings starting with ' ' and with many repetitions of ' '. | test.cpp:548:43:548:59 | [[:space:]]\\{1,\\} | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | +| test.cpp:550:28:550:32 | *input | test.cpp:107:27:107:30 | **argv | test.cpp:550:28:550:32 | *input | This $@ that depends on a $@ may run slow on strings with many repetitions of ' '. | test.cpp:548:24:548:40 | [[:space:]]\\{1,\\} | regular expression | test.cpp:107:27:107:30 | **argv | user-provided value | diff --git a/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.qlref b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.qlref new file mode 100644 index 000000000000..6b9a90569e63 --- /dev/null +++ b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/PolynomialReDoS.qlref @@ -0,0 +1,2 @@ +query: Security/CWE/CWE-1333/PolynomialReDoS.ql +postprocess: utils/test/InlineExpectationsTestQuery.ql diff --git a/cpp/ql/test/query-tests/Security/CWE/CWE-1333/test.cpp b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/test.cpp new file mode 100644 index 000000000000..bd4b7564aa10 --- /dev/null +++ b/cpp/ql/test/query-tests/Security/CWE/CWE-1333/test.cpp @@ -0,0 +1,569 @@ +// Minimal std::regex/std::basic_string stubs, mirroring the ones in +// cpp/ql/test/library-tests/regex/test.cpp. Real headers are not available +// in the extractor sandbox. + +namespace std { + +template +class basic_string { +public: + basic_string() {} + basic_string(const CharT*) {} + const CharT* c_str() const { return 0; } + const CharT* data() const { return 0; } + unsigned long size() const { return 0; } +}; + +typedef basic_string string; + +namespace regex_constants { + enum syntax_option_type { + ECMAScript = 1, basic = 2, extended = 4, awk = 8, grep = 16, egrep = 32, + icase = 256, nosubs = 512, optimize = 1024, collate = 2048, multiline = 4096 + }; + enum match_flag_type { match_default = 0 }; +} + +template +class basic_regex { +public: + typedef regex_constants::syntax_option_type flag_type; + basic_regex(const CharT* p) {} + basic_regex(const CharT* p, flag_type f) {} + basic_regex(const basic_string& p) {} +}; + +typedef basic_regex regex; + +template +bool regex_match(const basic_string& s, const basic_regex& re) { return false; } +template +bool regex_search(const basic_string& s, const basic_regex& re) { return false; } +template +bool regex_search(const CharT* s, const basic_regex& re) { return false; } +template +basic_string regex_replace(const basic_string& s, const basic_regex& re, + const basic_string& fmt) { return basic_string(); } + +// Iterator stubs - just enough for a construction to be recognized as +// "regex used as regex" by RegexFlowConfigs. +template +class regex_iterator { +public: + regex_iterator() {} + regex_iterator(BidirIt first, BidirIt last, const basic_regex& re) {} +}; + +template +class regex_token_iterator { +public: + regex_token_iterator() {} + regex_token_iterator(BidirIt first, BidirIt last, const basic_regex& re) {} +}; + +typedef regex_iterator cregex_iterator; +typedef regex_token_iterator cregex_token_iterator; + +// Stubs for a couple of C standard-library sources modeled by +// `semmle.code.cpp.security.FlowSources` so that we can exercise +// non-`argv` sources in the polynomial-ReDoS test suite. +char* getenv(const char* name); +typedef unsigned long size_t; +struct FILE; +size_t fread(void* ptr, size_t sz, size_t n, FILE* stream); + +} // namespace std + +// ----------------------------------------------------------------------------- +// Length-restricted helper stubs. These deliberately match the name-based +// heuristic in PolynomialReDoSQuery::LengthRestrictedFunction so that values +// returned from them act as barriers. +// ----------------------------------------------------------------------------- + +struct Request { + std::string getHeader(const char* name) const { return std::string(); } + std::string getRequestUri() const { return std::string(); } + std::string getRequestUrl() const { return std::string(); } + std::string getUserAgent() const { return std::string(); } + std::string getPath() const { return std::string(); } // matches "%get%path%" + std::string getUserName() const { return std::string(); } // matches "get%user%" + std::string getQueryString() const { return std::string(); } // matches "%querystring%" +}; + +struct Cookie { + std::string getValue() const { return std::string(); } // Cookie::get% matches +}; + +// A non-restricted source: a plain member function unrelated to headers/cookies. +struct Widget { + std::string readAll() const { return std::string(); } +}; + +// ----------------------------------------------------------------------------- +// Tests +// ----------------------------------------------------------------------------- + +// argv is a LocalFlowSource (and hence a FlowSource). +int main(int argc, char** argv) { // $ Source + std::string input(argv[1]); + + // ------------------------------------------------------------------------- + // 1. Original coverage (kept for regression stability). + // ------------------------------------------------------------------------- + + // BAD: polynomial-backtracking pattern applied to user input. + { + std::regex re("^\\s+|\\s+$"); + std::regex_replace(input, re, std::string("")); // $ Alert + } + + // Note: passing `argv[1]` (a raw `char*`) inline to `regex_search` + // is not currently modeled by the Phase 2 `regexMatchedAgainst` + // predicate (which only tracks values reaching a `std::regex` + // variable), so no alert is expected here. + { + std::regex re("^\\s+|\\s+$"); + std::regex_search(argv[1], re); + } + + // BAD: quadratic \\d+E?\\d+ pattern applied to user input. + { + std::regex re("^0\\.\\d+E?\\d+$"); + std::regex_search(input, re); // $ Alert + } + + // GOOD: pattern is linear (a fixed prefix, no ambiguous repetition). + { + std::regex re("^abc.*$"); + std::regex_search(input, re); + } + + // GOOD: input is length-restricted (a header-like getter). + { + Request req; + std::string h = req.getHeader("X-Foo"); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(h, re, std::string("")); + } + + // GOOD: input is not user-controlled. + { + std::string s("literal"); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 2. Additional superlinear patterns applied to user input (BAD). + // ------------------------------------------------------------------------- + + // BAD: (a|b)*a$ - quadratic on strings of a's followed by a non-match. + { + std::regex re("^(a|b)*a$"); + std::regex_match(input, re); + } + + // BAD: \s+X? repeated - quadratic on strings of whitespace. + { + std::regex re("\\s+X?\\s+$"); + std::regex_search(input, re); // $ Alert + } + + // ------------------------------------------------------------------------- + // 3. Coverage of every modeled match API. + // ------------------------------------------------------------------------- + + // BAD: std::regex_match with user input. + { + std::regex re("^\\s+|\\s+$"); + std::regex_match(input, re); // $ Alert + } + + // BAD: std::regex_search with user input. + { + std::regex re("^\\s+|\\s+$"); + std::regex_search(input, re); // $ Alert + } + + // BAD: std::regex_replace with user input (already covered above; kept for + // symmetry with the other APIs). + { + std::regex re("^\\s+|\\s+$"); + std::regex_replace(input, re, std::string("")); // $ Alert + } + + // BAD: std::regex_iterator constructed with a superlinear regex - the + // literal is recognized as a std::regex pattern by Phase 2, so the + // pattern's terms are analyzed by the polynomial ReDoS query. + // Note: the iterator constructor does not itself take a `std::string` + // subject, so no cpp/polynomial-redos alert is expected here; this case + // exists to confirm the pattern literal is still parsed (a + // cpp/redos alert would fire if the pattern were exponential). + { + std::regex re("^\\s+|\\s+$"); + (void)std::cregex_iterator(input.data(), input.data() + input.size(), re); + } + + // Same for std::regex_token_iterator. + { + std::regex re("^\\s+|\\s+$"); + (void)std::cregex_token_iterator(input.data(), input.data() + input.size(), re); + } + + // ------------------------------------------------------------------------- + // 4. Additional length-restricted / barrier true-negatives (GOOD). + // ------------------------------------------------------------------------- + + // GOOD: request URI is treated as length-restricted. + { + Request req; + std::string s = req.getRequestUri(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: request URL is treated as length-restricted. + { + Request req; + std::string s = req.getRequestUrl(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: user agent is treated as length-restricted. + { + Request req; + std::string s = req.getUserAgent(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: Request::getPath matches the request/path member-getter heuristic. + { + Request req; + std::string s = req.getPath(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: Request::getUserName matches the request/user member-getter heuristic. + { + Request req; + std::string s = req.getUserName(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: query string is treated as length-restricted. + { + Request req; + std::string s = req.getQueryString(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: Cookie::getValue matches the "cookie" class + "get%" prefix heuristic. + { + Cookie c; + std::string s = c.getValue(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // GOOD: integer types are treated as small-fixed-size and act as barriers. + // (No direct string flow, but demonstrates isSmallFixedSizeType.) + { + int n = argc; // integer, small fixed size + (void)n; + std::regex re("^\\s+|\\s+$"); + std::regex_search(input, re); // $ Alert // BAD (already reported above) + } + + // ------------------------------------------------------------------------- + // 5. Non-user-controlled subjects with a superlinear pattern (GOOD). + // ------------------------------------------------------------------------- + + // GOOD: subject is a compile-time literal, not user-controlled. + { + std::string s("static-value"); + std::regex re("^(a|b)*a$"); + std::regex_match(s, re); + } + + // GOOD: subject comes from an ordinary (non-source) function. + { + Widget w; + std::string s = w.readAll(); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 6. Flag / dialect gating. + // ------------------------------------------------------------------------- + + // BAD: icase does not suppress the polynomial alert. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::icase); + std::regex_replace(input, re, std::string("")); // $ Alert + } + + // GOOD: BRE grammar (`basic`) is now modeled by the parser + // (`BreRegExp`, Phase D). Under BRE, the pattern `^\s+|\s+$` parses + // as: `^` anchor, literal `\s`, literal `+`, literal `|`, literal + // `\s`, literal `+`, `$` anchor - there is no `+` quantifier in BRE + // (bare `+` is a literal) and no `|` alternation, so the polynomial + // shape disappears and the pattern is not flagged. (See Section 12 + // below for the proper BRE spelling with `\{1,\}` intervals.) + { + std::regex re("^\\s+|\\s+$", std::regex_constants::basic); + std::regex_replace(input, re, std::string("")); + } + // BAD: extended selects the ERE grammar (modeled by `EreRegExp` since + // Phase C) and is backtracking-eligible, so the polynomial engine + // flags the superlinear pattern on user input - same as the + // ECMAScript case. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::extended); + std::regex_replace(input, re, std::string("")); // $ Alert + } + // GOOD: awk parses as ERE (same as `extended`) but is excluded from + // the ReDoS queries by `isBacktrackingEngine`, *not* by any + // parsing/grammar difference. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::awk); + std::regex_replace(input, re, std::string("")); + } + // GOOD: grep selects BRE (`BreRegExp`, Phase D) - the pattern is + // parsed, but `grep` is excluded from the polynomial-ReDoS query by + // `isBacktrackingEngine`. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::grep); + std::regex_replace(input, re, std::string("")); + } + // GOOD: egrep parses as ERE (same as `extended`) but is excluded from + // the ReDoS queries by `isBacktrackingEngine`, *not* by any + // parsing/grammar difference. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::egrep); + std::regex_replace(input, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 7. Additional superlinear patterns (ported from Java/JS polynomial suites). + // ------------------------------------------------------------------------- + + // BAD: (\d+)*$ - the outer `*` allows quadratic backtracking on non-matching + // digit-heavy input (Java polynomial-ReDoS test `Test.java`). + { + std::regex re("(\\d+)*$"); + std::regex_search(input, re); // $ Alert + } + + // BAD: .*.*=.* - classic polynomial pattern, cf. JS + // `polynomial-redos/tst.js`. + { + std::regex re(".*.*=.*"); + std::regex_search(input, re); // $ Alert + } + + // GOOD (observed): `^(\w+\s?)*$` - a "trim-and-split" style pattern that + // the shared engine's polynomial analysis does not currently flag as + // super-linear (the optional `\s?` inside the outer `*` does not + // produce a polynomial-backtracking pivot term). The Java and + // JavaScript polynomial-ReDoS queries behave the same way on this + // shape; documented here as a negative case rather than divergence. + { + std::regex re("^(\\w+\\s?)*$"); + std::regex_match(input, re); + } + + // ------------------------------------------------------------------------- + // 8. Source variety - non-`argv` C++ threat-model sources. + // ------------------------------------------------------------------------- + + // BAD: `std::getenv` is modeled as a LocalFlowSource. + { + const char* env = std::getenv("USER_REGEX_INPUT"); // $ Source + std::string s(env); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); // $ Alert + } + + // BAD: `std::fread` is modeled as a RemoteFlowSource (bytes read from a + // stream / file). This exercises the RemoteFlowSource half of the + // `FlowSource` union used by `PolynomialRedosConfig::isSource`. + { + char buf[256]; + std::fread(buf, 1, sizeof(buf), (std::FILE*)0); // $ Source + std::string s(buf); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(s, re, std::string("")); // $ Alert + } + + // ------------------------------------------------------------------------- + // 9. Sanitized / length-checked user input (GOOD). + // + // The polynomial ReDoS query treats calls to `LengthRestrictedFunction`s + // as barriers, so user input that has been laundered through such a + // getter (here: `Request::getHeader`, which is a header-style getter) + // should not produce an alert even though the pattern is superlinear + // and the *original* value was user-controlled. + // ------------------------------------------------------------------------- + + // GOOD: `argv[1]` is user-controlled but is passed through a + // header-style barrier before reaching the regex match. + { + Request req; + std::string sanitized = req.getHeader(argv[1]); + std::regex re("^\\s+|\\s+$"); + std::regex_replace(sanitized, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 10. Overlap sanity: an exponential pattern on user input. + // + // The shared engine splits reporting between `cpp/redos` (exponential) + // and `cpp/polynomial-redos` (polynomial). Many "exponential" nested- + // quantifier patterns also contain a polynomial-backtracking sub-term, + // so BOTH queries fire - matching the behaviour observed in the Java + // and JavaScript suites. `(a+)+b` is such a pattern: `cpp/redos` + // reports the outer nested quantifier, and `cpp/polynomial-redos` + // reports the inner `a+` term. + // ------------------------------------------------------------------------- + + // BAD (also reported by cpp/redos): the inner `a+` is a polynomial + // backtracking term on user input. + { + std::regex re("(a+)+b"); + std::regex_match(input, re); // $ Alert + } + + // ------------------------------------------------------------------------- + // 11. Non-backtracking POSIX tool-style grammars (`awk`, `grep`, `egrep`) + // combined with other flags via bitwise-OR. + // + // Regression guard for the `isBacktrackingEngine` gate: `awk`, + // `egrep`, and `grep` are all parsed (`awk`/`egrep` as ERE via + // `EreRegExp` in Phase C; `grep` as BRE via `BreRegExp` in Phase D) + // - the same trees the `extended`/`basic` positive cases would be + // analysed with - so suppression here is exclusively due to + // `isBacktrackingEngine`, not any parsing/grammar difference. + // + // The equivalent default-grammar (ECMAScript) patterns above fire, + // showing that the gate - not some unrelated exclusion - suppresses + // these cases. + // ------------------------------------------------------------------------- + + // GOOD: awk grammar - non-backtracking. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::awk); + std::regex_replace(input, re, std::string("")); + } + // GOOD: grep grammar combined with icase - non-backtracking. + { + std::regex re("^\\s+|\\s+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::grep | std::regex_constants::icase)); + std::regex_replace(input, re, std::string("")); + } + // GOOD: egrep grammar combined with icase - non-backtracking. + { + std::regex re("^\\s+|\\s+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::egrep | std::regex_constants::icase)); + std::regex_replace(input, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 12. End-to-end ERE-grammar coverage for cpp/polynomial-redos. + // + // These cases exercise the ERE parser (Phase C, `EreRegExp`) end-to-end + // through the shared backtracking engine with a user-controlled + // subject. All three flags below (`extended`, `egrep`, `awk`) select + // the *same* grammar (ERE) and therefore produce structurally- + // identical parse trees for the same pattern string - the only axis + // they differ on is `isBacktrackingEngine`: + // + // * `extended` -> ERE + backtracking-eligible -> source->sink alert. + // * `egrep` / `awk` -> ERE + non-backtracking -> NO alert. + // + // The pattern `^\s+|\s+$` (the same superlinear trim pattern used by + // the existing polynomial cases above) uses only ERE-legal constructs + // (`\s` is an ERE-legal escape), so the ERE parse tree is + // structurally equivalent to the ECMAScript one. The taint source + // (`argv[1]` -> `input`) and the match harness (`std::regex_replace`) + // are reused verbatim so that only the grammar flag differs. + // ------------------------------------------------------------------------- + + // BAD: ERE grammar via `extended` - parsed as ERE and backtracking-eligible. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::extended); + std::regex_replace(input, re, std::string("")); // $ Alert + } + // GOOD: same pattern/flow under `egrep` - parses identically as ERE + // but excluded by `isBacktrackingEngine`. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::egrep); + std::regex_replace(input, re, std::string("")); + } + // GOOD: same pattern/flow under `awk` - parses identically as ERE but + // excluded by `isBacktrackingEngine`. + { + std::regex re("^\\s+|\\s+$", std::regex_constants::awk); + std::regex_replace(input, re, std::string("")); + } + // GOOD: `egrep | icase` bitwise-OR combination - still ERE-parsed and + // still excluded by `isBacktrackingEngine`. + { + std::regex re("^\\s+|\\s+$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::egrep | std::regex_constants::icase)); + std::regex_replace(input, re, std::string("")); + } + + // ------------------------------------------------------------------------- + // 13. End-to-end BRE-grammar coverage for cpp/polynomial-redos. + // + // These cases exercise the BRE parser (Phase D, `BreRegExp`) end-to-end + // through the shared backtracking engine with a user-controlled + // subject. Both flags below (`basic`, `grep`) select the *same* grammar + // (BRE) and therefore produce structurally-identical parse trees for + // the same pattern string - the only axis they differ on is + // `isBacktrackingEngine`: + // + // * `basic` -> BRE + backtracking-eligible -> source->sink alert. + // * `grep` -> BRE + non-backtracking -> NO alert. + // + // The pattern must be expressed in BRE's inverted metacharacter + // convention: `\{1,\}` is the "one-or-more" interval quantifier (BRE + // has no `+`) and there is no `|` alternation, so the ECMAScript + // `\s+.*\s+` polynomial shape is spelled here as + // `[[:space:]]\{1,\}.*[[:space:]]\{1,\}` - a superlinear pattern + // matched against the tainted subject. + // ------------------------------------------------------------------------- + + // BAD: BRE grammar via `basic` - parsed as BRE and backtracking-eligible. + { + std::regex re("[[:space:]]\\{1,\\}.*[[:space:]]\\{1,\\}$", + std::regex_constants::basic); + std::regex_replace(input, re, std::string("")); // $ Alert + } + // GOOD: same pattern/flow under `grep` - parses identically as BRE + // but excluded by `isBacktrackingEngine`. + { + std::regex re("[[:space:]]\\{1,\\}.*[[:space:]]\\{1,\\}$", + std::regex_constants::grep); + std::regex_replace(input, re, std::string("")); + } + // GOOD: `grep | icase` bitwise-OR combination - still BRE-parsed and + // still excluded by `isBacktrackingEngine`. + { + std::regex re("[[:space:]]\\{1,\\}.*[[:space:]]\\{1,\\}$", + (std::regex_constants::syntax_option_type) + (std::regex_constants::grep | std::regex_constants::icase)); + std::regex_replace(input, re, std::string("")); + } + + return 0; +}