[p5.js 2.0+ Bug Report]: splitTokens() returns [] for a caret delimiter and throws for a backslash

[rajanpanth]

Most appropriate sub-area of p5.js?

Utilities

p5.js version

2.x main (4b096e2)

Actual vs expected behavior

splitTokens builds a regex character class from the delimiter characters with only [ and ] special-cased (the 2015 fix for #1004), so other regex metacharacters corrupt the class:

splitTokens('a^b', '^');   // actual []                       expected ['a', 'b']
                           // the ^ negates the class, so every character becomes a delimiter
splitTokens('a\b', '\'); // actual SyntaxError: Invalid regular expression: /[\]/g
                           // expected ['a', 'b']

A - between delimiter characters also silently forms a range instead of matching the literal characters.

Steps to reproduce

Outputs above are from executed runs against current main.

Note

I am aware splitTokens was deprecated in 2.0 (#7624), so feel free to close this as wontfix if the function is not receiving fixes anymore. If it is, I have a small fix ready (escape \, ], ^, - when building the class) with unit tests for both broken cases plus regression coverage for the existing bracket, array-delimiter and whitespace behaviors; the two new tests fail on current main and pass with the change. Will open the PR once approved.

[aayushi-tomar]

Hi! I'd like to work on this issue as a first contribution.
Since splitTokens() was deprecated in p5.js 2.0, is a fix for this issue still welcome?