change(web): generalize determineSuggestionRange 🚂 🔪

[jahorton]

The changes of this PR facilitate using the same suggestion-application-range logic for all model types, not just the first-class ones that implement LexiconTraversals. This way, we may standardize the logic used when constructing multi-token suggestions.

Build-bot: skip build:web
Test-bot: skip

[keymanapp-test-bot]

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

• Web
  • KeymanWeb Test Home - build : all tests passed (no artifacts on BuildLevel "build")

[jahorton]

#15907 has some changes to this method that should be moved up to this PR.

[jahorton]

From devin.ai:

web/src/engine/predictive-text/worker-thread/src/main/predict-helpers.ts:R450-456

deleteLeftCalc returns incorrect value when predictCount > 1

The deleteLeftCalc helper at web/src/engine/predictive-text/worker-thread/src/main/predict-helpers.ts:450-456 only returns the last token's codepointLength when predictCount > 1, rather than summing all tokens to remove. This is acknowledged with a TODO comment but means the deleteLeft field in SuggestionReplacement will be incorrect for multi-token prediction scenarios (e.g., the 'large variation in intermediate tokens' test would compute deleteLeft = 3 for 'dog' instead of the expected sum of 17 for 'over the lazy dog'). Since this function isn't called in production yet and the test assertions don't validate deleteLeft, this is currently dormant but will need to be fixed before activating multi-tokenization.

This is done this way to preserve existing unit test functionality; multi-token corrections are not put in place at this stage. It should be fully implemented in a later PR, though exactly which PR has yet to be fully determined, with #15897 #16072 appearing to be a good candidate.