Skip to content

Semantic tokens: 'end character > model.getLineLength(lineNumber)' fired during typing #30

Description

@char0n

Where it shows up

While editing an OpenAPI YAML/JSON document the console fires:

  main.workbench-<hash>.js:17 Invalid Semantic Tokens Data From Extension:
    end character > model.getLineLength(lineNumber)

What VSCode is complaining about

A registered DocumentSemanticTokensProvider returned a token whose
(line, startChar, length) triple has startChar + length greater than the
actual line length in the model. VSCode drops the offending token and continues,
so highlighting may briefly flicker but is otherwise unaffected.

Likely cause

The provider is computing tokens against a slightly older document revision
than the model currently has (typical race: user deletes characters at end of
line between provider snapshot and result return). Less likely: byte vs.
UTF-16-code-unit offset mismatch.

Suggested fix

In the semantic tokens provider:

  1. Snapshot document.version at the start of computation.
  2. Before returning, drop the result if document.version has changed.
  3. Clamp startChar + length to document.lineAt(line).text.length as a
    belt-and-braces guard against off-by-one bugs.

VSCode's model uses UTF-16 code units, so position math must match.

Repro

Open https://github.com/speclynx/speclynx-editor, load petstore.yaml, and
type/delete a few characters at the end of any line — the warning fires within
a beat or two.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions