Skip to content

fix(lsp): share the line index with WASM and keep positions out of CRLF line breaks - #1070

Merged
psteinroe merged 10 commits into
mainfrom
refactor/wasm-line-index
Nov 6, 2026
Merged

psteinroe merged 10 commits into
mainfrom
refactor/wasm-line-index

Conversation

@psteinroe

@psteinroe psteinroe commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #1063.

The WASM bindings converted LSP positions with their own functions, which #969 fixed for CRLF files. Now they use the shared pgls_line_index::LineIndex, like the native language server, and the index takes over the rules #969 introduced. This changes behaviour in the native language server for positions next to a \r\n:

select 1;\r\n
select 2;
-- (0, 100), a column past the end of line 0:
--   before: offset 10, between the \r and the \n; now: offset 9, before the \r
-- offset 10, between the \r and the \n:
--   before: (0, 10), a column after the \r; now: (0, 9), the end of the line

The LSP says a position can't point between \r and \n, and that a column past the end of a line means the end of the line, which doesn't include its line break.

What the index does now:

  • The end of a line is the start of its line break, so before the \r of a \r\n. It records the offsets of the \ns that follow a \r, so LF files pay nothing extra.
  • line_col turns an offset between \r and \n into the end of its line, and an offset past the end of the text into the end of the text.
  • offset_clamped turns a column inside a character into the start of that character. Before, the native language server returned an offset inside the character, and the WASM bindings moved to the start of the next character. The start matches what fix(lsp): debug assertions at range boundaries, keep client positions on char boundaries #952 does for the native language server, so select '😀'; at UTF-16 column 9 (inside the surrogate pair) is offset 8 in both now. That's the only change for the WASM bindings.
  • A lone \r still doesn't end a line, in both.

The WASM bindings keep a LineIndex per document, built on didOpen and didChange, like the native Document, instead of scanning the text on every conversion. changed_range still widens a fix-all edit that changes only one half of a \r\n, since that's about the edit, not the conversion.

Tests: line_breaks_and_wide_characters in pgls_line_index now pins the new behaviour for \r\n (end of line, offset between \r and \n), a lone \r, an offset past the end, and columns inside 2-byte and 4-byte characters in UTF-8 and UTF-16. crlf_belongs_to_the_line_break checks the same through the native from_lsp/to_lsp and fails without this change. The WASM conversion tests from #969 run against the shared index unchanged, apart from the surrogate pair case above, plus a new lone \r test. cargo test -p pgls_line_index -p pgls_lsp -p pgls_wasm, cargo clippy --workspace --all-targets -- -D warnings, and the WASM package's bun run build && bun test (49 pass).

Not covered: the native to_lsp::text_edits doesn't widen edits that change only one half of a \r\n the way the WASM fix-all does, so such an edit (e.g. \r\n to \n) becomes an empty range. Before, it ended at a column after the \r, which editors clamp to the same empty range.

Update: merged main, which brought #952's debug assertions. to_lsp::position now asserts that a position maps back to the offset of its line and column, so an offset between the \r and the \n (mapped to the end of the line) passes. LineIndex::line_col keeps #952's assertion that the offset is inside the text, so the clamp of an offset past the end only applies in release builds, and the line_col(100) case is out of the pgls_line_index test.

The language server's LineIndex converts offsets to line/column positions
in UTF-8, UTF-16 and UTF-32. It lives in pgls_lsp, which pulls in tokio and
tower-lsp, so the WASM bindings and pgls_suppressions couldn't use it and
have their own line handling. Move it, with LineCol, WideLineCol,
WideEncoding and WideChar, into a small crate that only depends on
pgls_text_size and rustc-hash. No behaviour change.
The WASM bindings had their own position conversion, which treats the \r of
a \r\n as part of the line break (#969), while the language server's index
clamped a column past the end of a CRLF line to the \n and reported an
offset between \r and \n as a column after the \r. Neither is a position the
LSP allows.

The shared LineIndex now follows the WASM rules: the end of a line is before
its \r\n, an offset between \r and \n is the end of its line, and an offset
past the end of the text is the end of the text. A column inside a
character is the start of the character. The WASM bindings keep a LineIndex
per document and convert positions with it.
Keep the index in line_index.rs, nearly unchanged, with the position types
in lib.rs and the tests in tests.rs, so that git still follows the file
from pgls_lsp when merging branches that change it.
…/wasm-line-index

# Conflicts:
#	crates/pgls_line_index/src/lib.rs
@psteinroe
psteinroe added this pull request to stack #1100 November 1, 2026 09:17
Base automatically changed from refactor/line-index-crate to main November 2, 2026 12:09
psteinroe added a commit that referenced this pull request Nov 2, 2026
…res reports (#1107)

Postgres reports the position of an error as a 1-based character
position
([`parser_errposition`](https://github.com/postgres/postgres/blob/REL_18_6/src/backend/parser/parse_node.c#L106)),
one past the last character for an error at the end of the input. Three
places converted it to a byte offset on their own: typecheck
(`byte_offset_of_position`, #1050), plpgsql_check
(`byte_offset_from_one_based_char_offset`) and the statement execution
code action (`rejected`). They now call
`pgls_line_index::offset_of_char_position`, next to the other position
conversions.

There's no behaviour change: the three copies did the same thing. 0 and
positions past the end of the text give no offset, and the position
after the last character is the end of the text. plpgsql_check computed
the offset as a `usize` and now gets a `TextSize`, like the other two,
which only differs for texts over 4 GiB. Its `OneBasedCharOffset`
newtype is gone with the function.

AGENTS.md and ARCHITECTURE.md now say that positions are converted with
`pgls_line_index`, not by hand.

Tests: the typecheck test `error_positions_are_character_positions`
moved to `pgls_line_index` as `character_positions_are_one_based`, with
an empty text and position 1 added. The plpgsql_check tests for query
and compile error positions with non-ASCII text run against the shared
function unchanged. `cargo test -p pgls_line_index -p pgls_typecheck -p
pgls_plpgsql_check -p pgls_workspace`, `cargo clippy --workspace
--all-targets -- -D warnings`.

Not covered: the other position helpers stay where they are for now. The
LSP position conversions in the WASM bindings move onto the line index
in #1070, and the remaining copies (fix-all edits in the language server
and WASM, the CLI's line numbers, the regression fixtures' 1-based
character columns) are follow-ups.
@psteinroe psteinroe added the ready label Nov 6, 2026
# Conflicts:
#	ARCHITECTURE.md
#	crates/pgls_line_index/src/lib.rs
#	crates/pgls_line_index/src/line_index.rs
#	crates/pgls_lsp/src/adapters/mod.rs
@psteinroe
psteinroe merged commit ba6f0fa into main Nov 6, 2026
12 checks passed
@psteinroe
psteinroe deleted the refactor/wasm-line-index branch November 6, 2026 19:59
psteinroe added a commit that referenced this pull request Nov 6, 2026
Stacked on #1070.

The fix-all code action replaces only the part of the file that its
fixes changed. The language server and the WASM bindings each had their
own `changed_range` for that, and only the WASM one (#969) took in the
whole `\r\n` when only one half of it changed. In the language server,
such an edit ended between the `\r` and the `\n`, a position the LSP
doesn't allow, so it lost the half that changed:

```sql
-- the fixes turn "select 1;\r\nselect 2;" into "select 1;\nselect 2;"
-- before: the edit is empty at (0, 9), the end of line 0, and the \r stays
-- now: the edit replaces the \r\n at (0, 9)-(1, 0) with \n
```

`changed_range` now lives in `pgls_line_index`, next to the rule that a
position can't point into a `\r\n`, and both use it. The WASM bindings
keep their behaviour; the language server gets the widening. Each caller
still turns the range into an LSP edit in its own position types
(`fix_all_edit`).

Tests: `fix_all_edits_keep_crlf_whole` in `pgls_lsp` applies the fix-all
edit through `from_lsp` for CRLF and LF changes and a non-ASCII line; it
fails without this change.
`changed_range_keeps_the_common_start_and_end` (moved from `pgls_lsp`)
and `changed_range_keeps_crlf_whole` test the function in
`pgls_line_index`; the WASM test from #969 runs against it unchanged.
`cargo test -p pgls_line_index -p pgls_lsp -p pgls_wasm`, `cargo clippy
--workspace --all-targets -- -D warnings`, and `bun run build && bun
test` in the WASM package (49 pass).

Not covered: `to_lsp::text_edits`, which formatting uses, still doesn't
widen a diff that changes one half of a `\r\n` (see #1070). #1107 also
adds a module to `pgls_line_index`; whichever merges second gets a small
conflict in the module list of `lib.rs`, where both lines stay.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant