[psteinroe]

Stacked on #993.

goose's -- +goose ENVSUB ON replaces ${NAME} with environment variables before it runs the SQL (parser.go, using interpolate). We parsed the expansions as written, so every statement with one was a syntax error and got no linting:

-- +goose Up
-- +goose ENVSUB ON
create table ${SCHEMA}.events (id bigint, at timestamp);   -- before: syntax error at or near "$"

Behaviour

Real substitution doesn't fit a linter: the environment of the editor or CI isn't the one goose runs in, and a missing variable becomes an empty string. Instead, between ENVSUB ON and ENVSUB OFF, each braced expansion (${NAME}, ${NAME:-default}, ${NAME:?}, … with nested braces) is read as the named parameter $NAME, padded with spaces to the same length:

create table ${SCHEMA}.events (...)   ->   create table $SCHEMA  .events (...)

Named parameters already get the psql-variable treatment (:name, @name, $name): they are identifiers where an object name is expected and value parameters elsewhere, and statements with identifier parameters aren't typechecked. The unbraced $NAME that interpolate also expands is a named parameter already. The escapes \$, $$ and \\ are left as written, as is $(. The text keeps its length, so diagnostics, fixes and LSP positions need no offset mapping.

• Document keeps the content as written (get_document_content), and splits and parses the substituted text.
• Fixes with an edit inside an expansion are dropped in code actions and in fix_file. Without that, constraintMissingNotValid put NOT VALID inside ${PARENT}, once per pass, because the end of the constraint was the padded placeholder. There's a test for that. Fixes elsewhere in the statement still apply, and the expansion stays as written.
• The formatter leaves statements with expansions as they are, so it can't write the placeholder back.
• It applies to any file with the annotation, also without sectionMarkers: "goose". The annotation is specific enough.

To avoid new syntax errors, I had to extend the identifier heuristic of the named-parameter conversion in pgls_lexer: parameters after INTO, JOIN, ON, UPDATE, TRUNCATE, REFERENCES, EXISTS, VIEW, SEQUENCE, EXTENSION, FUNCTION and TRIGGER are now identifiers. references ${PARENT} was converted to a value parameter, which doesn't parse. This also helps psql variables, e.g. insert into :items.

Tests

Unit tests for the substitution (ON/OFF regions, nested defaults, escapes, offsets, no annotation). Workspace tests: no syntax error between ON and OFF but one after OFF, with the rules still running; fix_file keeping ${SCHEMA} while fixing around it; no fix inside an expansion; the formatter leaving the statement unchanged. A lexer test covers the new identifier contexts. The migrations guide describes the behaviour.

Not covered

$$ inside an ENVSUB region becomes $ for goose, so dollar-quoted bodies break there in goose itself. We still read them as dollar quotes.

Update: Stacked on #993, which moves the migration tools into one adapter each (pgls_workspace::migrations) and lets the document take its tool from the settings. This PR now builds on that:

• The substitution is MigrationTool::preprocess of the goose adapter (migrations/goose.rs), returning the text with placeholders and their ranges (Preprocessed). Other tools don't preprocess.
• It only applies with "sectionMarkers": "goose", no longer to any file with the annotation. The document doesn't run it unconditionally anymore: it preprocesses with the tool from the settings of its file when it's opened or edited, and again when the settings change. fix_file keeps the tool for its intermediate documents.
• Since fix(analyser): only offer fixes that keep the SQL valid #906 on main, a fix is only offered if the fixed statement parses as written, which a statement with an expansion doesn't. So statements with an expansion now get no fixes at all, rather than fixes around the expansion; test_goose_envsub_expansions_stay_as_written and the guide say so. The touches_placeholder filter stays as a guard.
• New tests: test_goose_envsub_needs_the_goose_section_markers (no substitution with dbmate or without markers) and test_goose_envsub_follows_settings_changes (switching to goose re-splits the open document).

Update (review follow-up):

• Executing a statement with an expansion is refused, and its execute action is disabled with the reason. The action sent the analysed text, so values ('${MESSAGE}') inserted the placeholder. Sending the text as written instead wouldn't help: it isn't what goose runs either.
• The expansions are found with a port of interpolate's parser (migrations/goose/interpolate.rs). A { in a default is text (${SCHEMA:-app{test}), the unbraced $NAME is an expansion too, and a line that interpolate rejects, like one with ${NAME:?}, stays as written, since goose fails on it.
• The placeholder replaces the whole identifier that an expansion is part of: ${PREFIX}_events becomes $PREFIX_events__ and "${SCHEMA}" becomes $SCHEMA____, both identifier parameters. It's padded with _ rather than spaces, so the token has room for its index (the tenth expansion panicked). Expansions in string literals and comments stay as written, but still count as expansions for fixes, formatting and execution.
• pgls_lexer: ON only makes the next named parameter an identifier if it names an object (create index ... on :rel, grant ... on :t, create trigger ... on :t). After a JOIN, or in a MERGE, it's the condition, so join customers on :enabled stays a value parameter and is typechecked. A value parameter whose index doesn't fit in its text, like a tenth :a, is now read as an identifier instead of panicking.

Not covered: an expansion in a string literal is analysed as written, so '${N}'::int is still reported as an invalid integer, as before this PR. Fixing that needs a way to skip the typecheck of a statement without tainting the catalog.

Update (redesign): Merged #993's redesign and main. The substitution moved with goose into pgls_migrations: Goose::read returns it as Script::preprocessed (templating::Preprocessed, with the expansion parsing in templating/{envsub,interpolate}.rs), and the document splits and parses its text as before. The guards moved with main's code action split: lint fixes and fix_file in features/code_actions/lint.rs, the disabled execute action and PLACEHOLDER_EXECUTION_DISABLED in features/code_actions/execute_statement.rs, the formatter in workspace/server/formatting.rs.

Behaviour change from main's #1037 (fixes are checked with parameters converted): statements with an expansion get fixes again, as long as no edit touches an expansion, e.g. create table if not exists ${SCHEMA}.events and references ${PARENT} not valid. test_goose_envsub_expansions_stay_as_written, test_no_fix_inside_a_goose_envsub_expansion and the guide say so.

[psteinroe]

at this point, we should properly structure the different adapters and cleanup their integration

[edjubert]

Reading the preprocessed text here also changes the content returned by ExecuteStatementMapper, which still returns id.content(). With Goose section markers enabled, this statement:

-- +goose Up
-- +goose ENVSUB ON
insert into audit_log(message) values ('${MESSAGE}');

is returned to the execution path as insert into audit_log(message) values ('$MESSAGE ');. It parses successfully, and execute_statement forwards that content directly to PostgreSQL once execution is enabled for the connection. This can therefore write synthetic analysis text into the database.
We should disable execution for statements containing expansions, or keep the original source separate from the analysis text so the execution path cannot send placeholders.

[edjubert]

The padding preserves the expansion's total length, but the lexer only sees the two-byte named parameter token in $A . This valid Goose input panics during analysis:

-- +goose Up
-- +goose ENVSUB ON
select ${A}, ${B}, ${C}, ${D}, ${E}, ${F}, ${G}, ${H}, ${I}, ${J};

The tenth distinct parameter becomes $10, which cannot fit in the two-byte $J token, triggering named parameter replacement must preserve source length in push_padded_replacement. The original ${J} span has enough space, but the converter cannot use its trailing padding. This exposes an existing named-parameter width limitation through the new preprocessing path

[edjubert]

ON also introduces a join condition, so it cannot always classify the next parameter as an identifier:

select * from orders join customers on :enabled;

This now becomes ... on enabled instead of ... on $1, and has_identifier_parameters is set to true. That flag disables typechecking for the entire statement. With an empty catalog, I reproduced the loss of both unknownRelation diagnostics that the value-parameter interpretation produces. This affects existing named-parameter SQL even without Goose.
We should make this classification depend on the surrounding SQL construct, keeping the identifier behavior for cases such as CREATE INDEX ... ON <relation> while preserving value parameters in join conditions.

[edjubert]

With Goose section markers enabled, a quoted expansion is substituted inside the quotes:

-- +goose Up
-- +goose ENVSUB ON
select * from "${SCHEMA}".events;

The analysis text becomes select * from "$SCHEMA ".events;. The lexer treats "$SCHEMA " as an ordinary quoted identifier rather than a named parameter, so has_identifier_parameters remains false. With a catalog available, this produces a false unknownSchema diagnostic for the literal name $SCHEMA . Goose's interpolation library accepts this input and produces "public".events when SCHEMA=public.

We should carry the placeholder-range information into identifier classification so quoted expansions also suppress checks that depend on their unknown names.

[edjubert]

Inserting whitespace immediately after the placeholder splits a valid interpolated identifier:

-- +goose Up
-- +goose ENVSUB ON
create table ${PREFIX}_events (id bigint);

This becomes create table $PREFIX _events ..., leaving two separate identifiers after parameter conversion. The statement still produces a syntax error and receives no linting. Goose's interpolation library accepts it and produces create table app_events (id bigint); when PREFIX=app.

Maybe we could represent the complete composite identifier as a placeholder, placing any padding after the complete token rather than between the expansion and its suffix.

[edjubert]

interpolate treats a bare { in a default expression as literal text; only another expansion introduces recursive parsing.
For example, this input is accepted by Goose's interpolation library:

-- +goose Up
-- +goose ENVSUB ON
create schema ${SCHEMA:-app{test};

With SCHEMA=public, it produces create schema public;. This scan instead increments the depth for the literal {, fails to find the expansion's end, and leaves ${SCHEMA:-app{test} unchanged, producing a SQL syntax error.
I think this scan should follow interpolate's expression grammar when finding the closing brace, distinguishing literal braces from nested ${...} expansions.