Cross-package references silently dropped in src-layout projects (autoSearchPaths is never enabled)

[dreddnafious]

Summary

scip-python never sets autoSearchPaths on the CommandLineOptions it builds, so pyright's
built-in src-layout detection is disabled. Every reference from outside src/ into a src-layout
package is silently dropped from the index — definitions are filed under src.pkg.mod while
imports resolve to pkg.mod, and the two namespaces never join.

pyright's own CLI enables this unconditionally, with an explicit comment saying so, so
scip-python and pyright disagree about the same project.

The index is still produced, exit code is 0, and nothing in the output says references were lost.

Reproduction

demo/
├── pyproject.toml          # [project] name = "demo", version = "0.1.0"  (no [tool.pyright])
├── src/demo/__init__.py
├── src/demo/core.py        # def handler() -> int: return 1
└── tests/test_core.py      # from demo.core import handler

scip-python index --output index.scip --project-name demo
scip print --json index.scip

Observed — the reference to handler is absent entirely:

src/demo/core.py
    def: scip-python python demo 0.1.0 `src.demo.core`/handler().
tests/test_core.py
    ref: scip-python python demo 0.1.0 `demo.core`/__init__:      ← module ref only
                                                                   ← no handler ref at all

Expected (and what you get after adding [tool.pyright] extraPaths = ["src"]):

src/demo/core.py
    def: scip-python python demo 0.1.0 `demo.core`/handler().
tests/test_core.py
    ref: scip-python python demo 0.1.0 `demo.core`/handler().
    ref: scip-python python demo 0.1.0 `demo.core`/handler().

pyright disagrees with scip-python on this exact tree. Same directory, no [tool.pyright]:

$ npx pyright@latest --outputjson
{ "summary": { "filesAnalyzed": 3, "errorCount": 0, ... } }

pyright resolves from demo.core import handler with zero configuration. scip-python does not.

Root cause

packages/pyright-scip/src/config.ts:

48:  const options = new CommandLineOptions(process.cwd(), false);
      // ... autoSearchPaths is never assigned

138:  configOptions.ensureDefaultExtraPaths(
139:      this.fs,
140:      commandLineOptions.autoSearchPaths || false,   // always false
141:      commandLineOptions.extraPaths
142:  );

CommandLineOptions.autoSearchPaths is declared boolean | undefined with no default
(pyright-internal/src/common/commandLineOptions.ts), so || false always wins.
autoSearchPaths appears exactly once in the whole of pyright-scip — that read, never a write.

The disabled code is precisely the src-layout handler
(pyright-internal/src/common/configOptions.ts):

ensureDefaultExtraPaths(fs, autoSearchPaths, extraPaths) {
    if (autoSearchPaths) {
        // Auto-detect the common scenario where the sources are under the src folder
        const srcPath = resolvePaths(this.projectRoot, pathConsts.src);
        if (fs.existsSync(srcPath) && !fs.existsSync(resolvePaths(srcPath, '__init__.py'))) {
            paths.push(srcPath);
        }
    }
    ...

pyright's CLI (pyright-internal/src/pyright.ts) sets it unconditionally:

// Always enable autoSearchPaths when using the command line.
options.configSettings.autoSearchPaths = true;

Since pyright-scip replaces that entry point, the setting is lost.

There is a second consequence in the same function: ensureDefaultExtraPaths also expands
.pth files, but only for directories already in extraPaths. With extraPaths empty, an
editable install's .pth pointing at src/ is never consulted either — so having the package
pip install -e .'d does not rescue the situation.

Proposed fix

  const options = new CommandLineOptions(process.cwd(), false);
+ // Match pyright's CLI, which always enables this (see pyright.ts).
+ options.autoSearchPaths = true;

Optionally expose it as --no-auto-search-paths for parity, but the default should match
pyright's CLI. An explicit [tool.pyright] extraPaths still takes precedence, so existing
configured projects are unaffected.

Why this is worth fixing rather than documenting

The failure is silent and the damage is downstream. Nothing errors; the index is smaller. Every
consumer built on the index inherits wrong answers with no signal:

• "find references" returns a subset
• test-to-source navigation is empty for src-layout projects
• any dead-code or coverage analysis over the index reports false positives, because symbols used
  only from outside src/ look unreferenced

scip-python does print Pyproject file "..." is missing "[tool.pyright]" section. — but it reads
as informational, doesn't say what is lost, and appears on a run that exits 0.

Environment

• @sourcegraph/scip-python 0.6.6 (current npm latest at time of writing)
• verified against scip branch @ packages/pyright-scip/src/config.ts — same code
• compared against pyright 1.1.411
• Linux, Node v24.13.0

Workaround

Add to pyproject.toml:

[tool.pyright]
extraPaths = ["src"]

[linear]

OSS-24216