Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions config-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,14 @@
"none"
],
"default": "full"
},
"include": {
"type": "array",
"description": "Extra files and directories to index, relative to the workspace root unless absolute. Listing a path here indexes it: directories are walked recursively, files are scanned directly, and entries that do not exist are ignored. Included paths are indexed after the workspace scan, so they fill gaps rather than shadowing symbols the scan already found.",
"items": {
"type": "string"
},
"default": []
}
}
},
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -744,7 +744,7 @@ The indexing strategy is configurable via `[indexing] strategy` in `.phpantom.to

The merged pipeline works in three steps: (1) load `autoload_classmap.php` into a `HashMap<String, PathBuf>`, (2) collect the classmap's file paths into a `HashSet<PathBuf>` skip set, (3) self-scan all PSR-4 and vendor directories, skipping files already in the skip set. The result is a merged index: classmap entries for everything Composer already knew about, plus self-scanned entries for everything it missed. When the classmap is complete (the common case), the self-scanner walks directories but skips every file, finishing almost instantly. When the classmap is empty or absent, it falls back to a full self-scan. When the classmap is partial (e.g. vendor classes only), vendor files are skipped and only user code is scanned. Every state of the classmap helps.

When self-scanning with a `composer.json` present, the scanner reads `autoload.psr-4`, `autoload-dev.psr-4`, `autoload.classmap`, and `autoload-dev.classmap` to determine which directories to walk. PSR-4 directories are filtered: only classes whose FQN matches the namespace prefix plus the relative file path are included. Vendor packages are discovered from `vendor/composer/installed.json` (both Composer 1 and 2 formats); the JSON packages array is borrowed rather than cloned to avoid allocating a copy of the entire vendor manifest. All directory walkers (full-scan, PSR-4 scanner, vendor package scanner, and go-to-implementation file collector) use the `ignore` crate for gitignore-aware traversal. Hidden directories are skipped automatically, and `.gitignore` rules are respected at every level. When no `composer.json` exists at all, the scanner falls back to walking all `.php` files under the workspace root.
When self-scanning with a `composer.json` present, the scanner reads `autoload.psr-4`, `autoload-dev.psr-4`, `autoload.classmap`, and `autoload-dev.classmap` to determine which directories to walk. PSR-4 directories are filtered: only classes whose FQN matches the namespace prefix plus the relative file path are included. Vendor packages are discovered from `vendor/composer/installed.json` (both Composer 1 and 2 formats); the JSON packages array is borrowed rather than cloned to avoid allocating a copy of the entire vendor manifest. All directory walkers (full-scan, PSR-4 scanner, vendor package scanner, and go-to-implementation file collector) use the `ignore` crate for gitignore-aware traversal. Hidden directories are skipped automatically, and `.gitignore` rules are respected at every level. Paths listed under `[indexing] include` in `.phpantom.toml` are scanned after that walk with both filters disabled, which is how first-party PHP in a dotted directory and generated stub files that are gitignored as build artefacts get indexed; because they run last, they fill gaps rather than displacing what the walk found. When no `composer.json` exists at all, the scanner falls back to walking all `.php` files under the workspace root.

The scan results are converted to URI strings and inserted into `fqn_uri_index`. Everything downstream (resolution, diagnostics, go-to-definition) uses the unified index.

Expand Down
1 change: 1 addition & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added

- **`analyze` takes more than one path.** `phpantom_lsp analyze app/ lib/Helper.php tests/` scans the union of everything named, mixing directories and single files freely, so a pre-commit hook or a CI step can hand it exactly the paths that changed instead of running the whole project or invoking the binary once per path. Overlapping arguments are reported once, and a path that does not exist still stops the run with exit code 2. Naming no path scans the entire project, as before.
- **Allow scanning extra files hidden or gitignored.** The scan is gitignore-aware and skips hidden entries, which is what keeps it off `vendor/`, `node_modules/`, and build output. The same filtering hid two things a project genuinely depends on: first-party PHP living in a dotted directory, and generated stub files, which are gitignored precisely because they are build artefacts, yet are the only declaration of the symbols the project calls. A project relying on either got no completion for those symbols, no way to navigate to them, and an unknown-function error on every call. A new `[indexing] include` setting names such paths and indexes them regardless of either filter: directories are walked, files are read directly, and entries that no longer exist are skipped rather than failing the scan. They are indexed after the workspace scan, so they fill gaps rather than displacing what the scan already found. See [configuration.md](configuration.md#extra-include-paths). Contributed by @joelwurtz.

### Changed

Expand Down
32 changes: 29 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,35 @@ message = "^Call to deprecated function some_legacy_helper\\(\\)"

### `[indexing]`

| Key | Type | Default | Description |
| ---------- | ------ | -------- | ----------- |
| `strategy` | string | `"full"` | Class discovery strategy: `"full"`, `"composer"`, `"self"`, or `"none"`. See [Indexing Strategy](#indexing-strategy) below. |
| Key | Type | Default | Description |
| ---------- | --------------- | -------- | ----------- |
| `strategy` | string | `"full"` | Class discovery strategy: `"full"`, `"composer"`, `"self"`, or `"none"`. See [Indexing Strategy](#indexing-strategy) below. |
| `include` | array of string | `[]` | Extra files and directories to index, relative to the workspace root. See [Extra Include Paths](#extra-include-paths) below. |

#### Extra Include Paths

The workspace scan is gitignore-aware and skips hidden entries, which is
what keeps it off `vendor/`, `node_modules/` and build output. That same
filtering hides two things a project can legitimately depend on:
first-party PHP living in a dotted directory, and generated IDE stub
files, which are gitignored precisely because they are build artefacts —
yet are the only declaration of the symbols the project calls.

Listing a path under `include` indexes it regardless of both filters.
Directories are walked recursively, files are scanned directly, and
entries that do not exist are ignored. Absolute paths are accepted;
relative ones resolve against the workspace root.

A project whose helpers live in a dotted directory and whose signatures
come from a generated stub needs both halves:

```toml
[indexing]
include = [".lib", ".lib.stub.php"]
```

Included paths are indexed after the workspace scan, so they fill gaps
rather than shadowing symbols the scan already found.

### `[semantic_tokens]`

Expand Down
5 changes: 4 additions & 1 deletion docs/todo/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,10 @@ access provides complete `FunctionInfo`/`DefineInfo`. All directory
walkers (full-scan, PSR-4 scanner, vendor package scanner, and
go-to-implementation file collector) use the `ignore` crate for
gitignore-aware traversal instead of hardcoded directory name
filtering. Hidden directories are skipped automatically.
filtering. Hidden directories are skipped automatically. Paths listed
under `[indexing] include` are scanned on top of that walk, bypassing
both filters, for first-party PHP in a dotted directory and for
generated stub files that are gitignored as build artefacts.

The monorepo path activates when there is no root `composer.json` but
`discover_subproject_roots` finds subdirectories with their own
Expand Down
46 changes: 46 additions & 0 deletions src/classmap_scanner/discovery.rs
Original file line number Diff line number Diff line change
Expand Up @@ -894,6 +894,52 @@ pub fn scan_workspace_fallback_full(
scan_files_parallel_full(&php_files, progress)
}

/// Scan the files and directories named by `[indexing] include`,
/// bypassing both the gitignore and the hidden-entry filters.
//////
/// Directories are walked recursively; files are scanned directly. A
/// path the normal walk already covered is simply scanned twice and
/// loses the first-wins merge, so callers need not deduplicate.
pub fn scan_include_paths(
paths: &[PathBuf],
progress: Option<&ScanProgress>,
) -> WorkspaceScanResult {
use ignore::WalkBuilder;

let mut php_files: Vec<(PathBuf, crate::ClassCompletionOrigin)> = Vec::new();
for path in paths {
if path.is_file() {
if is_php_file(path) {
php_files.push((path.clone(), crate::ClassCompletionOrigin::Project));
}
continue;
}

let walker = WalkBuilder::new(path)
.git_ignore(false)
.git_global(false)
.git_exclude(false)
.ignore(false)
.parents(false)
.hidden(false)
.build();
for entry in walker.flatten() {
let entry_path = entry.path();
if is_php_file(entry_path)
&& (entry.file_type().is_some_and(|ft| ft.is_file()) || entry_path.is_file())
{
php_files.push((
entry_path.to_path_buf(),
crate::ClassCompletionOrigin::Project,
));
}
}
}

progress_add_total(progress, php_files.len());
scan_files_parallel_full(&php_files, progress)
}

/// Scan Drupal-specific directories for PHP symbols, bypassing `.gitignore`.
///
/// Drupal projects typically exclude their web root directories
Expand Down
102 changes: 102 additions & 0 deletions src/classmap_scanner/discovery_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,108 @@ fn scan_workspace_fallback_full_skips_hidden_dirs() {
);
}

// ── scan_include_paths ──────────────────────────────────────────

/// A hidden directory holding first-party PHP (Castor's `.castor/`) is
/// invisible to the workspace walk, which is the case `include` exists
/// for.
#[test]
fn scan_include_paths_walks_hidden_directory() {
let dir = tempfile::tempdir().unwrap();
let hidden = dir.path().join(".castor");
std::fs::create_dir_all(&hidden).unwrap();
std::fs::write(
hidden.join("docker.php"),
"<?php\nnamespace docker;\nfunction up(): void {}\n",
)
.unwrap();

let skip = std::collections::HashSet::new();
assert!(
!scan_workspace_fallback_full(dir.path(), &skip, None)
.function_index
.contains_key("docker\\up"),
"precondition: the workspace walk must not see it"
);

let result = scan_include_paths(&[hidden], None);
assert!(
result.function_index.contains_key("docker\\up"),
"got: {:?}",
result.function_index.keys().collect::<Vec<_>>()
);
}

/// A generated stub is hidden *and* gitignored; naming it directly has
/// to defeat both filters.
#[test]
fn scan_include_paths_reads_hidden_gitignored_file() {
let dir = tempfile::tempdir().unwrap();
// `ignore` only honours `.gitignore` inside a git repository.
std::fs::create_dir_all(dir.path().join(".git")).unwrap();
std::fs::write(dir.path().join(".gitignore"), "/.castor.stub.php\n").unwrap();
let stub = dir.path().join(".castor.stub.php");
std::fs::write(
&stub,
"<?php\nnamespace Castor {\n function run(string $command): void {}\n function io(): void {}\n}\n",
)
.unwrap();

let skip = std::collections::HashSet::new();
assert!(
!scan_workspace_fallback_full(dir.path(), &skip, None)
.function_index
.contains_key("Castor\\run"),
"precondition: the workspace walk must not see it"
);

let result = scan_include_paths(&[stub], None);
assert!(result.function_index.contains_key("Castor\\run"));
assert!(result.function_index.contains_key("Castor\\io"));
}

#[test]
fn scan_include_paths_reports_classes_and_constants() {
let dir = tempfile::tempdir().unwrap();
let hidden = dir.path().join(".tools");
std::fs::create_dir_all(&hidden).unwrap();
std::fs::write(
hidden.join("helpers.php"),
"<?php\nclass Helper {}\nconst TOOL_VERSION = 1;\n",
)
.unwrap();

let result = scan_include_paths(&[hidden], None);
assert!(result.classmap.contains_key("Helper"));
assert!(result.constant_index.contains_key("TOOL_VERSION"));
}

/// Non-PHP files in an included directory must be left alone.
#[test]
fn scan_include_paths_ignores_non_php_files() {
let dir = tempfile::tempdir().unwrap();
let included = dir.path().join(".castor");
std::fs::create_dir_all(&included).unwrap();
std::fs::write(included.join("notes.md"), "function notAFunction() {}").unwrap();
std::fs::write(
included.join("tasks.php"),
"<?php\nfunction realFunction(): void {}\n",
)
.unwrap();

let result = scan_include_paths(&[included], None);
assert!(result.function_index.contains_key("realFunction"));
assert!(!result.function_index.contains_key("notAFunction"));
}

#[test]
fn scan_include_paths_with_no_paths_is_empty() {
let result = scan_include_paths(&[], None);
assert!(result.classmap.is_empty());
assert!(result.function_index.is_empty());
assert!(result.constant_index.is_empty());
}

// ── is_drupal_php_file ──────────────────────────────────────────

#[test]
Expand Down
2 changes: 1 addition & 1 deletion src/classmap_scanner/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ mod lexer;

pub(crate) use discovery::vendor_package_roots;
pub use discovery::{
scan_directories, scan_drupal_directories, scan_psr4_directories,
scan_directories, scan_drupal_directories, scan_include_paths, scan_psr4_directories,
scan_psr4_directories_with_skip, scan_vendor_packages, scan_vendor_packages_with_skip,
scan_workspace_fallback, scan_workspace_fallback_full,
};
Expand Down
99 changes: 99 additions & 0 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -508,12 +508,44 @@ pub struct IndexingConfig {
/// if present, still resolves on demand, but never falls back to
/// self-scan.
pub strategy: Option<IndexingStrategy>,

/// Extra files and directories to index, relative to the workspace
/// root (absolute paths are also accepted).
///
/// Listing a path here indexes it regardless of both filters.
/// Directories are walked recursively; files are scanned directly.
///
/// ```toml
/// [indexing]
/// include = [".lib", ".lib.stub.php"]
/// ```
pub include: Vec<String>,
}

impl IndexingConfig {
pub fn strategy(&self) -> IndexingStrategy {
self.strategy.unwrap_or_default()
}

/// Resolve [`include`](Self::include) against the workspace root,
/// dropping entries that do not exist.
///
/// Relative entries are joined to `workspace_root`; absolute ones
/// are taken as-is.
pub fn include_paths(&self, workspace_root: &Path) -> Vec<PathBuf> {
self.include
.iter()
.map(|entry| {
let path = Path::new(entry);
if path.is_absolute() {
path.to_path_buf()
} else {
workspace_root.join(path)
}
})
.filter(|path| path.exists())
.collect()
}
}

/// The indexing strategy that controls class discovery behaviour.
Expand Down Expand Up @@ -1287,6 +1319,73 @@ paths = ["database/schema", "extra/schema.sql"]
assert_eq!(config.indexing.strategy, Some(IndexingStrategy::None));
}

/// Asserted on the default value rather than through
/// [`load_config`], which merges the developer's own global
/// `.phpantom.toml` and would make the result machine-dependent.
#[test]
fn indexing_include_defaults_to_empty() {
let dir = tempfile::tempdir().unwrap();
let config = IndexingConfig::default();
assert!(config.include.is_empty());
assert!(config.include_paths(dir.path()).is_empty());
}

#[test]
fn parses_indexing_include() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(CONFIG_FILE_NAME);
std::fs::write(
&path,
"[indexing]\ninclude = [\".castor\", \".castor.stub.php\"]\n",
)
.unwrap();
let config = load_config(dir.path()).unwrap();
assert_eq!(config.indexing.include, [".castor", ".castor.stub.php"]);
}

#[test]
fn include_paths_resolve_relative_to_workspace_root() {
let dir = tempfile::tempdir().unwrap();
std::fs::create_dir_all(dir.path().join(".castor")).unwrap();

let config = IndexingConfig {
include: vec![".castor".to_string()],
..Default::default()
};
assert_eq!(
config.include_paths(dir.path()),
[dir.path().join(".castor")]
);
}

#[test]
fn include_paths_accept_absolute_entries() {
let dir = tempfile::tempdir().unwrap();
let stub = dir.path().join(".castor.stub.php");
std::fs::write(&stub, "<?php\n").unwrap();

let config = IndexingConfig {
include: vec![stub.display().to_string()],
..Default::default()
};
// Resolved against a *different* root: an absolute entry is
// taken as-is rather than joined.
let other_root = tempfile::tempdir().unwrap();
assert_eq!(config.include_paths(other_root.path()), [stub]);
}

/// A stale entry must not become a phantom path handed to the
/// scanner — dropping it keeps the walk honest.
#[test]
fn include_paths_drop_missing_entries() {
let dir = tempfile::tempdir().unwrap();
let config = IndexingConfig {
include: vec!["does/not/exist".to_string()],
..Default::default()
};
assert!(config.include_paths(dir.path()).is_empty());
}

#[test]
fn invalid_indexing_strategy_returns_parse_error() {
let dir = tempfile::tempdir().unwrap();
Expand Down
Loading