Reuse a C# project's in-memory PE reference while its semantic version holds - #20460
xperiandri wants to merge 5 commits into
Conversation
✅ No release notes required |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
xperiandri
left a comment
There was a problem hiding this comment.
Review from the architecture side. The direction is right, and it has a precedent worth citing in the description: Roslyn's own cross-language reference cache, SolutionCompilationState.SkeletonReferenceCache, keys the metadata-only skeleton of a project on Project.GetDependentSemanticVersionAsync and, for an unchanged version, returns what it has "regardless of whether it succeeded or not". The editor already treats that version as "the reference did not change" in WorkspaceExtensions.getReferencedProjectVersions (the snapshot-reuse guard), so this PR makes the PE-reference stamp consistent with that guard instead of racing ahead of it.
Findings, most important first. Each has an inline suggestion; the first three go together.
- A cache hit still awaits
GetCompilationAsync. On a hit the compilation only feedsRefresh, and after the first emit result nobody reads it again — but when Roslyn has dropped the final compilation under memory pressure the call rebuilds it and the entry pins it: one full compilation per referenced C# project per options recompute, in exactly the scenario the PR targets. Roslyn's cache checks the version first and touches the compilation only on a miss. RefreshracesEmitted()(reactor thread vs. FCS emit thread, no shared lock) and can re-pin a compilation after its emit succeeded;TryGetCompilationalso ignorespinned. With (1),Refreshdisappears and the entry becomes single-writer.and!would be the first use in the repository (FSharp.instructions.md: no foothold) and buys nothing here. Gone with (1).- The new
peReferences.TryRemoveinClearOptionsis dead code — it sits in the arm that is entered for F# project ids only, so a removed C# project keeps its entry until the next sweep. TheClearSingleFileOptionsCachetwin is dead for the same reason (miscellaneous-files project), and so are the pre-existinglastSuccessfulCompilationslines next to them. - Nits: three
// Stop strongly holding…comments restateEmitted(); the comment abovepeReferencesis now covered by the type summary; the release note carries the war story.
Tests (the checklist asks where this could be covered): the editor test host cannot hold a C# project today — TestHostWorkspaceServices.GetLanguageServices in RoslynHelpers.fs throws for anything but F#, so SupportsCompilation is never true there. Two routes: (a) compose Microsoft.CodeAnalysis.CSharp.Workspaces into the test host and assert that TryGetOptionsByProject hands back the same ReferencedProjects.[0] object across a body-only edit of the C# project and a different one after a declaration edit; (b) cheaper — make the version/pin logic an internal type with the emit function injected and test it directly: same object for the same version, new object on a version change, pin dropped after a result, pin kept and stamp bumped on cancellation. I checked the suggested blocks with Fantomas and with a stand-in script covering the cases in (b); a full FSharp.Editor build was not run.
Not for this PR, but worth knowing: with the transparent compiler off, Stamp = hash(GetDependentVersionAsync) still creates a new IncrementalBuilder — which imports its non-framework references afresh — on every edit in a referenced C# project, body-level included, so the legacy path does not benefit. And FCS keys on a wall-clock DateTime: two version changes inside one clock tick share a stamp; carrying the previous entry's stamp forward (max UtcNow (previous + 1 tick)) would make it strictly increasing per project.
|
🔍 Tooling Safety Check — Affects-Design-Time
|
| referencedProjects.Add(peRef) | ||
| let! version = referencedProject.GetDependentSemanticVersionAsync(ct) | ||
|
|
||
| match tryGetPEReference referencedProject version with |
There was a problem hiding this comment.
🤖🕵️⏱️🔥
Add an exact-head trace or focused test that counts PE emits, imports, and checks across unchanged and changed dependent semantic versions. Until then, limit the claim to the proven avoided PE emit and reader construction.
There was a problem hiding this comment.
Added a focused test in 3d74593: tests/fsharp/Compiler/Service/MultiProjectTests.fs counts PE emits directly.
Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation— checks the same F# file against twoPEReferences backed by two independentCSharpCompilationinstances sharing one stamp; the second reference's emit counter stays at 0.Changing a CSharp reference's stamp does re-emit the new Compilation— same setup with a different stamp; the second reference's emit counter is ≥ 1 (the control, ruling out the first test passing for an unrelated reason).
Both pass locally (2/2), pre-existing MultiProjectTests facts (5 total) unaffected.
This proves the emit/import avoidance directly (no editor Workspace needed — TestHostWorkspaceServices.GetLanguageServices throws for non-F#, which is why this lives at the FCS level rather than exercising FSharpProjectOptionsManager itself). I also softened the PR description to lead with what's proven and call the CPU trace an illustration rather than a measurement, per your note.
There was a problem hiding this comment.
@xperiandri TODO - I think this hasnt been addressed
There was a problem hiding this comment.
It is in, in tests/fsharp/Compiler/Service/MultiProjectTests.fs (lines 276 and 291) — two facts that count emits directly rather than timing anything:
Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation— one F# file checked against twoPEReferences backed by two independentCSharpCompilationinstances that share a stamp. The second reference's emit counter stays at 0.Changing a CSharp reference's stamp does re-emit the new Compilation— the same setup with a different stamp, asserting the counter is at least 1. It is there as the control: without it the first test would also pass if the reference were never consulted at all.
So of the three things asked for, PE emits are counted and proven, in both directions. Imports and checks are not counted — nothing in the test host exposes those counters, and reaching them would mean driving FSharpProjectOptionsManager through an editor workspace, which TestHostWorkspaceServices.GetLanguageServices refuses for a non-F# project. The PR description was narrowed at the same time to claim only the avoided emit and reader construction, and to call the CPU trace an illustration rather than a measurement.
If you want the import and check counts as well, I would rather do that as a separate change that gives FCS a test hook for them, since the same hook is what any future claim about reference invalidation will need.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2dd1a17 to
ced4562
Compare
…ce stamps Addresses dotnet#20460 (comment) by proving the specific claim the fix depends on, at the FCS level rather than through FSharpProjectOptionsManager (which the editor test host cannot exercise with a real C# project - see the review's earlier note on that gap). mkCountedCSharpPEReference builds a PEReference backed by an independent CSharpCompilation each time, with a counted getStream (the PE emit / metadata-import entry point DelayedILModuleReader lazily invokes). Two facts: - Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation: checking the same F# file first against one reference, then against a second reference for a different Compilation but the SAME stamp, leaves the second reference's emit count at 0 - FSharpProjectOptions.AreSameForChecking's structural fallback (both Stamp fields are None here) treats the two ReferencedProjects arrays as equal via PEReference's custom Equals (OutputFile + getStamp()), so the checker's incrementalBuildersCache reuses the existing build and never touches the new reference. - Changing a CSharp reference's stamp does re-emit the new Compilation: the same setup with a different stamp does invoke the second reference's getStream at least once - the control that rules out the first test passing by some unrelated always-cached path. Both verified locally (FSharpSuite.Tests.fsproj, filter-method "*CSharp reference's stamp*"): 2/2 pass. The full MultiProjectTests class (5 facts) still passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ce stamps Addresses dotnet#20460 (comment) by proving the specific claim the fix depends on, at the FCS level rather than through FSharpProjectOptionsManager (which the editor test host cannot exercise with a real C# project - see the review's earlier note on that gap). mkCountedCSharpPEReference builds a PEReference backed by an independent CSharpCompilation each time, with a counted getStream (the PE emit / metadata-import entry point DelayedILModuleReader lazily invokes). Two facts: - Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation: checking the same F# file first against one reference, then against a second reference for a different Compilation but the SAME stamp, leaves the second reference's emit count at 0 - FSharpProjectOptions.AreSameForChecking's structural fallback (both Stamp fields are None here) treats the two ReferencedProjects arrays as equal via PEReference's custom Equals (OutputFile + getStamp()), so the checker's incrementalBuildersCache reuses the existing build and never touches the new reference. - Changing a CSharp reference's stamp does re-emit the new Compilation: the same setup with a different stamp does invoke the second reference's getStream at least once - the control that rules out the first test passing by some unrelated always-cached path. Both verified locally (FSharpSuite.Tests.fsproj, filter-method "*CSharp reference's stamp*"): 2/2 pass. The full MultiProjectTests class (5 facts) still passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
3d74593 to
1119a1e
Compare
T-Gro
left a comment
There was a problem hiding this comment.
🤖🕵️ If this fixes an issue or implements an RFC/suggestion, link it (Fixes #... when applicable). Otherwise, give a short management-level summary in simplified technical English: what user scenario improves and what this achieves.
Please apply this PR-description guidance. Remove the implementation inventory already visible in Files, but keep necessary scope, compatibility, and dependency caveats.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ce stamps Addresses dotnet#20460 (comment) by proving the specific claim the fix depends on, at the FCS level rather than through FSharpProjectOptionsManager (which the editor test host cannot exercise with a real C# project - see the review's earlier note on that gap). mkCountedCSharpPEReference builds a PEReference backed by an independent CSharpCompilation each time, with a counted getStream (the PE emit / metadata-import entry point DelayedILModuleReader lazily invokes). Two facts: - Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation: checking the same F# file first against one reference, then against a second reference for a different Compilation but the SAME stamp, leaves the second reference's emit count at 0 - FSharpProjectOptions.AreSameForChecking's structural fallback (both Stamp fields are None here) treats the two ReferencedProjects arrays as equal via PEReference's custom Equals (OutputFile + getStamp()), so the checker's incrementalBuildersCache reuses the existing build and never touches the new reference. - Changing a CSharp reference's stamp does re-emit the new Compilation: the same setup with a different stamp does invoke the second reference's getStream at least once - the control that rules out the first test passing by some unrelated always-cached path. Both verified locally (FSharpSuite.Tests.fsproj, filter-method "*CSharp reference's stamp*"): 2/2 pass. The full MultiProjectTests class (5 facts) still passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1119a1e to
69cb60a
Compare
…n holds createPEReference keyed the reference on the identity of the Roslyn Compilation and stamped it with DateTime.UtcNow at creation. Roslyn recreates Compilation instances freely - on every solution fork, and under memory pressure because it holds the final compilation weakly - so a dependent F# project kept receiving a reference with a fresh stamp. That stamp feeds the project snapshot's base version, so BootstrapInfo was invalidated, every reference re-imported and every file re-checked, even though the referenced assembly's metadata had not changed. The reference is now cached per referenced project and reused while the project's dependent semantic version is unchanged; a newer Compilation only refreshes the source the delayed reader emits from. Metadata-only emit depends on the public surface, which is what that version tracks, so C# edits below the declaration level no longer invalidate F# checking either. The ConditionalWeakTable is gone with it: its value pinned the compilation until the first emit, which kept the key alive, so entries were never collected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mpilation with <see cref> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dropped - Look the reference up by dependent semantic version before asking for a Compilation, so a cache hit no longer rebuilds and re-pins one Roslyn dropped under memory pressure (matching SolutionCompilationState.SkeletonReferenceCache). This removes Refresh, and with it the reactor/emit-thread race that could re-pin a compilation after a successful emit, so no lock is needed; and! is gone with it. - TryGetCompilation prefers the pinned compilation and falls back to the weak reference, instead of assuming the two agree. - Split tryGetPEReference from createPEReference and rename createNewPEReference to buildPEReference. - ClearOptions: hoist the lastSuccessfulCompilations and peReferences removals out of the cache.TryRemove arm - cache holds F# project ids, both dictionaries hold referenced C# ids, so the removals never ran. Drop their ClearSingleFileOptionsCache twins, where the id is the miscellaneous-files project and can never be a key. - Drop comments the code already states, and trim the release note to the user-visible effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ce stamps Addresses dotnet#20460 (comment) by proving the specific claim the fix depends on, at the FCS level rather than through FSharpProjectOptionsManager (which the editor test host cannot exercise with a real C# project - see the review's earlier note on that gap). mkCountedCSharpPEReference builds a PEReference backed by an independent CSharpCompilation each time, with a counted getStream (the PE emit / metadata-import entry point DelayedILModuleReader lazily invokes). Two facts: - Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation: checking the same F# file first against one reference, then against a second reference for a different Compilation but the SAME stamp, leaves the second reference's emit count at 0 - FSharpProjectOptions.AreSameForChecking's structural fallback (both Stamp fields are None here) treats the two ReferencedProjects arrays as equal via PEReference's custom Equals (OutputFile + getStamp()), so the checker's incrementalBuildersCache reuses the existing build and never touches the new reference. - Changing a CSharp reference's stamp does re-emit the new Compilation: the same setup with a different stamp does invoke the second reference's getStream at least once - the control that rules out the first test passing by some unrelated always-cached path. Both verified locally (FSharpSuite.Tests.fsproj, filter-method "*CSharp reference's stamp*"): 2/2 pass. The full MultiProjectTests class (5 facts) still passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
69cb60a to
3951c85
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved moderate correctness issues remain, and coverage does not exercise the manager’s new cache behavior.
Review effort: Lite
Findings: 2
Open (2)
What changed in this PR
Caches in-memory C# PE references by dependent semantic version to reduce redundant metadata emission and F# rechecking.
Changes:
- Adds per-project PE reference caching and cleanup.
- Adds focused reuse and invalidation tests.
- Adds Visual Studio release notes.
| File | Review findings |
|---|---|
vsintegration/src/FSharp.Editor/LanguageService/FSharpProjectOptionsManager.fs |
Four moderate findings (1 vote each): body-only C# edits still invalidate F# checks; same-version cache hits can retain a failed compilation; DateTime.UtcNow may produce equal stamps for replacements; and output-path changes are not included in the cache identity. |
tests/fsharp/Compiler/Service/MultiProjectTests.fs |
Nit (2 votes): tests exercise FCS directly rather than FSharpProjectOptionsManager, so the new per-project cache behavior is not covered. |
docs/release-notes/.VisualStudio/18.vNext.md |
Nit (4 votes): three duplicate release-note bullets should be reduced to one, preferably the entry with the PR link. |
| * Reuse the in-memory PE reference of a referenced C# project while its dependent semantic version is unchanged, instead of creating one per Roslyn `Compilation` instance. A recreated `Compilation` produced a reference with a fresh stamp, which invalidated the FCS bootstrap of every dependent F# project and made it re-import all of its references before re-checking every file. | ||
| * Reuse the in-memory PE reference of a referenced C# project while its dependent semantic version is unchanged, instead of creating one per Roslyn `Compilation` instance. A recreated `Compilation` produced a reference with a fresh stamp, which invalidated the FCS bootstrap of every dependent F# project and made it re-import all of its references before re-checking every file. ([PR #20460](https://github.com/dotnet/fsharp/pull/20460)) | ||
| * Reuse the in-memory PE reference of a referenced C# project while its dependent semantic version is unchanged, so dependent F# projects no longer re-import every reference and re-check every file each time Roslyn recreates the `Compilation`. ([PR #20460](https://github.com/dotnet/fsharp/pull/20460)) |
| let ``Reusing a CSharp reference's stamp avoids re-emitting a recreated Compilation``() = | ||
| let stamp = DateTime(2024, 1, 1) | ||
| let csRefProj1, emitCount1 = mkCountedCSharpPEReference stamp | ||
| let csRefProj2, emitCount2 = mkCountedCSharpPEReference stamp | ||
|
|
||
| let fsOptions = projectReferencing csRefProj1 | ||
| checkUsesCSharpClass fsOptions | ||
| Assert.Equal(1, emitCount1()) | ||
|
|
||
| // Same dependent semantic version (stamp unchanged): the checker must reuse its cached | ||
| // project build and never touch the recreated Compilation behind the new reference. | ||
| checkUsesCSharpClass { fsOptions with ReferencedProjects = [|csRefProj2|] } |

The in-memory PE reference FCS caches for a referenced C# project was stamped with the time it was created, and Roslyn hands back a new
Compilationinstance for that project on every solution fork and, under memory pressure, even when nothing changed — so FCS treated an unchanged C# project as a different reference each time and kept re-importing and re-checking it, continuously at solution scale.The reference is now cached per referenced project and reused while the project's dependent semantic version is unchanged; a same-version
Compilationonly refreshes the source the delayed reader will emit from, when asked to. C# edits below the declaration level no longer invalidate F# checking either, since metadata-only emit depends on the public surface the dependent semantic version tracks. What is retained is bounded by the number of referenced C# projects — one compilation is pinned per project until its first emit result, then held weakly. Cancellation is unchanged: a cancelled emit keeps the compilation pinned and bumps the stamp so the reference is retried. The same stamp equality also lets the legacy (non-transparent-compiler) checker recognize two builds as the same project, so the reuse isn't specific to either checking path.A focused test proves the mechanism directly — repeated emit counts across an unchanged vs. a changed stamp — at the FCS level; the editor test host cannot construct a C# project, so
FSharpProjectOptionsManageritself is exercised only by building the VSIX and watching the Debug pane's FCS trace against a large solution.🤖 Generated with Claude Code