Skip to content

Import: share a referenced project's CCU instead of pickling it - #20416

Open
auduchinok wants to merge 3 commits into
dotnet:mainfrom
auduchinok:il-projectReferenceHandover
Open

Import: share a referenced project's CCU instead of pickling it#20416
auduchinok wants to merge 3 commits into
dotnet:mainfrom
auduchinok:il-projectReferenceHandover

Conversation

@auduchinok

@auduchinok auduchinok commented Sep 1, 2026

Copy link
Copy Markdown
Member

A project reference pickles its signature once and every consumer unpickles its own copy: five projects
referencing one hub hold five copies of its signature TAST. The pickled tree is already in the shape
consumers need, so offer it directly through IImportedProjectCcu, and pickle only when something asks.
Depends on the shared imported assemblies change (#20296), whose keys let a consumer establish it will hold
the same ccus.

Four things make it usable:

  • Remapped against a ccu made for this view, not the project's live one, which a consumer would else hold
    whole.
  • PruneExportedSignatureInPlace brings it to the shape unpickling produces: no value definitions, no
    display-only data, no compiled-representation cache. Rescoping happens in the same traversal that builds
    the tree - ILScopeRef.Local is what internal is rooted at, so a consumer would otherwise read the
    reference's internals as its own.
  • Every non-local reference is re-pointed at the reading project's ccu of the same name, as u_ccuref does
    for a consumer reading the bytes; a name the reader lacks keeps ours. This is what lets a consumer on
    another framework take the contents.
  • What each name bound to is recorded, and a second consumer takes that copy only where it resolves all of
    them alike. Consumers that disagree get a copy each, capped at eight.

Binding runs once the batch is registered and before anything relinks: an assembly unpickled beside it
resolves the names it mentions against what is registered, and a delayed CCU is an error there rather than
a wait.

Solution Projects main This Retained
ReSharper.FSharp 10 385.94 MB 304.64 MB -81.30 MB (-21.1%)
Fantomas 8 350.42 MB 275.79 MB -74.63 MB (-21.3%)
FSharp.Compiler.Service 14 879.43 MB 818.39 MB -61.04 MB (-6.9%)
FsToolkit.ErrorHandling 8 87.77 MB 69.36 MB -18.42 MB (-21.0%)
Oxpecker 16 131.67 MB 114.24 MB -17.43 MB (-13.2%)
Prime 5 112.22 MB 97.93 MB -14.29 MB (-12.7%)
IcedTasks 7 96.97 MB 92.37 MB -4.61 MB (-4.8%)
consoleapp 1 29.38 MB 29.37 MB 0.00 MB

The numbers are smaller due to other FSharpChecker creation flags.

consoleapp has no project references. Not all of this is the handover: switching the handover alone off and
on inside one build moves the compiler's own solution by 2 MB rather than 61, the rest coming from dropping
the language version out of the import reuse key, which lets projects that differ only in it share one
imported graph.

Also fixes a check-then-act race in BackgroundCompiler that predates this: builders were cached without a
second look under the gate, so callers arriving together each built the project, and a project built twice
is two of everything it imports. Nothing here survives that.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

❗ Release notes required

You can open this PR in browser to add release notes: open in github.dev


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/Compiler` docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

@github-actions github-actions Bot added the AI-Tooling-Check-Scanned-Clean Tooling check: diff analyzed, no interesting infrastructure files label Sep 1, 2026
@auduchinok
auduchinok force-pushed the il-projectReferenceHandover branch from befdc7b to d62a9b1 Compare September 2, 2026 14:31
@auduchinok

Copy link
Copy Markdown
Member Author

This is ready.

@auduchinok
auduchinok force-pushed the il-projectReferenceHandover branch from d62a9b1 to cda7c71 Compare September 7, 2026 12:05
@github-actions github-actions Bot added the ⚠️ Affects-Compiler-Output Tooling check: PR touches IL emission or codegen label Sep 7, 2026
@github-actions

This comment has been minimized.

@T-Gro T-Gro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ AI review — verify independently.

Comment thread src/Compiler/TypedTree/TcGlobals.fs Outdated
fslibCcu,
directoryToResolveRelativePaths,
isInteractive,
checkNullness,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ FS3261 despite --checknulls- after another project warms the same checker: this copy keeps the cached project's checkNullness (both compiler modes).

// First project: --langversion:8.0 --checknulls+
module Warm
let x = 1

// Then, same FSharpChecker:
// --langversion:9.0 --checknulls- --warnaserror+
module Target
let x: string = null
// Warm checker: FS3261. Fresh checker: no diagnostics.

then
shareable.Add name |> ignore
match data with
| :? IImportedProjectCcu when takenFromProject name -> shareableNames.Add name |> ignore

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ Second consumer gets FS0001 (Shared.Carried versus Shared.Carried): cached Middle.dll stays linked to the first consumer's bound copy. Sharing disabled: both pass.

// Dependency.fs: two distinct projects, both named Dependency
module Dependency
type Marker = Marker

// Shared.fs: one project referencing Dependency project 1
module Shared
type Carried = { Value: Dependency.Marker option }

// Middle.fs: compiled DLL referencing Shared.dll and Dependency.dll
module Middle
let carry (x: Shared.Carried) = x

// Check consumer 1, then consumer 2, using the same checker:
// 1 references Shared project + Dependency project 1 + Middle.dll
// 2 references SAME Shared project + Dependency project 2 + SAME Middle.dll
module Consumer
let x: Shared.Carried = Middle.carry { Value = None }

let ctxt =
mkRemapContextLeavingAssembly g (StackGuard("RemapExprStackGuardDepth")) rescopeAccessTo

remapTyconToNonLocal ctxt tmenv x

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ FS0001 on a valid consumer extension with shared project references: the exported trait retains the producer's resolution context. Unpickling clears it, letting the consumer supply its own.

// Library.fs, --langversion:preview
module Library
let inline negate (x: ^T) =
    (^T : (static member Negate: ^T -> ^T) x)

// Consumer.fs, --langversion:preview; references Library project
module Consumer
type System.Int32 with
    static member Negate(x: int) = -x
let value = Library.negate 42
// Sharing on: FS0001, int lacks Negate. Off: no diagnostics.

| true, (already, _) -> already
| _ ->
let entry =
match resolve ccu.AssemblyName with

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ Missing-reference FS0074 disappears: the producer fallback lets the consumer use the public Leaf.Carried type without referencing Leaf. Sharing disabled: FS0074; enabled: no diagnostics.

// Leaf.fs
module Leaf
type Carried = { Value: int }

// Middle.fs, references Leaf project
module Middle
let make () : Leaf.Carried = { Value = 1 }

// Consumer.fs, references Middle project but NOT Leaf
module Consumer
let value = (Middle.make()).Value

let remapping =
MakeExportRemappingWith ccuRebind viewedCcu generatedCcu.Contents

let contents =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ Imported anonymous-record assembly metadata becomes (None, ""): its owner still points to the compiling CCU rather than viewedCcu, for both struct and reference records.

// Library.fs
module Library
let record = {| Value = 1 |} // also: struct {| Value = 1 |}

// Consumer.fs, references Library project
module Consumer
let record = Library.record

// FCS client: apply to Library.record from the referenced assembly
let metadata (value: FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue) =
    let owner = value.FullType.AnonRecordTypeDetails.Assembly
    owner.FileName, owner.QualifiedName
// Sharing on: (None, ""). Off: output path and full assembly identity.

let PruneExportedSignatureInPlace (mspec: ModuleOrNamespace) =
let rec pruneEntity (entity: Entity) =
entity.entity_il_repr_cache <- null
pruneContents entity.ModuleOrNamespaceType

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ MembersFunctionsAndValues returns the default C.M twice: the explicit implementation survives pruning and relinks to it. p_tcaug filters these entries before export.

// Library.fs
module Library
type IFoo =
    abstract M: unit -> int
type C() =
    abstract M: unit -> int
    default _.M() = 2
    interface IFoo with
        member _.M() = 1

// FCS client: apply to imported Library.C in a referencing project
let members (classType: FSharp.Compiler.Symbols.FSharpEntity) =
    classType.MembersFunctionsAndValues
    |> Seq.filter (fun m -> m.LogicalName = "M" && not m.IsDispatchSlot)
    |> Seq.map (fun m -> m.DisplayName, m.IsExplicitInterfaceImplementation)
    |> Seq.toArray
// Sharing on: [|("M", false); ("M", false)|]. Off: [|("M", false)|].

@T-Gro T-Gro added the AI-reviewed PR reviewed by AI review council label Sep 9, 2026
@T-Gro
T-Gro self-requested a review September 9, 2026 14:12
@auduchinok
auduchinok force-pushed the il-projectReferenceHandover branch 2 times, most recently from 09afc51 to 7ee068d Compare September 10, 2026 08:17
auduchinok and others added 3 commits September 11, 2026 09:31
A project reference pickles its signature once and every consumer unpickles its own copy: five projects
referencing one hub hold five copies of its signature TAST. The pickled tree is already in the shape
consumers need, so offer it directly through IImportedProjectCcu, and pickle only when something asks.

Four things make it usable:

- Remapped against a ccu made for this view, not the project's live one, which a consumer would else hold
  whole.
- PruneExportedSignatureInPlace brings it to the shape unpickling produces: no value definitions, no
  display-only data, no compiled-representation cache.
- Every non-local reference is re-pointed at the reading project's ccu of the same name, as unpickling
  does; a name the reader lacks keeps ours. So a consumer on another framework takes it too.
- What each name bound to is recorded, and a second consumer takes that copy only where it resolves all of
  them alike.

Binding runs once the batch is registered and before anything relinks: an assembly unpickled beside it
resolves the names it mentions against what is registered, and a delayed CCU is an error there.

Depends on sharing imported assemblies (dotnet#20296). Also fixes a check-then-act race in BackgroundCompiler:
builders were cached without a second look under the gate, so callers arriving together each built the
project.

Retained memory, under editor options. Diagnostics identical with the change off and on:

  Fantomas           8 proj  350.2 -> 275.7 MB  -74.5 (-21.3%)
  ReSharper.FSharp  10 proj  384.9 -> 304.6 MB  -80.3 (-20.9%)
  FsToolkit          8 proj   87.7 ->  70.1 MB  -17.6 (-20.1%)
  Oxpecker          16 proj  131.7 -> 114.2 MB  -17.5 (-13.3%)
  Prime              5 proj  112.2 ->  97.9 MB  -14.3 (-12.7%)
  IcedTasks          7 proj   97.0 ->  92.4 MB   -4.6  (-4.7%)
  FCS solution      14 proj  820.8 -> 818.7 MB   -2.1  (-0.2%)
  consoleapp         1 proj   29.4 ->  29.4 MB    0.0

The FCS solution gains least: nearly every edge in it crosses framework import layers, so each consumer
rebuilds a copy of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@auduchinok
auduchinok force-pushed the il-projectReferenceHandover branch from 7ee068d to 552a152 Compare September 11, 2026 07:31
@github-actions github-actions Bot added the ⚠️ Affects-Design-Time Tooling check: PR touches type providers or dependency manager label Sep 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Tooling Safety Check — Affects-Compiler-Output, Affects-Design-Time
Affects-Compiler-Output: Typed-tree remapping changes project-reference compilation inputs.
Affects-Design-Time: FCS background compilation now shares imported project signatures.

Generated by PR Tooling Safety Check · gpt56 707.4K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚠️ Affects-Compiler-Output Tooling check: PR touches IL emission or codegen ⚠️ Affects-Design-Time Tooling check: PR touches type providers or dependency manager AI-reviewed PR reviewed by AI review council AI-Tooling-Check-Scanned-Clean Tooling check: diff analyzed, no interesting infrastructure files

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

2 participants