Skip to content

Add persistent cache and incremental parallel analysis #41

Closed

Description

@SizzleUnrlsd

Cost model: Ttotal = Tcore + Tunique_analyses + Σ Tplugin, never P × Tcore.

  • Persistent on-disk cache keyed by IR fingerprint and analysis/plugin versions (feat: persistent cache #42: cache.py, keys from source, engine/schema/plugin versions, plugin code, models, advisories, dependency graph and transitive import keys; --cache DIR)
  • Incremental parsing with Tree-sitter (retired by decision on 2026-09-04, docs: frontend decision #50: the frontend stays the stdlib ast adapter behind build_hir; the per-module persistent cache covers incremental analysis at this scale; decision note in docs/architecture.md §3.1)
  • Incremental summaries (feat: persistent cache #42: cached summaries seed the project index, only changed modules and their importers are re-analysed)
  • Parallel analysis over the DAG at module granularity (feat: parallel analysis and memory eviction #43: ModuleGraph.schedule waves of strongly connected components, --jobs N process pool; function-level parallelism inside a module is not needed while summaries are per module)
  • Memory eviction of PyIR and results once summaries exist (feat: parallel analysis and memory eviction #43: ResultsEvicted pass keeps semantic tables and engine inputs only)

Reference: docs/architecture.md §11 Cache, §29 Performance Model, §30 Memory Complexity.

Delivery: #42 (cache and incremental summaries)
Delivery: #43 (scheduling, parallel analysis, eviction)
Delivery: #50 (decision note, Tree-sitter retired)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions