Skip to content

context: expose measured host/device allocation plans on the loaded model - #103

Draft
joelteply wants to merge 1 commit into
codex/s4-per-mind-integrationfrom
context-memory-plan-1d8e616b
Draft

joelteply wants to merge 1 commit into
codex/s4-per-mind-integrationfrom
context-memory-plan-1d8e616b

Conversation

@joelteply

Copy link
Copy Markdown

Embedding admission currently carries a Metal-derived estimate into every backend. This adds a context-only measurement API on the already loaded model: the existing memory constructors and graph allocator receive an explicit no-allocation mode, and report host/device buffer placement through a typed callback. No weights are reloaded or mutated. Output growth is included, compute buffers shared by backend types are counted once, and measurement and materialization use the same context parameter validation.

Validation: Windows CPU native build passed; Windows CUDA native build (SM120, CUDA13.2) passed. Rust consumer tests cover mixed-placement budgets, repeated buffer rows and null-model rejection. Real-model comparison is exposed as an explicitly opt-in isolated test in the paired Continuum change and remains pending independent review. No runtime deployment or learning acceptance is claimed.

This patch starts at the installed native pin316ad8b9250d114533e70b846b5ffbb4c7ebd80a. The three later diagnostic commits on the integration branch are preserved and are not part of the proposed consumer pin. Review alongside Continuum card1d8e616b. AI implementation assisted by Codex under Joel's explicit direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant