[Native] Serve many sequences from one ggml engine KV cache - #25007

[Native] Serve many sequences from one ggml engine KV cache#25007

metascroy wants to merge 1 commit into

offgraph-kv-ggml-kv-poolpytorch/executorch:offgraph-kv-ggml-kv-poolfrom

offgraph-kv-ggml-cell-cachepytorch/executorch:offgraph-kv-ggml-cell-cacheCopy head branch name to clipboard