Issues

Ornith-1.5: validate end-to-end support + plan Qwen3.5 vision

pekkah/SharpInference#468 · opened

on Aug 21, 2026

Sampler: implement XTC (eXclude Top Choices)

pekkah/SharpInference#466 · opened

on Aug 10, 2026

Sampler: implement no-repeat-ngram (hard ban on repeating an n-gram)

pekkah/SharpInference#465 · opened

on Aug 10, 2026

Sampler: implement DRY (Don't Repeat Yourself) sequence-repetition penalty

pekkah/SharpInference#464 · opened

on Aug 10, 2026

docs(readme): refresh the benchmark table — accumulated drift (stale numbers + descriptions) found during #440

pekkah/SharpInference#442 · opened

on Jul 13, 2026

perf(cuda): DSpark GPU draft round runs ~3× the bandwidth floor — small-M (B=7) projection GEMMs → weight-stationary matvec (#428 lever 1)

pekkah/SharpInference#441 · opened

on Jul 13, 2026

Centralize the KVarN support matrix + extend it to the server (TurboQuant head-dim gate)

pekkah/SharpInference#437 · opened

on Jul 11, 2026

GPU Lloyd-Max TqAttention writes compressed-region V aggregate in the rotated basis (no inverse WHT)

pekkah/SharpInference#435 · opened

on Jul 11, 2026

KVarN follow-ups: int8-TC prefill GEMM, split-tile decode kernel, minors (#180 successor)

pekkah/SharpInference#433 · opened

on Jul 11, 2026

perf(cuda): CUDA-graph-captured k-token spec verify — the ~10.6 ms fixed verify overhead is now DSpark's whole gap (#428 lever 2)

pekkah/SharpInference#430 · opened

on Jul 9, 2026

feat(engine): DSpark under continuous batching + load-aware verify length (spec Phases 5/6 remainder)

pekkah/SharpInference#427 · opened

on Jul 9, 2026

1

perf(cuda): dense int8-MMQ prefill ~2–2.6× behind llama.cpp (cp.async MMQ + flash attn) — Qwen3-8B / Gemma4

pekkah/SharpInference#409 · opened

on Jun 25, 2026

perf(cuda): full-offload MoE decode ~3.4× behind llama.cpp — OLMoE 126 vs 426 t/s (on-GPU experts, kernel-bound)

pekkah/SharpInference#408 · opened

on Jun 25, 2026

perf(cuda): batched int8-MMQ trunk prefill gated off for dense NORM-RoPE & QKV-bias models — SmolLM2 105×, VibeThinker/Qwen2 159×

pekkah/SharpInference#407 · opened

on Jun 25, 2026

perf(cuda): batched int8-MMQ trunk prefill gated off for MoE — OLMoE prefill 97 vs llama.cpp 17,137 t/s (176×)

pekkah/SharpInference#406 · opened

on Jun 25, 2026

perf(cuda): README vs llama.cpp CUDA benchmark gaps — full results + tracking

pekkah/SharpInference#405 · opened

on Jun 25, 2026

2

perf(cuda): CUDA-graph decode trunk follow-ups — default-on eval, dense/non-GDN coverage, CB-engine wiring

pekkah/SharpInference#401 · opened

on Jun 25, 2026

perf(cuda): Carnice decode toward llama.cpp ~58 t/s — GPU trunk decode kernels are the lever (6.17ms vs ~1ms/tok)

pekkah/SharpInference#400 · opened

on Jun 25, 2026

Add support for dense Qwen3.5-9B (hybrid Gated-DeltaNet + attention, dense FFN)

pekkah/SharpInference#389 · opened

on Jun 24, 2026

Carnice prefill toward llama.cpp 676 t/s — per-layer rewrite levers (op-offload follow-up to #387)

pekkah/SharpInference#388 · opened

on Jun 24, 2026

1

feat(core): constrain array/object value SHAPE inside Qwen3-Coder XML parameters (#383 follow-up)

pekkah/SharpInference#386 · opened

on Jun 24, 2026

feat(cli): sharpi run --tools — execute tools + multi-turn agentic loop (#382 follow-up)

pekkah/SharpInference#384 · opened

on Jun 24, 2026

Vulkan GDN-hybrid MTP decode collapses at context (verify attention has no split-KV; plain decode is ctx-stable)

pekkah/SharpInference#373 · opened

on Jun 23, 2026

Vulkan batched trunk: amortize the dense CPU FFN across batched/verify tokens (MatVec4In #209 analogue)

pekkah/SharpInference#371 · opened

on Jun 23, 2026

Vulkan MoE-MTP: batched verify reads routed experts k×/token (un-amortized) → MTP regresses vs plain decode

pekkah/SharpInference#370 · opened

on Jun 23, 2026

Navigation Menu

Repository navigation

Issues

Ornith-1.5: validate end-to-end support + plan Qwen3.5 vision

Sampler: implement XTC (eXclude Top Choices)

Sampler: implement no-repeat-ngram (hard ban on repeating an n-gram)

Sampler: implement DRY (Don't Repeat Yourself) sequence-repetition penalty

docs(readme): refresh the benchmark table — accumulated drift (stale numbers + descriptions) found during #440

perf(cuda): DSpark GPU draft round runs ~3× the bandwidth floor — small-M (B=7) projection GEMMs → weight-stationary matvec (#428 lever 1)

Centralize the KVarN support matrix + extend it to the server (TurboQuant head-dim gate)

GPU Lloyd-Max TqAttention writes compressed-region V aggregate in the rotated basis (no inverse WHT)

KVarN follow-ups: int8-TC prefill GEMM, split-tile decode kernel, minors (#180 successor)

perf(cuda): CUDA-graph-captured k-token spec verify — the ~10.6 ms fixed verify overhead is now DSpark's whole gap (#428 lever 2)

feat(engine): DSpark under continuous batching + load-aware verify length (spec Phases 5/6 remainder)

perf(cuda): dense int8-MMQ prefill ~2–2.6× behind llama.cpp (cp.async MMQ + flash attn) — Qwen3-8B / Gemma4

perf(cuda): full-offload MoE decode ~3.4× behind llama.cpp — OLMoE 126 vs 426 t/s (on-GPU experts, kernel-bound)

perf(cuda): batched int8-MMQ trunk prefill gated off for dense NORM-RoPE & QKV-bias models — SmolLM2 105×, VibeThinker/Qwen2 159×

perf(cuda): batched int8-MMQ trunk prefill gated off for MoE — OLMoE prefill 97 vs llama.cpp 17,137 t/s (176×)

perf(cuda): README vs llama.cpp CUDA benchmark gaps — full results + tracking

perf(cuda): CUDA-graph decode trunk follow-ups — default-on eval, dense/non-GDN coverage, CB-engine wiring

perf(cuda): Carnice decode toward llama.cpp ~58 t/s — GPU trunk decode kernels are the lever (6.17ms vs ~1ms/tok)

Add support for dense Qwen3.5-9B (hybrid Gated-DeltaNet + attention, dense FFN)

Carnice prefill toward llama.cpp 676 t/s — per-layer rewrite levers (op-offload follow-up to #387)

feat(core): constrain array/object value SHAPE inside Qwen3-Coder XML parameters (#383 follow-up)

feat(cli): sharpi run --tools — execute tools + multi-turn agentic loop (#382 follow-up)

Vulkan GDN-hybrid MTP decode collapses at context (verify attention has no split-KV; plain decode is ctx-stable)

Vulkan batched trunk: amortize the dense CPU FFN across batched/verify tokens (MatVec4In #209 analogue)

Vulkan MoE-MTP: batched verify reads routed experts k×/token (un-amortized) → MTP regresses vs plain decode