exl3

DeepSeek-V4.1-Flash on two DGX Sparks: EXL3 routed experts, 262k context, 2M-token KV, vision, tools, MTP, bundled with Pi

GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.

Kyojin: the Yamz inference engine for AMD Strix Halo (ROCm, gfx1151), built on ExLlamaV3. Runs 300B-class MoE models on one 128 GB mini PC.

Fast, capable GLM-5.3-Flash NVFP4 on two DGX Sparks: opinionated Atlas serving and reproducible inference research.

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.

vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.

Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights

Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090

Reproducible 262K GLM-5.3 Flash Uncensored EXL3+DFlash2 deployment for 2x RTX PRO 6000 Blackwell

GLM-5.3-Flash EXL3 on 4x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.

High-performance runtime extensions for vLLM.

Production-tuned deployment recipe: GLM-5.3-Flash 320B (EXL3 4-bit) on 2x NVIDIA DGX Spark - 850K context, DFlash2 + adaptive-k speculative decoding, full .env tuning and pitfalls

GLM-5.3-Flash EXL3 on three DGX Sparks (TP3/EP3, 1M context, vision, DFlash2) — jspark3 fork, traps, measurements

DeepSeek-V4-Flash-Vision-Exp (EXL3 MixedK, 256 experts, uncensored) on one NVIDIA DGX Spark with vLLM + sparkinfer: 245,760 context, vision + DSpark speculative decoding, CUDA graphs. Recipe, overlay patches, benchmarks, receipts.

GLM-5.3-Flash EXL3 on 2x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.

LLM inference server for ExLlamaV3 / EXL3, with OpenAI- and Anthropic-compatible APIs optimized for Agent workloads.

Qwen3.8-27B EXL3 on AMD gfx1201: RX 9070 XT (16 GB, tested) and Radeon AI PRO R9700 (32 GB). vLLM plugin with custom kernels: ~80 tok/s with MTP speculative decoding, int8 KV cache, 32k-64k context on 16 GB (up to 262k on 32 GB), vision and tool calls.

LLM / AI inference server for Windows + NVIDIA (EXL3/ExLlamaV3). OpenAI-compatible API, multi-GPU, Blazor admin. Ollama-like, no Docker.

Fast local LLM inference on AMD Radeon RX 7900 XTX / XT (RDNA3, ROCm): big Mixture-of-Experts models like Qwen3.8-Flash-Next on one or two consumer GPUs. 55 tok/s, 262k context, parallel agents, OpenAI/Anthropic API. A tuned build of FreeToken.