I work where inference engines meet real clusters.
AI infrastructure · inference systems · distributed runtime · performance engineering
I'm a software engineer on the Training and Inference Engine team at Baidu AI Computing, working on enterprise-scale LLM inference deployments.
My focus spans request & runtime scheduling, prefill/decode execution, KV cache management, P/D disaggregation, high-performance transfer engines, and heterogeneous accelerator backends — building systems that stay fast, observable, and reliable at extreme scale.
Building modular, engine-neutral components across the modern LLM serving stack:
System Architecture & Placement Model
Clients
│ (OpenAI / Native Protocols)
▼
┌───────────────────────────────────────────────────────────┐
│ Inference Frontend (Locus) │
│ Protocol · Templates · Tokenization · Tool/Reasoning IO │
└───────────────────────────┬───────────────────────────────┘
│ token request
▼
┌───────────────────────────────────────────────────────────┐
│ Global Inference Planner (Locus) │
│ Load · Cache Locality · Topology · P/D Aware Placement │
└─────────────┬───────────────────────────────┬─────────────┘
│ placement plan │ state plan
▼ ▼
┌───────────────────────────┐ ┌───────────────────────────┐
│ Inference Engines │ │ KV & State Plane │
│ SGLang / vLLM / sglang-rs│◄─►│ NexusKV / Mooncake │
│ GPU Compute & Kernels │ │ State Index · RDMA/TCP │
└───────────────────────────┘ └───────────────────────────┘
| Tier | Project | Focus & Highlights | Stack |
|---|---|---|---|
| Control Plane | Locus | Engine-neutral inference control plane for global compute and model-state placement across heterogeneous engines and state stores. | Rust Axum Control-Plane |
| State & Cache | NexusKV | Disaggregated KV cache platform separating control plane, data plane, prefix reuse indexing, and engine adapters. | Go Rust Python |
| Engine Runtime | sglang-rs | Rust runtime exploring request lifecycle, gRPC routing, prefix caching, KV page allocation, and P/D KV transfer boundaries. | Rust gRPC Runtime |
| Engineering Notes | imreese.github.io | Personal site and source-level systems engineering notes with interactive components and deep dives. | Next.js React MDX |
- 前缀缓存命中 50%，预填充为什么没有快 2 倍？
- Kimi Linear 的 KDA 缓存：SGLang、vLLM 与 Mooncake Store 全链路
- SGLang v0.5.14 接入 Mooncake Store：缓存页标识、零拷贝与共享 Transfer Engine
- SGLang HiCache 读路径：预取、回载和调度流程
- SGLang HiCache 写路径：GPU KV 如何写入主机内存和外部存储
AI Serving & Runtime
Storage, Transfer & Infra
Languages
Earlier systems work
Before Baidu, I worked on cloud workload characterization and CPU architecture at Huawei Cloud's Shuhai Lab, and on distributed-storage control-plane systems at Huawei Data Storage.
Beijing, China · building systems that stay understandable under load