Chapters on building an LLM inference stack in Rust — tensors through an OpenAI-compatible server.https://inference.yashkothari.dev/en-ushttps://inference.yashkothari.dev/post/act-i-01-gguf/https://inference.yashkothari.dev/post/act-i-01-gguf/I.1: GGUFhttps://inference.yashkothari.dev/post/act-i-02-tokenizer/https://inference.yashkothari.dev/post/act-i-02-tokenizer/I.2: Tokenizerhttps://inference.yashkothari.dev/post/act-i-03-tensor/https://inference.yashkothari.dev/post/act-i-03-tensor/I.3: Tensorhttps://inference.yashkothari.dev/post/act-i-04-backend/https://inference.yashkothari.dev/post/act-i-04-backend/I.4: Backendhttps://inference.yashkothari.dev/post/act-i-05-qwen3-forward/https://inference.yashkothari.dev/post/act-i-05-qwen3-forward/I.5: Qwen3 forwardhttps://inference.yashkothari.dev/post/act-i-06-greedy-generate/https://inference.yashkothari.dev/post/act-i-06-greedy-generate/I.6: Greedy generationhttps://inference.yashkothari.dev/post/act-ii-01-benchmark-harness/https://inference.yashkothari.dev/post/act-ii-01-benchmark-harness/II.1: Benchmark harnesshttps://inference.yashkothari.dev/post/act-ii-02-kv-cache/https://inference.yashkothari.dev/post/act-ii-02-kv-cache/II.2: KV cachehttps://inference.yashkothari.dev/post/act-ii-03-simd-cpu/https://inference.yashkothari.dev/post/act-ii-03-simd-cpu/II.3: SIMD CPU backendhttps://inference.yashkothari.dev/post/act-ii-04-multithreaded-cpu/https://inference.yashkothari.dev/post/act-ii-04-multithreaded-cpu/II.4: Multithreaded CPU backendhttps://inference.yashkothari.dev/post/act-ii-05-metal-gpu/https://inference.yashkothari.dev/post/act-ii-05-metal-gpu/II.5: Metal GPU backendhttps://inference.yashkothari.dev/post/act-ii-06-q8-0-quantization/https://inference.yashkothari.dev/post/act-ii-06-q8-0-quantization/II.6: Q8_0 quantizationhttps://inference.yashkothari.dev/post/act-iii-01-chat-pipeline/https://inference.yashkothari.dev/post/act-iii-01-chat-pipeline/III.1: Chat pipelinehttps://inference.yashkothari.dev/post/act-iii-02-http-api/https://inference.yashkothari.dev/post/act-iii-02-http-api/III.2: HTTP APIhttps://inference.yashkothari.dev/post/act-iii-03-sse-streaming/https://inference.yashkothari.dev/post/act-iii-03-sse-streaming/III.3: SSE streaminghttps://inference.yashkothari.dev/post/act-iii-04-paged-kv-cache/https://inference.yashkothari.dev/post/act-iii-04-paged-kv-cache/III.4: Paged KV cachehttps://inference.yashkothari.dev/post/act-iii-05-radix-prefix-cache/https://inference.yashkothari.dev/post/act-iii-05-radix-prefix-cache/III.5: Radix prefix cachehttps://inference.yashkothari.dev/post/act-iii-06-decode-scheduler/https://inference.yashkothari.dev/post/act-iii-06-decode-scheduler/III.6: Decode schedulerhttps://inference.yashkothari.dev/post/act-iii-07-batched-decode/https://inference.yashkothari.dev/post/act-iii-07-batched-decode/III.7: Batched decode