RunInfra

7 stories

Stories

Topic: RunInfra

RunInfra is now inference for agents
RunInfra now focuses on hosted inference for agents, with pay-as-you-go Model APIs and setup guides for Claude Code, Codex and other coding agents.
RunInfra BlogSep 26, 20261 min read
With software alone, one B200 beats the LPU and gets close to Cerebras
Stock SGLang served gpt-oss-120b at 411 tokens per second on one rented B200. Four settings took the same card and the same weights to 1,366, and single prompts to 2,215. No kernel written, nothing recompiled.
RunInfra BlogAug 6, 202617 min read
Lossless Inference
How to make LLM serving faster without touching the model. Exact kernels, speculative decoding, lossless compression, KV reuse and scheduling, with the math and how to verify it with logit parity.
RunInfra BlogAug 3, 20268 min read
The fastest way to serve DeepSeek V4 Flash
Historical measurements published August 2, 2026 for DeepSeek V4 Flash 0731 on four B200 GPUs. As of September 21, 2026, no current measured optimization package is published for it.
RunInfra BlogAug 2, 20268 min read
$0.09 and $290.12: What Actually Moves Your Inference Bill
Nine cents and $290.12 are both prices for one million output tokens. The gap is driven by accelerator, provider, concurrency, utilization, and reasoning token volume.
RunInfra BlogJul 31, 202610 min read
Serving Kimi K3 on vLLM was hard. Here is what we measured.
No released vLLM build served Kimi K3 when we started. We resolved several distinct failures on 8x B300 and measured the result, its operating crossover, and its honest limits.
RunInfra BlogJul 30, 202610 min read
vLLM vs SGLang vs TensorRT-LLM: a serving benchmark
Self-reported Llama-3.1-8B results for three LLM inference engines. Published unique-prompt throughput and latency rows cover H100 bf16. FP8 and L40S appear only in derived cost results. The harness and raw data are not publicly available.
RunInfra BlogJun 20, 20269 min read