engineering

5 stories

Stories

Inside Action Gateway in Managed Agents
Understand how Action Gateway in DigitalOcean Managed Agents validates tool calls, applies approval policies, and keeps credentials away from the model.
DigitalOcean BlogOct 13, 202612 min read
Under the Hood: Serving Kimi K3
How DigitalOcean served Kimi K3 on day 0: GPU selection (B300/MI350X), vLLM tuning, and the OpenAI-compatibility fixes for tool calling, streaming, and reasoning.
DigitalOcean BlogJul 30, 202614 min read
The Inference Alpha: Maximizing Frontier Models on AMD
Helping millions of developers easily build, test, manage, and scale applications of any size - faster than ever before.
DigitalOcean BlogJun 10, 202613 min read
The Inference Tax: How Prefix-Aware Routing Eliminates the Hidden Cost of LLMs at Scale
vLLM prefix caching and prefix-aware KV cache routing eliminate redundant LLM prefill costs at scale. Learn how DigitalOcean Serverless Inference reduces GPU compute waste by up to 4x on the same hardware.
DigitalOcean BlogJun 1, 202615 min read
DigitalOcean Serverless Inference: A Deep Dive
A deep dive into DigitalOcean Serverless Inference—how we built a scalable, API-first AI inference platform and the architecture powering it.
DigitalOcean BlogJun 1, 202616 min read