My GPU Was Busy. I Finally Found Out Who Was Using It.
The Local LLM Lab — Part 10 One controlled experiment. One number. One verdict. Every homelab has a moment like this: the GPU is at 100%, the fan is screaming, and you have no idea which of the twenty services on the box is doing it.
This article details a homelabber's experiment to identify GPU usage patterns in a Docker-based AI environment, focusing on caller-level attribution to diagnose unexpected GPU utilization.
What happened
The author developed a GPU monitoring system that tracks usage at the caller level, revealing that their AI assistant core was the primary GPU consumer (47% of usage) over seven days, followed by observability tools (17%) and CI processes (3.4%).
Why it matters
For AI developers and homelab operators, understanding GPU usage patterns helps optimize resource allocation, identify unexpected workloads, and improve system efficiency in shared environments.
- Traditional GPU monitoring tools lack granularity for Docker-based AI workloads.
- Caller-level attribution reveals hidden GPU usage patterns in shared services.
- AI assistants and observability stacks can consume significant GPU resources unexpectedly.
- Implementing caller tagging requires minimal overhead but offers substantial diagnostic value.
- Optimizing homelab GPU usage requires visibility into both direct and indirect workloads.
The bigger picture
The method provides a practical solution for diagnosing GPU bottlenecks in complex AI environments, enabling more precise resource management and troubleshooting.
The article demonstrates how caller-level GPU monitoring exposes hidden usage patterns in AI homelabs, helping users optimize resource allocation and understand the true drivers of GPU utilization.
Technical details
- Caller-level GPU attribution: Tags GPU samples with the originating service/requestor, not just container processes.
- Docker-based service architecture: Includes Whisper ASR, Ollama embeddings, and observability tools like Langfuse.
- Three-layer monitoring: Card-level metrics (utilization, VRAM), service-level container tracking, and caller-level request attribution.
- Low-overhead implementation: Caller tags are set once per request, minimizing performance impact.
FAQ
What caused the GPU to be busy in the author's homelab?
The GPU was primarily used by the author's AI assistant (47% of usage) for audio transcription, followed by observability tools (17%) and CI processes (3.4%).
How does caller-level attribution differ from traditional GPU monitoring?
Traditional tools like nvidia-smi show container processes, while caller-level attribution tracks GPU usage back to the originating service or request, providing more actionable insights.
What are the benefits of implementing caller-level GPU tracking?
It enables precise identification of GPU consumers, reveals unexpected workloads (like CI or observability tools), and helps optimize resource allocation in shared AI environments.
Can this approach work for all AI workloads?
It works best for request-driven services. For in-process workloads (e.g., training scripts), service-level tagging is the alternative, though less granular.
What is the main takeaway for homelab users?
GPU utilization spikes often have specific, identifiable causes. Implementing caller-level tracking transforms dashboards into diagnostic tools, avoiding unnecessary hardware upgrades.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by Dev.to AI on Oct 12, 2026.
Comments
Sign in to join the discussion.