Which API for general real-time LLM agents [D]

TL;DR what is the right interface for writing custom real-time LLM agents? I've noticed many projects (academic, startups, pet, etc) are trying to make LLMs work in real-time world where things happen while the model is thinking.