MirroS Open-Sources AgentGarten, Where AI Agents Learn by Playing in Code Worlds Rendered Live

What happened

MirroS released AgentGarten, an open-source framework that creates interactive AI learning environments by combining code-based physics simulation with real-time neural rendering, enabling agents to learn through iterative gameplay and self-improvement.

Why it matters

This framework provides a controlled, scalable environment for training AI agents in physical tasks, advancing research in reinforcement learning and embodied AI without requiring physical hardware.

• Separating physics simulation from rendering allows greater control over AI training environments
• Real-time neural rendering enables continuous agent interaction and learning
• Adversarial Forcing improves long-term stability in simulated environments
• Open-sourcing such frameworks accelerates AI research collaboration
• AgentGarten demonstrates rapid learning in complex tasks through iterative playbooks

The bigger picture

The framework enables more efficient and transparent AI training for physical tasks, potentially accelerating research in autonomous systems and simulation-based learning.

MirroS's AgentGarten provides a novel open-source platform for AI agents to learn through interactive simulations, combining code-driven physics with real-time rendering to create scalable, transparent training environments.

Technical details

• Code-defined physics engine handling geometry, contact, and game rules
• Neural renderer converting depth/surface normal data into realistic video frames
• Adversarial Forcing training with gradient replay and real-video discriminators
• Triton kernels and CUDA graphs for real-time performance optimization
• First-person frame observation with Python-based agent decision-making

FAQ