Reka AI's omni-model Rho-1 handles text, images, video, and robot control in a single model
Reka AI's Rho-1 is a 19-billion-parameter omni-model that processes and generates text, images, video, and robot control actions in a single neural network. Trained on 320 H100 GPUs in about three months, it uses a fraction of the compute today's top models need.
Reka AI has introduced Rho-1, a 19-billion-parameter omni-model capable of processing and generating text, images, video, and robot control actions within a single neural network.
What happened
Reka AI has released a research preview of Rho-1, an omni-model that processes and generates text, images, video, and robot control actions in a single neural network. The model uses a shared context window for all modalities and was trained on 320 H100 GPUs over three months. It generates real-time video and adapts to new instructions without restarting. Rho-1's weights predict camera images and drive robot movements through an inverse dynamics model, leveraging internet videos for robot training data.
Why it matters
Rho-1 represents a significant advancement in multimodal AI, offering a unified approach to handling diverse data types. Its ability to process multiple modalities in a single model could streamline AI development and improve efficiency in applications requiring real-time processing and adaptability.
- Rho-1 is a groundbreaking omni-model that processes multiple modalities in a single network.
- The model's real-time video generation and adaptability offer practical advantages for AI applications.
- Reka AI's use of an inverse dynamics model addresses the challenge of scarce robot training data.
The bigger picture
Rho-1's unified approach to multimodal processing may reduce the complexity of AI systems by eliminating the need for specialized models. Its real-time capabilities and adaptability could enhance applications in robotics, video generation, and interactive AI systems.
Reka AI's Rho-1 is a significant step forward in multimodal AI, demonstrating the potential of unified models to handle diverse data types efficiently and effectively.
Technical details
- 19-billion-parameter omni-model
- Shared context window for all modalities
- Inverse dynamics model for robot control
- Real-time video generation
- Training on 320 H100 GPUs
FAQ
What is Rho-1?
Rho-1 is a 19-billion-parameter omni-model developed by Reka AI that processes and generates text, images, video, and robot control actions in a single neural network.
How does Rho-1 handle multiple modalities?
Rho-1 processes all modalities as tokens in a shared context window without using external models or tool calls, allowing it to handle text, images, video, and robot control actions in one network.
What is the significance of Rho-1's real-time video generation?
Rho-1's ability to generate continuous video in real time and adapt to new instructions without restarting enhances its practical applications in areas like robotics and interactive AI systems.
How was Rho-1 trained?
Rho-1 was trained on 320 H100 GPUs over approximately three months, using a fraction of the compute required by today's top models.
What is the inverse dynamics model used for?
The inverse dynamics model in Rho-1 pulls control signals from ordinary internet videos to address the challenge of scarce robot training data, enabling the model to drive robot movements based on camera image predictions.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by The Decoder on Oct 5, 2026.
Comments
Sign in to join the discussion.