Ai2 Introduces ToyBlocks Simulation Framework for Robot Learning and Evaluation

Ai2 announced FoundationWorlds, a framework for controlled simulation environments designed to study robot learning and trace the causes of failure. Its first implementation, ToyBlocks, centers on a tabletop robot arm manipulating wooden blocks through pushing, flipping, and stacking tasks. To overcome the bottleneck of manual teleoperation, ToyBlocks utilizes reinforcement learning and task-and-motion planning (TAMP) in Nvidia's Isaac Lab to automatically generate short-horizon skills and multi-step demonstration datasets. The system runs hundreds of plans in parallel on a single GPU, achieving up to 24 times real-time simulation speeds. ToyBlocks separates high-level reasoning from low-level motor execution, allowing researchers to evaluate models such as Molmo 2, ManiFlow, and frontier models like Claude connected via the Model Context Protocol (MCP). The controlled setup enables systematic studies on spatial generalization, such as testing action models on unseen workspace regions or block configurations, while providing detailed performance metrics beyond simple task success rates.

Key takeaways

• ToyBlocks is an open simulation testbed built on Isaac Lab that generates skills and task demonstrations automatically without requiring human teleoperation.
• Data generation runs hundreds of plans in parallel on a single GPU, reaching data collection speeds up to 24 times real-time.
• The framework enables researchers to test low-level action model generalization across unseen spatial regions, heights, and block variations.
• Hierarchical setups pair high-level vision-language models or frontier agents with low-level policies, isolating whether failures stem from planning or physical execution.