Author a generator.
Agents write and validate category-level procedural code, then sample geometry and appearance variations.
Agentic assets + demonstration generation
SimScale
Generate diverse worlds.
Train robots for unseen geometries.
SimScale turns an object description and one human demonstration into diverse asset families and thousands of successful robot trajectories, so policies handle shapes and poses they were never shown.
Simulation at scale
Explore the pipeline ↓
Start here / Overview
From one example to a world of variation.
Collecting a new demonstration for every object geometry is expensive. SimScale separates the reasoning needed to build reusable tools from the computation needed to generate training data at scale.
Read the overview transcript
- The goal: train skills that handle new shapes without collecting a human demonstration for every geometry.
- AssetEngine: agents author category-level procedural code using coverage and geometry feedback.
- Asset sampling: reuse the generator to produce varied shapes without an agent conversation for every sample.
- DemoEngine: decompose one human demonstration and author an executable task program.
- Geometry adaptation: agents adapt targets and task logic using simulation feedback.
- Pose expansion: DexMimicGen transforms successful source trajectories and validates execution across poses.
- Policy learning: train in simulation; evaluate on held-out simulated and physical objects.
Adapt the skill.
Agents decompose one human demonstration and adapt executable task programs across object geometries.
Expand, then train.
DexMimicGen expands accepted source trajectories across poses. Imitation learning trains policies for held-out objects.
01 / Real-world results
SimScale policies generalize zero-shot to the real world.
Policies trained only in simulation are tested on physical objects without object-specific fine-tuning.
Recorded real-world results
Compare aggregate policy scores across three tasks. These are mean task scores from 0 to 100, including partial credit; they are distinct from simulation task-completion success rates.
| Training condition | Put pot | Hang mug | Put charger |
|---|
† Incomplete object panel: four of five charger combinations in the system study; four common charger pairs in the asset-source study. ‡ Recorded trial lists require reconciliation; the manuscript retains the explicitly recorded values. Unavailable results are not treated as zero.
Source: manuscript real-world results tables · Download the displayed score data
Put pot on cooktop
Hang mug on tree
Put charger in drawer
02 / Asset generation
SimScale generates realistic and diverse simulation assets.
Each task includes multiple object instances with different shapes, sizes, and appearances.
Explore one category at a time.
Choose a family to see its geometry and controls at a readable scale.
Mug trees
Changes the vertical arrangement of hanging locations.
Changes the support footprint.
Changes the approach and hook geometry.
Hands-on / Original generator outputs
Explore the real
procedural geometry.
Move the sliders to explore meshes evaluated from the original SimScale mug and mug-tree Python generators. Each family has 64 exact parameter combinations. Choose another combination or download the selected mesh.
Drag or use arrow keys to orbit · exact mesh at every step
The sliders select a precompiled grid from the original generators. Shapes are not approximated or interpolated in the browser. This preview shows geometry; simulator physics and robot task execution are evaluated separately. Mesh downloads use meters and a Z-up axis.
Generator provenance and exact parameters
Loading source identity…
03 / Demonstration generation
SimScale produces demonstrations on diverse assets and poses.
DemoEngine adapts agent-authored task programs across geometries. DexMimicGen then expands accepted source trajectories across object poses and retains successful executions.
Generated simulation trajectoryPut charger in drawerDexMimicGen pose expansion
Insertion, drawer closure, and release
Insertion, drawer closure, and release
Generated simulation trajectoryHang mug on treeDexMimicGen pose expansion
Hook engagement, release, and resting mug
Hook engagement, release, and resting mug
Generated simulation trajectoryPut pot on cooktopDexMimicGen pose expansion
Placement, release, and resting pot
Placement, release, and resting pot
04 / Simulation evaluation
SimScale policies generalize in simulation.
Compare task-completion success on held-out asset geometries. Each training condition uses 4,000 generated demonstrations per task; demonstration generation is held fixed to isolate the asset source.
30 SimScale configurations per task, a held-out 20% of RoboCasa assets, and five real-object digital-twin pairs.
GR00T N1.6 and DINOv3 + FMT are evaluated across generated families, curated assets, and digital twins.
Success requires completing the manipulation task. Scores below are percentages with reported standard errors.
| Training asset source | Put pot | Hang mug | Put charger |
|---|
Source: manuscript simulation results table · Download the displayed score data. RoboCasa has no charger assets; unavailable comparisons are shown explicitly.
05 / Baseline comparison
How SimScale compares.
SimScale combines category-level procedural assets with geometry-aware task adaptation. The simulation table above compares training asset sources; the selected real-world rollouts below show matched task objects.
Where SimScale extends prior work
Articraft & asset generation
Articraft uses agents to author individual articulated assets. SimScale uses its SDK to author reusable category-level generators, then samples controlled variation and connects it to robot data generation.
Articraft paper ↗RoboCasa & SimFoundry
Curated asset libraries and real-to-sim twins or cousins provide alternative training geometries. SimScale learns from a sampled category distribution; our simulation study compares these sources with the same demonstration budget.
View the measured comparison ↑DexMimicGen
DexMimicGen efficiently expands source demonstrations across poses. SimScale adds agent-authored task programs and geometry-specific source adaptation, then uses DexMimicGen for pose expansion.
DexMimicGen project ↗MolmoBot
MolmoBot scales task-and-motion-planning data for pick-and-place and articulated-object tasks. SimScale studies geometry-aware source generation for bimanual pot placement, mug hanging, and charger placement.
MolmoBot paper ↗These cards describe method scope. The measured evidence is the asset-source study above; no head-to-head benchmark against MolmoBot or DexMimicGen is claimed.
| Method | Source behavior | What scales | SimScale’s added role |
|---|---|---|---|
| DexMimicGen | Supplied source demonstrations | Object-centric trajectory transformation and replay | Agents author and adapt source programs across asset geometries before calling DexMimicGen for pose expansion |
| MolmoBot | Task-and-motion-planning experts | Procedural environments, assets, and expert trajectories for pick-and-place and articulated-object manipulation | Agent-authored, geometry-aware source programs for the demonstrated bimanual manipulation tasks |
| SimScale | One human reference guides agent-authored task programs | Procedural asset sampling, agent adaptation across geometry, and DexMimicGen expansion across poses | Combines flexible source generation with reusable execution tools |
Selected real-world rollouts · 8× playback · illustrative videos, not aggregate success rates
06 / Method
How it works: two engines, one scalable pipeline.
SimScale turns a short asset description and one human demonstration collected in simulation into a training set that spans object geometry and pose.
From prompt to assets
Input
Asset description
→
“Mugs with handles and branched racks”
Process
Agent-authored generator
→
An author agent encodes dimensions, part variants, dependencies, and articulation. A coverage critic and geometry validator guide revisions. Accepted code is reused for procedural sampling.
From demonstration to deployment
Input
One source demo
→
A single human trajectory collected in simulation.
Process
→
# Illustrative pseudocode
program = author_task(source_demo)
for asset in asset_family:
task = agent_adapt(program, asset)
rollout = task.execute()
if rollout.success:
dataset += dexmimicgen(rollout, poses)Agents reuse the task decomposition while adapting targets and logic to new geometry. DexMimicGen expands successful source runs across poses; task checks filter the resulting executions.
Data outputAccepted trajectoriesRGB + robot state + actions
→
TrainVisuomotor policyStandard imitation learning
→
Policy outputZero-shot transferHeld-out simulation + unseen real objects
Inside the engines
AssetEngine
Build a category, not one object.
AssetEngine writes one parameterized procedural generator whose controls describe meaningful variation across an object category.
- SearchCollect diverse real-world examples and identify task-relevant variation.
- AuthorEncode dimensions, part choices, dependencies, joints, and semantic metadata.
- ReviewA coverage critic checks the category range; a geometry validator checks simulation readiness.
- SampleCompile accepted parameter settings and split assets into training and held-out sets.
OutputDiverse, physically plausible simulation assets
DemoEngine
Adapt the skill across geometry and pose.
DemoEngine turns the reference trajectory into a reusable task program, then uses simulation feedback and robot tools to make it work across the asset family.
- DecomposeRead the demonstration as semantic stages such as grasp, handover, insert, and retreat.
- ProgramCompose motion planning, gripper control, collision checking, and object-centric targets.
- SpecializeRevise targets or task logic for each geometry and retain only successful executions.
- ExpandUse DexMimicGen to transform accepted trajectories across sampled object poses.
OutputDiverse, successful demonstrations across assets and poses
Project summary
Diverse assets. Generated demonstrations. One policy tested on objects it has not seen before.