ProcTHOR

ProcTHOR enables Embodied AI to scale by orders of magnitude by procedurally generating interactive 3D environments.

ProcTHOR uses procedural generation to sample massively diverse, realistic, interactive, customizable, and performant 3D environments to train simulated embodied agents. Here is an example of sampling virtual home environments.

HOUSE SELECTION

Customizability. ProcTHOR can construct custom scene types, such as classrooms, libraries, and offices.

Results. Pre-training with ProcTHOR improves downstream performance. Zero-Shot performance, from models pre-trained on ProcTHOR, often beats the same models trained on the training data from the benchmark it is evaluated on.

RoboTHOR ObjectNav

Semantic Object Navigation in Dorm Styled, 3D Artist-Designed Scenes

Baseline

trained on RoboTHOR

Zero-shot trained on ProcTHOR

Pre-Trained on ProcTHOR,

fine-tuned on RoboTHOR

Habitat ObjectNav

Semantic Object Navigation in Multi-Floor, 3D Matterport Scanned Scenes

Baseline

trained on Habitat

Zero-shot trained on ProcTHOR

Pre-Trained on ProcTHOR,

fine-tuned on Habitat

AI2-iTHOR ObjectNav

Semantic Object Navigation in Room-Sized, 3D Artist-Designed Scenes

Baseline

trained on AI2-iTHOR

Zero-shot trained on ProcTHOR

Pre-Trained on ProcTHOR,

fine-tuned on AI2-iTHOR

ArchitecTHOR ObjectNav

Semantic Object Navigation in Single-Floor House-Sized, 3D Artist-Designed Scenes

Baseline

trained on AI2-iTHOR

Zero-shot trained on ProcTHOR

AI2-THOR Rearrangement

Interactive Rearrangement in Room-Sized, 3D Artist-Designed Scenes

10

%

Percentage-Fixed Strict

Baseline

trained on AI2-THOR

Zero-shot trained on ProcTHOR

Pre-Trained on ProcTHOR,

fine-tuned on AI2-THOR

ManipulaTHOR ArmPointNav

Interactive Manipulation in Room-Sized, 3D Artist-Designed Scenes

Baseline

trained on ManipulaTHOR

Zero-shot trained on ProcTHOR

Scale Improves Performance. Scaling the number of training houses consistently improves zero-shot performance.

Floorplan Diversity. ProcTHOR houses sample extremely diverse floorplans. Here are examples of sampled floorplans with between 1 and 10 rooms.

...

Object Diversity. ProcTHOR includes 1,633 interactive household objects across 108 categories. A small subset of these objects is shown below.

...

Material Augmentation. ProcTHOR includes 3,278 materials that can be used to visually augment objects, walls, floors, and ceilings.

SCENE VIEW

Lighting Variation. Environment lighting can be rendered with significantly variation to simulate real-world lighting conditions at any time of day.

Interactivity. ProcTHOR objects are highly interactive, supporting object state changes, robotic arm manipulation, and multi-agent interaction.

Get Started. ProcTHOR is fully open-source and available to the Embodied AI community. We are excited to see what you build!

Team. ProcTHOR was created by the PRIOR team at the Allen Institute for AI.