Sample Factory

High-throughput reinforcement learning codebase. Resources:

• Paper: https://arxiv.org/abs/2006.11751

• Discord: https://discord.gg/BCfHWaSMkr

• Twitter (for updates): @petrenko_ai

What is Sample Factory?

Sample Factory is one of the fastest RL libraries focused on very efficient synchronous and asynchronous implementations of policy gradients (PPO).

Sample Factory is thoroughly tested and used by many researchers and practitioners. Our implementation is known to reach state-of-the-art (SOTA) performance across a wide range of domains, while minimizing the required training time and hardware requirements. Clips below demonstrate ViZDoom, IsaacGym, DMLab-30, Megaverse, Mujoco, and Atari agents trained with Sample Factory:

Key features

• Highly optimized algorithm architecture for maximum learning throughput
• Synchronous and asynchronous training regimes
• Serial (single-process) mode for easy debugging
• Optimal performance in both CPU-based and GPU-accelerated environments
• Single- & multi-agent training, self-play, supports training multiple policies at once on one or many GPUs
• Population-Based Training (PBT)
• Discrete, continuous, hybrid action spaces
• Vector-based, image-based, dictionary observation spaces
• Automatically creates a model architecture by parsing action/observation space specification. Supports custom model architectures
• Library is designed to be imported into other projects, custom environments are first-class citizens
• Detailed WandB and Tensorboard summaries, custom metrics
• HuggingFace 🤗 integration (upload trained models and metrics to the Hub)
• Multiple example environment integrations with tuned parameters and trained models

Next steps

Check out the following guides to get started:

• Installation
• Basic Usage