Creating Your Own Environment in NVIDIA Isaac Lab and Isaac Sim: A Fundamentals Guide

Today, we'll dive into the universe of NVIDIA Isaac Lab and NVIDIA Isaac Sim, two powerful tools that allow us to create and train AI agents in realistic simulation environments.

If you're already familiar with the NVIDIA Omniverse ecosystem and how Isaac Sim integrates with it – if not, I recommend checking out this NVIDIA Isaac Sim, Omniverse, and Cosmos – The Robotics & AI Simulation Ecosystem Explained, then you're ready for the next step: building your own training environment in NVIDIA Isaac Lab!

In this post, we'll break down the process, using a practical and simple example to understand each piece of the puzzle. Get ready to bring your robotic ideas to life!

NVIDIA Isaac Lab is an open-source framework built on NVIDIA Isaac Sim that facilitates the development and training of robots using Reinforcement Learning (RL). Although it provides a wide range of predefined environments, its true value lies in the ability to customize and create environments that match your specific tasks.

Since Isaac Lab is fully object-oriented, this tutorial assumes you already have a basic understanding of Object-Oriented Programming (OOP). That foundation will make it easier to follow along and build your own interactive environments.

For this example, we'll focus on a simple scenario: teaching an Anymal robot to interact with an obstacle. To find the Anymal robot environment and other available environments check out available environments.

Anymal robot on rough terrain. Taken from NVIDIA Isaac Lab Available Environments

Before We Start: Ground Knowledge and Requirements

Before starting, make sure you have the following dependencies  installed and available on your system:

• Please check System Requirements to make sure you meet the required CPU, OS, Cores, RAM, Storage, GPU and VRAM.

• Linux operating system - Ubuntu 22.04/24.04 is recommended

• Git – required to clone the Isaac Lab repository and manage your own code.

  To install git:

> sudo apt update
> sudo apt install git
> git --version -> this command displays the installed version
confirming that the installation was successful. 

• (Optional) GitHub or GitLab account

RL Frameworks in NVIDIA Isaac Lab

We'll use RSL-RL as our reinforcement learning library, since it is a minimal but powerful PyTorch open-source reinforcement learning library made for teaching robots new skills quickly in simulation and then using them in the real world. Additionally, NVIDIA Isaac Lab is flexible and allows integrating other frameworks. Check this information to find other reinforcement learning scripts.

NVIDIA Isaac Lab is a versatile ecosystem that supports other reinforcement learning frameworks through its "wrappers." This means you can choose the RL library that best suits your needs and preferences. Some of the frameworks that NVIDIA Isaac Lab can integrate include:

• RL-Games: A high-performance RL library from NVIDIA, often used to train complex policies in simulation environments.

• Stable-Baselines3: A collection of robust implementations of RL algorithms in PyTorch, known for its ease of use and good documentation.

The choice of wrapper will depend on the complexity of your task, the desired performance, and your familiarity with each framework. NVIDIA Isaac Lab provides the flexibility to experiment and find the best solution for your project.

RL Environments in NVIDIA Isaac Lab

In this tutorial, we'll focus on Direct RL Environment in Isaac Lab. I might be a mind reader 👀 but what's Direct RL Environment you might be asking...

The NVIDIA Isaac Lab Direct RL Environment is a framework for creating reinforcement learning (RL) environments with direct control over the implementation of reward, observation, and reset functions. This approach allows for more flexibility, because it provides control over the implementation of the environment’s logic.

The NVIDIA Isaac Lab Manager RL Environment is the alternative workflow. Instead of writing reward, observation, and reset logic directly in code, you define them through configuration classes. These classes act as modular building blocks that Isaac Lab manages for you which provides a highly reusable environment. The trade-off is less flexibility compared to Direct RL, because you work within the boundaries of the configuration system.

The Anatomy of your own environment in Isaac Lab

In Isaac Lab, what we usually think of as an environment is technically called a task. For the purposes of this tutorial, we’ll use the term environment for simplicity.

An Isaac Lab environment's basic structure consists of a Python file for the environment and an init.py file within an environment folder. This folder also contains an agent subfolder, which contains its own init.py and a configuration file for the reinforcement learning algorithm.

The diagram demonstrates the file hierarchy within an Isaac Lab environment.

File hierarchy in the Isaac Lab Environment

The following lists the responsibility of each file:

• environment.py: Defines the main environment logic, how to build the world, how the robot behaves, defines observations, rewards calculations and reinforcement learning constants are defined such as episode length.

• init.py: Responsible for registering the environment in Isaac Lab, making it accessible for training.

• agent/configuration_file.py: (configuration file for RL algorithm): specifies details for the reinforcement learning algorithm (RSL-RL, RL-Games, Stable-Baselines3, mentioned in this link) 

• agent/__init__.py: Default file used in existing examples in Isaac Lab repository.

Creating Our First NVIDIA Isaac Lab Environment

Before writing code, we need to clone the Isaac Lab repository and prepare the folder structure for our custom environment.

First, clone the official NVIDIA Isaac Lab repository from GitHub:

> git clone https://github.com/isaac-sim/IsaacLab.git
> $ISAAC_DIR=IsaacLab
> cd $ISAAC_DIR

This gives you access to all the built-in tasks, utilities, and environment templates.

Disclaimer: You do not need to push your changes back to the official Isaac Lab repository. Your work will live locally.

Recommendation: For good practice, keep your custom environments in a separate directory and link them into Isaac Lab when needed. This keeps your work organized and avoids conflicts when updating Isaac Lab in the future.

Create Your Own Repository

From your desire workspace directory (not inside Isaac Lab repository):

> ISAAC_ENV=$HOME/my-isaaclab-envs
> mkdir -p $ISAAC_ENV
> cd $ISAAC_ENV
> git init ->  Initialize it as a git repository

Now create your environment folder, just like it was explained in the Anatomy of Isaac Lab environment section:

> ISAAC_EXAMPLE_PROJECT=$ISAAC_ENV/<REPLACE_EXAMPLE_NAME>
> mkdir -p $ISAAC_EXAMPLE_PROJECT
> cd ISAAC_EXAMPLE_PROJECT
> mkdir agent
> touch env.py __init__.py
> cd agent
> touch __init__.py

At this point the repository should look like this

my-isaaclab-envs/

  └── <REPLACE_EXAMPLE_NAME>/

       ├── env.py

├── __​init__ .py

       └── agent/

├── __​init__ .py

You can commit and push this repo to GitHub or GitLab if you want to keep it online:

> git add .
> git commit -m "Initial commit of my Isaac Lab environment"
> git remote add origin <your_repo_url>
> git push -u origin main

Create a Symbolic Link to Isaac Lab:

It is important to create a symbolic link to integrate your project inside the Isaac Lab repository.

Inside the cloned repository, navigate to the folder that holds all Direct RL environment examples:

> cd $ISAAC_DIR/source/isaaclab_tasks/isaaclab_tasks/direct/

This directory contains sample environments  that you can use as references

Inside the direct/ folder, you can link your external repo there with a symbolic link. From the direct/ directory:

> ln -s $ISAAC_EXAMPLE_PROJECT

Now Isaac Lab sees <REPLACE_EXAMPLE_NAME> as if it were inside the repository, but your code actually lives in your external repo.

Let’s Dive Into Building Our World! 🚀

In this section, we’ll discuss how to design, integrate and register your own code to build your Isaac Lab Environment.

Isaac Lab Architecture. Taken from Isaac Lab Documentation.

In NVIDIA Isaac Lab every environment is built with two components:

• The Configuration Class: It defines all the parameters that describe how the environment should be built and simulated. This includes physics settings, robot setup, terrain, obstacles, sensors and more. 

• The Environment Class: The environment class defines the behavioral logic of the environment. This is where you implement how the agent interacts with the world: rewards, observations and reset. The environment class uses the parameters defined in the configuration class to construct the simulation.

Design the world (env.py)

For our example applies a Direct RL environment therefore we need to set up an environment file (inside the previously created: env.py ) for the Direct Reinforcement Learning Configuration Class and Direct Reinforcement Learning Environment Class.

Configuration Class

Now that we understand the role of configuration classes, let’s create a configuration class. Below is a minimal configuration class for a Direct RL meant for an Anymal interacting with an obstacle.

Understanding the environment’s configuration

Our Configuration class named <Name>EnvCfg must inherit from DirectRLEnvCfg parent class. By inheriting from DirectRLEnvCfg,  it gains access to a predefined structure that Isaac Lab expects for all Direct RL environments. This ensures compatibility with its internal systems for simulation, scene management, and physics.

Explanation of configuration class design for this example:

• Episode_length: maximum duration of an episode in seconds (not steps!! Really important to consider if you are used to Isaac Gym)

• Decimation: Number of simulation steps per agent action steps. What does it actually mean? Well a value of 4 means that for every action the agent sends, the simulation advances 4 times. Which is crucial for the physics engine to simulate at a higher frequency than the agent's decisions. 

• Action_scale: Scale applied to the agent's actions before being sent to the robot. A lower value can make training more stable.

• Action_space: Dimension of the robot’s action space. For this example the robot has 12 joints and each joint has an action. 

• Observation_space: Dimension of the observation space. In this case, it includes the robot's state (position, velocity, joint angles, joint velocities) and depth features from the camera.

• SimulationCfg dt: Simulation time per step 1/200 = 0.005 seconds. This means that 1 seconds is equivalent to 200 steps. Remember the decimation? Well now it makes more sense,  each agent’s step is every 4 simulation steps, in a second it advances 50 steps. 

• SimulationCfg render_interval: Rendering interval equals to decimation so rendering matches the action frequency

• Terrain: TerrainImporterCfg : Terrain configuration, for simplicity a simple terrain is defined. Isaac Lab has a ton of terrain variations worth investigating further.

• Scene num_envs: Number of parallel environments running simultaneously which speeds up the training process.

• Scene env_spacing: Spacing between parallel environments

• Robot ArticulationCfg: Robot configuration. ANYMAL_C_CFG is a predefined configuration for the Anymal C robot.

  • prim_path: Path in the USD stage where the robot will be instantiated. {ENV_REGEX_NS} is a placeholder for each environment's namespace.

• Obstacle RigidObjectCfg: Obstacle configuration (a simple box).

  • prim_path: Path of the obstacle.

  • spawn: How the obstacle is spawned ("default" means the default pose will be used).

  • scale: Scale of the obstacle (0.5, 0.5, 0.5 for a 0.5m side box).

  • mass_props: Mass properties.

  • physics_material: Physics material properties (friction, restitution).

• Contact_sensor ContactSensorCfg: Contact sensor configuration.

  • prim_path: Path where the sensor is attached (in this case, on the robot's base and its parts).

  • filter_prim_paths: Paths of objects with which the sensor will detect collisions (here, only the ground).

  • history_length: Number of simulation steps to store contact history.

• Camera CameraCfg: Camera configuration.

  • prim_path: Camera path.

  • offset: Camera offset relative to its prim_path.

  • width, height: Camera image resolution.

  • horizontal_fov: Horizontal field of view.

  • far_plane, near_plane: Distances of the camera clipping planes.

Environment Class

After defining the configuration, the next step is to implement the environment class in the same python file (env.py), which defines the behavior of the simulation.

Below is the implementation of our example class <Name>ObstacleEnv, which defines a minimal Anymal environment with a single obstacle and depth-based perception.

Understanding the environment

In NVIDIA Isaac Lab must inherit from DirectRLEnv parent class. DirectRLEnv is an abstract base class that defines the interface for RL environments, requiring you to implement specific methods like designscene, updatesimulation, getobservations, getrewards, and isdone. 

 Key methods on <Name>ObstacleEnv Class:

• __​init__ method: Initializes the environment, loads the robot, obstacle, and sensors, and adds them to the scene. It also initializes the obstacle's position and internal states.

• _​design​_​scene: Returns the scene configuration for the environment.

• _update​_​simulation: Applies the agent's actions to the robot and steps the simulation.

• _​get​_​observations: Collects observations from the environment, including robot state (position, velocity, joint angles, joint velocities) and camera depth features.

• _​get​_​rewards: Computes the reward for the agent at the current step. In this example, forward movement is rewarded, and falling, excessive joint effort, and close obstacle detection are penalized.

• _​is​_​done: Determines if an episode has ended (due to time or if the robot falls/moves too far away).

• _​reset​_​idx: Resets the environment for a specific set of environment IDs, restoring the robot to its initial state and randomly repositioning the obstacle.

  
  

💻 Reinforcement Learning Algorithm 💻

Think of this section as setting up the "brain" that will learn to control our robot. It's where we define how the Reinforcement Learning algorithm will operate. As a disclaimer this code was taken from several previous examples in the NVIDIA Isaac Lab repository which uses the same configuration for RSL-RL. We will simply explain what is going on behind the scenes.

This configuration must be placed inside the agent/ folder of your environment directory.

The folder structure should look like this:

my_example_name/

 ├── env.py

 ├──__​init__.py 

 └── agent/

      ├── __​init__.py 

      └── ppo_runner_cfg.py

Inside the configuration file (ppo_runner_cfg.py), you will define the following policy:

Understanding the configuration file

The policy configuration is handled by <Name>PPORunnerCfg. This is where we define the structure of the robot's "decision-making" neural networks. These networks are split into two main parts: the "actor" and the "critic." The actor is like the robot's action planner; it decides what moves to make. Its actor_hidden_dims define the complexity of its internal thought process, with layers of 256, 128, and 64 "neurons" helping it process information. The critic, on the other hand, is like the robot's internal judge; it evaluates how good a particular situation is. Its critic_hidden_dims similarly define its internal structure for this evaluation. The init_noise_std introduces a bit of randomness to the robot's initial actions, encouraging it to explore different behaviors at the beginning of training. The activation refers to the mathematical function used within these neural networks, helping them learn complex patterns.

Finally regarding the algorithm section:

• RslRlPpoAlgorithmCfg: dictates the specific rules for how the robot's brain actually learns and updates itself based on its experiences.

• value_loss_coef and use_clipped_value_loss: about how much we care about the critic's judgment and how we prevent it from making overly drastic changes.

• clip_param: a core part of PPO that ensures the robot doesn't change its behavior too radically in one go, keeping the learning stable.

• entropy_coef: encourages the robot to try new things and not get stuck doing the same actions repeatedly.

• num_learning_epochs and num_mini_batches: define how many times we go over the collected experience and how we break it down into smaller chunks for efficient learning.

• learning_rate: a crucial setting that controls how big of a "step" the robot takes when updating its knowledge; a smaller step means slower but often more stable learning.

• schedule: "adaptive" means that this learning rate can change dynamically as the robot learns.

• Gamma and lam: parameters that help the robot think about future rewards and how much they matter compared to immediate ones.

• desired_kl: another mechanism to control how much the robot's behavior changes, ensuring it doesn't deviate too much from its previous understanding.

• max_grad_norm: a safety net that prevents the learning process from going haywire due to very large updates.

Together, these configurations create a learning environment that allows our robot to explore, make decisions, receive feedback, and gradually improve its ability to navigate and avoid obstacles in the simulated world.

🧾 Registering the Environment 🧾

Registration is done inside a file named __​init__.py, located inside the my_example_name/ folder. As small and simple as this file might look like, it is crucial to tell Isaac Lab how to find and load our new environment.

my_example_name/

 ├── env.py

 ├── __​init__.py 

 └── agent/

      ├── __​init__.py

      └── ppo_runner_cfg.py

Here’s an example implementation of the __​init__.py, file:

Understanding the registration

The function gym.resgister (...) registers our environment with the Gym registration system, allowing it to be loaded by its id.

The id is a unique identifier of our environment, entry_point is the path to the file and class that implemented the environment, disable env checker flag disables internal checkers of gym which are not necessary in the development. 

Finally, kwargs are the key arguments that bind the environment, which include the following:

• env_cfg_entry_point: This is the environment python file we have created in the past sections. 

• rsl_rl_cfg_entry_point: Path to the RSL-RL runner configuration. 

• Any additional reinforcement learning configuration file needs to be added to kwargs.

Tip 💡: Initialization (init) files are crucial for successful integration. If you encounter issues such as a missing window, "file not found" errors, or the system getting stuck during execution, your first suspect should be the init file. Double-check that it is properly configured and correctly referenced.

Hands on! Commands to train and execute 🤖

How do we integrate everything? Next, we will discuss the primary commands you'll use to interact with your custom Isaac Lab environment. 

At this stage, we’ve defined all the necessary components:

• The configuration class

• The environment class

• The agent configuration (agent/ppo_runner_cfg.py)

• And the registration file (__init__.py)

All Isaac Lab commands must be executed from the root directory of the repository you cloned earlier.

If you followed the previous steps, just run:

> cd $ISAAC_DIR

If you execute the following command:

> ls

You should see folders such as source/, scripts/, and logs/ in this directory.

From here, you can train or test your agent.

To train your agent using the RSL-RL algorithm.

You will use a command structured as follows in the root directory of the repository:

> ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=<id>--num_envs 32 --enable_cameras --checkpoint logs/rsl_rl/<env name>/<date and hour>/model_<number>.pt

In this command,

• ./isaaclab.sh: is the main script to launch Isaac Lab.

• -p scripts/reinforcement_learning/rsl_rl/train.py: is an argument that points to the specific Python script responsible for initiating the RSL-RL training process.

• --task=<id>: flag specifies the unique identifier of your custom environment that we registered in the init.py file. 

• --enable_cameras: flag ensures that the camera sensors defined in your environment configuration are active during training, which is crucial for the depth observations used by our agent (side note: this might not apply if you choose to integrate a different sensor). 

• num_env: flag specifies how many environments run in parallel during training, which accelerates the learning process by collecting more experience in less time. 

• --checkpoint logs/rsl_rl/<env name>/<date and hour>/model_<number>.pt: allows you to resume training from a previously saved model checkpoint. If you are starting a new training run, you can omit this flag.

To test your agent using the RSL-RL algorithm.

When you want to see your trained agent in action, use a command similar to the following inside the root directory of the cloned IsaacLab repository:

> ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task=<id> --enable_cameras --checkpoint logs/rsl_rl/<env name>/<date and hour>/model_<number>.pt  --num_envs 1 --video --video_length 3000

Same as before ./isaaclab.sh is the launcher. The -p scripts/reinforcement_learning/rsl_rl/play.py argument directs Isaac Lab to the script designed for running trained policies. As before, --task=<id> identifies your environment.

New arguments to test the agent are:

• --video: flag tells Isaac Lab to record a video of the simulation. 

• --video_length 3000: specifies the duration of the recorded video in agent steps. Remember when we discussed the dt and decimation variables inside the environment’s configuration class? In our case, dt = 1/200 and decimation = 4. Which means every 4 steps of simulation the agent takes 1 step. Which means, each second the simulation makes 200 steps and the agent makes 50 steps. Since video_length equals 3000 agents steps, in seconds is equivalent to 3000/50 = 60 seconds. Therefore, executing the test command will generate a one minute video.

All training and playback logs, including saved models, are stored under:

logs/rsl_rl/<env_name>/

Each run automatically creates a new subfolder with a timestamp, allowing you to track generated videos and checkpoints easily.

Happy training!

Creating a custom environment in NVIDIA Isaac Lab with NVIDIA Isaac Sim is like having a blank canvas to start creating robots and AI. We hope this guide has given you inspiration and guidance to start building your own simulation worlds. Happy training!