Complete Guide to Deploying NVIDIA Isaac Sim on AWS for ROS 2 Robot Simulation and Data Collection

Challenges in Traditional Robot Development

The robotics R&D field has long relied on physical hardware for data collection and algorithm validation. While this approach is intuitive and effective, it has significant limitations: high hardware procurement and maintenance costs, equipment prone to wear or damage during repeated testing, experimental environments constrained by physical conditions such as site, temperature, and lighting, making large-scale parallel testing difficult. For startups or research institutions, these factors often become bottlenecks to innovation. More troublesome is that when algorithms need validation under extreme conditions, physical testing may pose safety risks, and certain edge cases are difficult to reproduce in reality.

The rise of cloud-native robot simulation platforms is changing this landscape. By migrating compute-intensive simulation tasks to the cloud, developers can conduct experiments in safe, repeatable, and scalable virtual environments, breaking free from the various limitations of physical testing environments. The combination of AWS’s powerful GPU computing capabilities with NVIDIA Isaac Sim simulation engine provides a mature technical foundation for this transformation. Isaac Sim, built on the NVIDIA Omniverse platform, delivers physically accurate robot simulation including rigid body dynamics, sensor simulation, and photorealistic visual rendering—features that make it an ideal tool for developing robot perception and control algorithms.

Solution Architecture Design

Overall Architecture Overview

This solution builds a cloud-based Isaac Sim environment on AWS EC2, achieving seamless integration between local robot devices and cloud simulation through a dual-channel architecture. Taking the Lerobot SO-101 robotic arm as an example, developers can perform teleoperation locally while completing high-fidelity simulation and data collection in the cloud. The core advantage of this architecture lies in separating computational load from the operational interface—GPU-intensive physical simulation and rendering tasks are handled by the cloud, while developers only need to interact through a lightweight client.

Core Components

Local environment includes the following elements:

• Lerobot SO-101 physical robot platform, serving as the teleoperation input device
• ROS 2 data collection nodes, responsible for sensor data acquisition and preprocessing
• Developer workstation for coding, debugging, and experiment monitoring
• Amazon DCV client, providing visualized remote access capability with hardware-accelerated 3D graphics transmission

AWS cloud environment consists of the following services:

• EC2 GPU instances running NVIDIA Isaac Sim, hosting the simulation compute core
• Amazon DCV Server, providing high-performance remote desktop access optimized for graphics workloads
• rosbridge-suite, enabling ROS 2 communication with local devices, converting WebSocket protocol to ROS messages
• Amazon S3, storing simulation data, URDF models and related artifacts, supporting version control and lifecycle management

Data Flow Mechanism

Real-time data synchronization: Local SO-101 device collects sensor data through ROS 2 nodes, transmits via WebSocket to cloud rosbridge service, rosbridge bridges the data stream to Isaac Sim, simulation results feed back to local device in real-time, forming closed-loop control. End-to-end latency of the entire chain is typically in the 50-100 millisecond range, completely acceptable for most robotic arm operation tasks.

Remote development access: Developers connect to EC2 workstation through DCV client, gaining complete graphical access to Isaac Sim for real-time simulation debugging and configuration. This approach avoids the performance bottlenecks of traditional VNC solutions in 3D rendering scenarios. DCV employs adaptive encoding technology that dynamically adjusts quality and frame rate based on network conditions, maximizing visual quality while ensuring smoothness.

Data persistence: Training data and model parameters generated during simulation are automatically captured, dedicated data collection modules ensure centralized management and consistency, all artifacts stored in Amazon S3, supporting backup, version control and downstream machine learning workflows. For teams requiring multi-cloud billing payment solutions, this architecture also facilitates unified management of cross-region cloud resource costs, avoiding financial complexity from multiple accounts.

Detailed Implementation Steps

Phase 1: Deploy Isaac Sim on AWS

Launch NVIDIA Isaac Sim Development Workstation

Enter AWS console, navigate to EC2 → Launch Instance → AWS Marketplace AMIs, search for “NVIDIA Isaac Sim”. Select NVIDIA Isaac Sim Development Workstation (Linux) image, configure security groups and launch instance. This AMI comes pre-installed with all dependencies required to run Isaac Sim, including NVIDIA drivers, CUDA toolkit, and Omniverse core components, significantly reducing environment setup time.

Recommended configuration parameters:

• Instance type: g6e.8xlarge (GPU-accelerated), equipped with 32 vCPU, 128GB memory and NVIDIA L40S GPU
• Storage: 500GB EBS gp3, recommend configuring 3000 IOPS to optimize asset loading speed
• Open ports: 22 (SSH), 8443 (DCV remote desktop), 9090 (WebSocket communication)

The g6e series instances feature NVIDIA L40S GPU, excelling in graphics rendering and AI inference, capable of smoothly running Isaac Sim’s complex physical simulation scenarios. If budget is limited, g5 series (with A10G GPU) is also a viable alternative, though may require reduced rendering quality in high-polygon scenes. For initial validation phases, g4dn instances can serve as entry-level options, but are not recommended for production-grade simulation tasks.

Phase 2: Configure ROS 2 Data Collection and rosbridge Service

Set up local development environment: Ensure local workstation has ROS 2 installed (Humble or Iron version recommended), and complete Lerobot SO-101 driver configuration. ROS 2 Humble as an LTS version has longer support lifecycle and more stable package ecosystem, suitable for production environments.

Deploy rosbridge service on AWS: Log into EC2 instance, install rosbridge-suite package, this component will serve as the communication bridge between local ROS 2 network and cloud simulation environment. rosbridge exposes ROS topics and services through WebSocket protocol, enabling non-ROS clients to interact with the ROS ecosystem. After installation, configure rosbridge_server node launch parameters including listening port, authentication policy, etc.

Establish local robot device connection: Connect Lerobot SO-101 robotic arm on local development workstation, execute corresponding device initialization commands. Confirm joint state topics (such as /joint_states) are publishing normally with expected data frequency.

Establish ROS 2 communication bridge: Run forwarding command on local machine to automatically forward Lerobot device USB data to ROS 2 topics on remote EC2 instance. Then log into AWS EC2 instance to verify ROS 2 topics are receiving normally. Use ros2 topic echo command to check data integrity, ensuring joint angles, velocities and other information are not lost.

Phase 3: Configure Amazon DCV Visualization Channel

Connect to Amazon DCV Remote Desktop

Download and install Amazon DCV client from https://www.amazondcv.com/ on local computer. DCV client supports Windows, macOS and Linux platforms, recommend using latest version for best compatibility. Configure connection with following parameters:

• Server: <EC2 Public IP>:8443
• Username: ubuntu
• Password: Ubuntu password configured in step 2

First connection may encounter certificate warning, as DCV Server uses self-signed certificate by default. Production environments should configure trusted SSL certificates to eliminate this warning.

Initialize Isaac Sim Environment

Access remote EC2 instance through DCV client, launch Isaac Sim application. Complete following configuration in Isaac Sim interface:

URDF model import: File → Import → Select ~/Documents/SO-ARM100/Simulation/SO101/so101_new_calib.urdf. URDF file defines robotic arm’s kinematic structure and visual/collision geometry, Isaac Sim will automatically construct corresponding physical simulation model after import.

ROS 2 Bridge extension activation: Window → Extensions → Search “ROS 2 Bridge” → Enable “isaacsim.ros2.bridge”. This extension provides native integration between Isaac Sim and ROS 2 ecosystem, supporting topic publish/subscribe, service calls and Action communication.

Joint state subscription configuration: Tools → Robotics → ROS2 OmniGraphs → Joint States, configure parameters as follows:

• Articulation Root: /World/so101_new_calib/root_joint
• Enable Subscriber option
• Confirm and apply configuration

Topic relay service startup: Open new terminal, run ros2 run topic_tools relay /joint_states /joint_command to convert state messages to command data. This step ensures joint states sent by local device can be correctly parsed by Isaac Sim as control commands.

Execute simulation: Click Play button on left panel of Isaac Sim interface to start simulation. At this point, use local teleoperation device to control robot in remote simulation environment in real-time. During simulation, monitor physics engine performance, rendering frame rate and ROS communication status through Isaac Sim’s built-in tools.

Phase 4: Build Enterprise-Grade AI-Driven Data Pipeline

Building upon the ROS 2–Isaac Sim cloud simulation infrastructure described above, an end-to-end AI-driven robot development workflow can be further established. The goal of this phase is to transform simulation-generated data into trainable machine learning assets and enable continuous model iteration and deployment.

Key AWS service integration solutions:

Amazon S3 – Data Lake Architecture: Leverage S3’s 11 nines durability to build tiered storage, optimize storage costs for large-scale robot datasets through S3 Intelligent-Tiering. Use Standard storage class for frequently accessed training data, automatically migrate historical data to low-cost tiers. Recommend organizing object key prefixes by date and experiment batch for subsequent data retrieval and lifecycle management.

Amazon FSx for Lustre – High-Performance Storage: Provides sub-millisecond latency parallel file system, delivering optimized I/O performance for Vision-Language-Action (VLA) model training. FSx for Lustre can be directly mounted to EC2 instances and seamlessly integrates with S3, supporting transparent data import/export. For training tasks processing large volumes of images and point cloud data, FSx’s aggregate throughput can reach hundreds of GB/s.

Amazon SageMaker – Machine Learning Operations Platform:

• Use SageMaker Processing Jobs for large-scale multimodal data preprocessing, supporting custom containers and distributed processing
• Scale distributed training through Spot instances, reducing training costs by up to 90%, suitable for time-insensitive batch training tasks
• Leverage Model Registry for enterprise-grade version management and A/B testing, tracking model lineage and performance metrics

AWS Batch – Large-Scale Parallel Simulation: Use Spot Fleet to configure cost-optimized GPU clusters, supporting large-scale parallel data generation in Isaac Sim. By defining job queues and compute environments, automatic scheduling and resource management of hundreds of simulation instances can be achieved. Each simulation instance can run different scenario configurations, rapidly accumulating diverse training data.

Amazon Kinesis – Real-Time Data Streams: Enable low-latency ingestion and processing of sensor data, supporting real-time robot telemetry and analytics. Kinesis Data Streams can process millions of records per second, combined with Kinesis Data Analytics for streaming anomaly detection and metric aggregation.

Implementation Considerations and Best Practices

Network Latency Optimization

For real-time teleoperation scenarios, network latency directly impacts control experience. Recommend selecting AWS region closest to development team to deploy EC2 instances, consider using AWS Global Accelerator for cross-region access optimization when necessary. WebSocket connection stability is equally critical, production environments should configure automatic reconnection mechanisms and implement message queue buffering to handle transient network jitter. For latency-sensitive precision operation tasks, consider implementing predictive control algorithms at rosbridge layer to compensate for network round-trip time.

Cost Control Strategies

On-demand pricing for GPU instances is relatively high, for non-continuous workloads, adopt following strategies:

• Use Spot instances for batch simulation tasks, costs can be reduced by 60%-90%, but need to design interruption fault-tolerance mechanisms
• Configure instance auto start/stop scripts based on CloudWatch events or Lambda functions to avoid charges from idle resources
• Leverage Reserved Instances or Savings Plans to lock in long-term usage discounts, suitable for stable development teams
• Regularly review EBS snapshots and S3 storage, clean up expired data to control storage costs

Security Considerations

Security group configuration should follow principle of least privilege, only opening necessary ports. For sensitive robot control data, recommend enabling VPN or AWS PrivateLink for transmission encryption. IAM roles should be finely divided, ensuring different team members can only access resources within their scope of responsibility. Additionally, recommend enabling CloudTrail audit logs to track all API calls and resource changes, meeting compliance requirements.

Simulation Fidelity and Real-World Gap

Although Isaac Sim provides highly realistic physical simulation, an unavoidable gap exists between simulation and real world (Sim-to-Real Gap). Recommend introducing Domain Randomization techniques in simulation, applying random perturbations to lighting, textures, physical parameters, etc., to improve model generalization in real environments. Regularly calibrate simulation parameters using real hardware data to narrow the gap between the two.

Architecture Value and Advantages Analysis

This dual-entry AWS cloud robot simulation architecture combining rosbridge WebSocket data channel and Amazon DCV visualization channel provides an enterprise-ready development environment for robot development.

Cloud-native elasticity: AWS EC2 on-demand scaling capability supports simulation workloads of different scales, dynamic GPU configuration achieves resource utilization optimization, multi-region deployment reduces global access latency. Teams can flexibly adjust resource configuration according to project phase, seamlessly transitioning from prototype validation to large-scale data generation.

End-to-end robot development support: Amazon S3 provides virtually unlimited scalable data storage, seamless integration with AWS AI/ML services accelerates algorithm iteration, cloud full lifecycle supports data generation, training and simulation validation. Developers can complete the entire process from concept to deployment on a unified platform, reducing efficiency loss from tool chain switching.

Lower entry barriers: Eliminates need for expensive hardware investment, enabling more teams to participate in robot innovation. Accelerate development cycles through parallel simulation and cloud collaboration, enterprise-grade infrastructure ensures stability and reliability. Startups can access simulation capabilities comparable to large enterprises without significant upfront capital investment.