Spatial computing experiences are transforming how we interact with data, connecting the physical and digital worlds through technologies like extended reality (XR) and digital twins. These advancements are enabling more intuitive and immersive ways to analyze and understand complex datasets.
This post explains how developers can now engage with Universal Scene Description (OpenUSD)-based NVIDIA Omniverse datasets in immersive environments, bringing new levels of understanding and insights.
A unique hybrid rendering solution
Creating immersive spatial experiences poses unique challenges. Often, XR devices cannot render complex datasets at the level of detail needed for realistic and engaging experiences. Hardware limitations can result in reduced fidelity, making it difficult to preserve intricate details. To address these challenges, NVIDIA announced at CES 2025 that the Omniverse platform now includes the spatial streaming for Omniverse digital twins workflow that enables XR experiences with unmatched visual fidelity and performance. Partners Accenture, Innoactive, and Katana have begun to integrate this workflow into their existing spatial workflows.
This hybrid rendering workflow combines local and cloud-based rendering using NVIDIA RTX GPUs and NVIDIA Graphics Delivery Network (GDN). This approach ensures high-quality immersive experiences while reducing the strain on local devices. Developers can now stream detailed, high-fidelity OpenUSD-based Omniverse digital twins to Apple Vision Pro.
Figure 1. The spatial streaming for Omniverse digital twins workflow enables XR experiences with unmatched visual fidelity and performance
Setting up spatial streaming
Follow the steps below to build your own hybrid rendering application and stream immersive experiences using the Omniverse platform and Apple Vision Pro.
1. Create an Omniverse Kit Application
Begin by creating an Omniverse Kit Application. This will act as the server for your dataset, handling computationally intensive rendering tasks.
2. Add XR extensions
Enable spatial streaming by including the XR extensions bundle. This simplifies setup and ensures all necessary components are included.
3. Build and run the Kit Application
Build and run the Kit Application to verify it processes and streams data correctly.
4. Set up the client project
Open Xcode on your Mac and configure the Client project. This will render the final visuals on Apple Vision Pro.
5. Test with Apple Vision Pro simulator
Use Apple Vision Pro simulator in Xcode to test and debug without needing a physical headset.
6. Configure the sample project
Adjust the provided sample project to work with your specific dataset. The default configuration uses a Purse USD dataset as a reference.
7. Connect to the Omniverse Server
Finally, link your Client project to the Omniverse Server. Communication between the two is managed through ActionGraph Logic in Omniverse.
Key considerations for setting up spatial streaming
There are several components needed in order to set up your spatial streaming environment, detailed below.
Kit Application as the server: Handles rendering tasks to reduce local hardware strain, ensuring smooth operation.
XR extensions: Simplifies development and minimizes setup errors.
Build and run early: Identifies potential issues in the server setup before proceeding.
Xcode client project: Manages seamless communication between the server and local device.
Simulator testing: Speeds up development by eliminating dependency on physical hardware.
Sample project: Provides a robust starting point to customize for specific datasets.
Best practices for performance
Optimizing rendering
Ensure optimal performance by minimizing latency and optimizing network conditions. This maximizes the responsiveness and quality of spatial experiences.
Lighting and content
Adhere to best practices for lighting and content design to achieve visually stunning, realistic environments that enhance user engagement.
Figure 2. View of a wind turbine through Apple Vision Pro, powered by NVIDIA Omniverse. Image courtesy of Innoactive
Benefits of hybrid rendering
Streaming immersive experiences with the NVIDIA hybrid rendering workflow delivers significant advantages:
High-fidelity viewing: Maintain full dataset fidelity without compromising quality. Now a large, enterprise dataset can be rendered with intricate detail and streamed to the high resolution displays of the Apple Vision Pro.
Performance and responsiveness: Cloud GPUs can render more complex datasets, handle more AI workflows, and scale to reach many simultaneous users.
Cost and energy savings: Leverage scalable cloud resources to reduce costs and energy consumption.
Accessibility: Enable immersive digital twin experiences without requiring expensive, powerful workstations. This democratizes access to high-quality XR applications.
Sustainability: Reduce material waste and production costs by relying on digital models instead of physical prototypes. For example, digital rendering can save up to $9,000 per project.
Leveraging Apple Vision Pro: Fully utilize advanced features like high-resolution displays and spatial audio, delivering unmatched user experiences.
SAGECONNECT TECHNOLOGY GROUP CO., LIMITED
Sageconnect is a reliable provider of IT network system integration and solutions for global enterprise customers.
