Splat World: Exploring New Dimensions of Gaussian Splatting in VR

The Splat World: Exploring New Dimensions of Gaussian Splatting in VR

How one developer turned Gaussian splats into a new language for building interactive worlds.
A case study with Marble Labs

Overview

VR developer Daniel Skaale wanted to test how far Marble’s Gaussian splats could go inside a real-time engine.
His goal was to explore how splat-based 3D scenes could be used not only for environment design but also for lighting, animation, physics, and interactive effects.

The result is Splat World, a VR first-person experience built in Unity using environments generated and composed in Marble. The first scene — a realistic house — and the second — a colorful, surreal “splat world” — were both created inside the Marble Composer tool.

Despite sharing the same workflow, the two environments represent opposite ends of Marble’s creative range: one grounded in photorealism, the other bursting with abstract, fantastical color and form.

Inside Unity, Daniel developed a suite of custom C# tools that allowed him to interact with Marble’s splats directly — adjusting their lighting, density, and physical behavior in real time.

Players can move through, collide with, and even manipulate the splats as dynamic materials, transforming them from static render data into responsive, living elements of the experience.

Reimagining the Creative Process

Daniel began by using Marble Composer to build his game's environments. The goal was to test how multiple generated rooms could be aligned, lit, and optimized for VR navigation.

Once the structure was complete, he focused on transforming Gaussian splats into interactive materials that could respond to movement and physics inside Unity.

To make this possible, Daniel built a custom C# toolbox capable of manipulating splats in real time — changing their lighting, triggering animations, and generating particle-like reactions to user interaction.
This allowed him to explore splats not just as visual data, but as an expressive, interactive medium.

The experience was developed and tested primarily on a Meta Quest 3 headset tethered to a computer, ensuring high performance and accurate rendering of Marble’s splat data.

He also conducted additional testing on the standalone Quest 3, verifying that the environments maintained stability and visual fidelity even without PC tethering — an important benchmark for real-world VR performance.

Technical Workflow

Daniel’s workflow focused on integrating Marble data into Unity while maintaining high performance and interactivity in VR.

• Scene Generation: Base environments were generated in Marble using concept images from Midjourney, then refined using Marble’s pano editing tool.
• Composition: The Marble Composer tool was used to stitch, align, and light multiple rooms into a cohesive enviornment.
• Splat Interaction Tools: Daniel developed a set of C# scripts that gave splats physical properties — allowing them to react to player presence, respond to collisions, and animate dynamically.
• Splat VFX: Gaussian splats were used for real-time effects such as impact bursts, energy waves, and dissolving surfaces. Players could touch or shoot at surfaces and watch the splats ripple or break apart.
• Lighting and Animation: The tools also enabled real-time relighting and motion behaviors, giving each splat the ability to shift density, color, or transparency during gameplay.
• Optimization: Each environment was structured around a 10×10 meter grid to balance frame rate and fidelity. GPU profiling kept splat counts within limits, maintaining smooth performance on both tethered and standalone Quest 3 setups.

This workflow turned Gaussian splats into an interactive design layer — one that merged visuals, physics, and gameplay systems in a single, unified process.

Creative Outcomes

Splat World demonstrated that Gaussian splats can serve as more than just a visual technique — they can become interactive, physics-driven components of a game world. With Marble, Daniel could generate complex 3D environments quickly, then extend their functionality through Unity’s real-time systems.

By combining Marble with his own custom interaction framework, Daniel developed a workflow that made splats responsive, destructible, and dynamic — behaving more like matter than pixels.

This opened up new creative possibilities for prototyping gameplay mechanics, building reactive environments, and experimenting with the relationship between player movement and visual form.

The contrast between the detailed house and the surreal splat dimension highlighted Marble’s flexibility.
A single generation pipeline could produce both realistic and abstract worlds — all fully interactive and built for real-time exploration.

Looking Ahead

Daniel is continuing to refine Splat World and expand his research into real-time Gaussian splatting inside Unity.

His next focus areas include:

• Increasing splat density for larger, more detailed environments
• Developing new shader systems for relighting, transparency, and destruction effects
• Testing Marble-generated scenes in both VR and AR contexts for future client and research projects

He’s also improving his custom C# splat toolbox to handle larger data sets and enable faster iteration directly inside Unity.

These updates aim to make splat-based workflows more practical and accessible for game developers, interactive artists, and researchers working with real-time generative worlds.

For Daniel, Splat World is both a creative experiment and a glimpse into a new kind of interactive worldbuilding — where splats don’t just form the scene, they become part of the experience.

Explore Marble worlds in VR →