MuJoCo-GS-Web

Realistic robot physics inside Marble worlds, running in any browser, even on a phone.

Built by Gaotian Wang, a PhD candidate at Rice University working on robot manipulation and world models, MuJoCo-GS-Web brings MuJoCo physics simulation into photorealistic Gaussian splat scenes on the web. Gaotian is also the creator of XLeRobot, an open-source dual-arm home robot with more than 5,500 stars on GitHub.

The goal: make realistic robot simulation as easy to open as a web page, on any device.

Highlights

• MuJoCo physics running live inside Gaussian splat scenes
• Runs entirely in the browser, on any device, including phones
• Drag the MuJoCo dummy, teleoperate XLeRobot and a Franka arm, or run Unitree G1 motion policies
• Scenes and optional collision meshes imported straight from Marble

How It Works

Load any MuJoCo robot file (.xml) together with a Spark-supported splat scene. MuJoCo, compiled to WebAssembly, runs the physics in the browser while Spark renders the Gaussian splat world around the robot.

Marble supplies the scenes, either generated from scratch or reconstructed from real spaces, along with optional collision meshes so the robot can interact with its surroundings. Collision meshes can also be tuned by hand.

The idea came from the releases of mujoco-wasm and Spark, which together made browser-based simulation in realistic scenes possible. One surprise along the way: Spark blends the splats so naturally that parts of the scene can sit in front of the robot, so it feels like part of the world rather than layered on top.

Tech Stack

• World Labs / Marble — Scene generation and collision meshes
• Spark — Gaussian splat rendering in the browser
• MuJoCo WASM — In-browser physics simulation
• XLeRobot, Franka arm, Unitree G1 — Simulated robots

What’s Next

Bringing online reinforcement learning training to the browser and adding more robots, such as the Open Duck Mini.