A weekend with the World API
World Labs opened an API for Marble. I spent the weekend building a small workbench for rendering its worlds as Gaussian splats and authoring camera paths through them. What broke: scale, coordinates and drift.
On Wednesday World Labs launched the World API, programmatic access to Marble, the world model I wrote about in November. You send it text, an image, a panorama, several views or a video, and it generates a navigable 3D world you can render on the web or export. I spent most of the weekend building something small on top of it, as a personal project unrelated to my job. Here’s what I learned.
The project is a workbench for one question I care about for video: can you use a generated 3D world as a set, and shoot consistent camera moves through it? It’s a React and TypeScript app that loads Marble worlds and renders them as Gaussian splats with Spark, World Labs’ open-source splat renderer for Three.js. You scrub through the world, place camera keyframes, and play back a path. Each world keeps its own saved paths, and you can import and export them. Behind it is a small Python runner that submits generation requests and polls for results.
What worked: the worlds are good. Walking through a generated space in the browser at interactive frame rates, with parallax and lighting that hold up, still feels a little unreal. The splat format is compact enough to load quickly.
What broke, roughly in the order it cost me time:
Scale. A generated world has no intrinsic notion of meters. A camera move that’s a slow push in one world is a wild flight through another, because the scene’s scale is arbitrary. I ended up storing an explicit scale factor with every world and every camera path, and converting everything to that before saving. It’s the kind of thing that seems too boring to matter until every sample you’ve made is subtly wrong.
Coordinates. Every system in the chain has its own convention for which way is up and which way the camera looks: the export format, the renderer, Three.js, and whatever tool you use next. I lost an evening to a world that rendered upside down and mirrored, which in hindsight is the classic mistake I’d warn a junior engineer about. The fix was one explicit transform, documented in one place, applied at the boundary.
Durability. Generation takes a while and costs real money per world. My first runner fired requests and waited, which meant a network hiccup or a closed laptop lost track of a world I’d already paid for. The second version writes a durable record for every request before submitting, polls with backoff, and can resume after a crash without resubmitting. At one point the API’s response shape didn’t match what I expected, and because the records were durable I could fix the parser and recover the finished worlds instead of regenerating them.
Drift. Once I had repeatable camera paths, I tried using them as control inputs for a video model, rendering the splat world along the path and using that as a structural guide for a stylized video, the ControlNet idea from 2023 applied to camera motion. The results are promising for camera adherence over a few seconds, and then they drift. The video model follows the path well at first and then starts to invent its own space.
After a weekend with it, I can see why a persistent 3D set helps: every shot starts from the same space. That was what I kept missing when I tried to cut generated clips together in 2023. The tooling around them, like scale, coordinates and handing the world to a video model without it wandering off, is where the work is now.