Change everything except the motion
At I/O this week, YouTube brought Gemini Omni to Shorts Remix: take a Short, change the characters, setting or style, and keep the motion and story. Why remix is the right first experience for video-to-video.
On Tuesday at Google I/O, YouTube announced that Gemini Omni is coming to Remix in Shorts and the YouTube Create app. You pick an eligible Short, add a prompt and optionally your own images, and get a new version: the scene turned into a 90s sitcom, you standing next to your favorite creator, the characters changed, the world restyled. The motion, timing and context of the original stay intact. It’s rolling out now at no cost, with English prompts, and not yet in the EU or UK.
I lead the engineering team that worked on this, with a lot of people across YouTube and Google DeepMind, so I’m biased. I’ll stick to what’s public and to why I think remix is the right shape for this capability.
The model is video-to-video: its input is an existing video, and it transforms it according to your instructions. The hard problem is deciding what to keep. Keep too little and you’ve just generated a new video loosely inspired by the old one. Keep too much and nothing interesting changes. A remix still needs to preserve the performance: the movement, the comic timing, the story beats. Those are what make the original recognizable after the characters and setting change.
Why start with remix instead of text-to-video from scratch? A few reasons I’ve written about here over the years.
The original gives structure. In 2023 I argued that control beats prompts, and that creators want to steer structure, not describe it. An existing Short is the richest possible control input: every movement, every beat and every cut, already decided by a person. The model doesn’t have to invent a performance. It inherits one.
It keeps people at the center. In my post about the Sora shutdown, I said generation works best inside places people already go, attached to things they already care about. Remix is anchored to a creator and a video people already liked. The new version points back to it. Every Short remixed with Gemini Omni carries a SynthID watermark and metadata identifying it as AI-generated, and links back to the original video.
It’s a conversation between creators, which is what short video has always been. Sounds, trends, duets and remixes are how culture moves on these platforms. Generation fits into that as a new way to respond to someone else’s work.
There’s much more to do, including more languages, more regions and more control. Personally, I’m happy to see video-to-video reach people through something as familiar as remix.