Runway’s AI video creation and editing service has added the promised video-to-video revamp feature to its Gen-3 Alpha model platform. The video-to-video tool lets you submit a video and a text prompt to alter it. The changes can adjust the setting, performers, or other elements of the video to match the text prompt or any of a handful of preset style suggestions.
Video-to-video is the last major addition to Runway’s video creation options. Runway already allows users to start with either text or images to define the look and motion of the video. Now, by starting with a real-world video, users can define the motion upfront and then use AI to alter the design and aesthetics. Runway trained the feature on a large set of videos and images captioned with details to teach the AI model how to understand uploaded films and the prompts for changing them. It’s like seeing a new video from another universe entirely. You can see how it works below.
“Video to Video represents a new control mechanism for precise movement, expressiveness, and intent within generations,” Runway explained on social media. “To use Video to Video, simply upload your input video, prompt in any aesthetic direction you like, or, choose from a collection of preset styles.”
Film Multiverse
When combined with Gen-3 Alpha’s existing ability to handle complex transitions and expressive human faces and emotions, the results can be pretty spectacular. Runway has been quick to augment Gen-3 Alpha with new options. Most recently, the company showed off the Gen-3 Alpha Turbo version of its model, which sacrifices some functionality for speed. Along with the video-to-video makeover option, the tool also works with Motion Brush, Advanced Camera Controls, Director Mode, and other, more granular video editing features.
Gen-3 Alpha comes off as far more versatile and user-friendly than the Gen-2 model. That said, it does have limitations despite its impressive output. Currently, it can generate video clips up to 10 seconds long, hardly a feature length film. Speed and quality may make up for that until it can go longer. There’s ever more competition for Runway to have to outdo, though.
The best-known is OpenAI’s Sora model, but it’s far from the only one. Stability AI, Pika, Luma Labs’ Dream Machine, and more are all racing to bring AI video models to the public. Even TikTok’s parent company, Bytedance, has an AI video maker called Jimeng.
If you want to try out Runway’s video-to-video tool, Gen-3 Alpha is accessible to users on paid plans starting at $12 per month.