How AI can turn your home video into a Hollywood blockbuster
Do you want to star in an animated film as an anthropomorphic animal version of yourself? Runway’s AI video creation platform has a new AI tool to do just that. The new Act-One feature could eliminate the need for motion-capture suits and manual computer animation to match live action.
Act-One streamlines what is typically a long process for facial animation. All you need is a video camera that looks at an actor and can capture his face as he performs.
The AI powering Act-One reworks the facial movements and expressions from the input video to match an animated character. Runway claims that even the most nuanced emotions are visible through micro-expressions, eyeliners and other facets of the performance. Act-One can even produce multi-character dialogue scenes, which Runway says is difficult for most generative AI video models.
To produce one, a single actor fills multiple roles, and the AI animates the different performances assigned to different characters in one scene as if they were talking to each other.
This is a far cry from the laborious traditional animation requirements and makes animation much more accessible to creators with limited budgets or technical experience. Not that it will always match the skills of talented teams of animators with large film budgets, but its relatively low barrier to entry could give amateurs and those with limited resources the chance to play with character designs that are still realistic in portraying emotions. all without breaking the bank or missing deadlines. Below you can see some demonstrations.
Animated runway
Act-One is in some ways an improvement on Runway’s video-to-video feature within its Gen-3 Alpha model. But while that tool uses a video and a text prompt to customize the setting, artists or other elements, Act-One goes straight to mapping human expressions onto animated characters. It also fits with how Runway has introduced more features and options to its platform, such as the Gen-3 Alpha Turbo version of its model, which sacrifices some functionality for speed.
Like its other AI video tools, Runway has some restrictions on Act-One to prevent people from abusing it or violating its terms and conditions. For example, you cannot create content featuring public figures, and techniques are used to ensure that everyone whose voice is used in the final video has given consent. The model is continuously monitored to detect any attempts to violate these or other rules.
“We’re excited to see what kinds of creative storytelling Act-One brings to animation and character performance. Act-One is another step forward in our goal of bringing previously advanced techniques to a broader range of creators and artists,” Runway wrote in its announcement. “We look forward to seeing how artists and storytellers will use Act-One to bring their visions to life in new and exciting ways.”
Act-One may be somewhat unique among AI video generators, although Adobe Firefly and Meta’s MovieGen have some similar efforts in their portfolio. Runway’s Act-One seems much easier to use than Firefly’s equivalent and more available than the limited MovieGen model.
Still, AI video competition is on the rise as OpenAI’s Sora model begins to spread, and Stability AI, Pika, Luma Labs’ Dream Machine, and others push out a steady stream of AI video production features. If you want to try Act-One, Runway’s paid plans start at $12 per month.