OpenAI’s Sora, the equivalent of creating images but for videos, sent huge shockwaves through the rapidly advancing world of AI last month, and we’ve just recorded a few new videos that are even more breathtaking than what we’ve already been treated to.
In case you somehow missed it, Sora is a text-to-video AI, which means you can write a simple request and put together a video (just like how image generation worked before, but obviously a much more complex undertaking).
Now OpenAI’s Sora research lead Tim Brooks has released new content generated by Sora on X (formerly Twitter).
This is Sora’s trick to fulfill the following request: “Fly through a tour of a museum with many paintings and sculptures and beautiful works of art in all styles.”
Quite impressive to say the least. Additionally, Bill Peebles, also Sora’s research lead, showed us a clip generated from the following prompt: “An alien naturally blending in with New York City, paranoia thriller style, 35mm film.”
Content creator Blaine Brown then stepped in to embellish the above clip, cutting it to repeat and lengthen the footage as the alien rapped, complete with lip-syncing. The music is generated by Suno AI (with the lyrics written by Brown, mind), and the lip sync is done with Pika Labs AI.
This Sora clip is 🔥 where the alien man bursts into a lip-synced rap about how hard it is to be different from everyone else. Workflow in the thread.@suno_ai_ @pika_labs (lip sync)Alienate yourself 🆙🔊🔊 pic.twitter.com/kc5FI83q5RMarch 3, 2024
Analysis: Still early for Sora
It is worth highlighting how quickly things seem to be evolving thanks to the capabilities of AI. The ability to create images was one thing – and extremely impressive in itself – but this is something else entirely. Especially when you consider that Sora is only just testing at OpenAI, with a limited number of ‘red teamers’ (testers who find bugs and iron out those wrinkles).
The camerawork in the museum fly-through flows realistically and feels nicely imaginative in the way it floats around (albeit with the occasional shake). And the latest tweet shows how you can take a basic clip and flesh it out with content, including AI-generated music.
Of course, AI can also write a script, and so the question arises: how long will it be before a blue alien stars in an AI-generated post-apocalyptic drama. Or maybe an (unintentional) comedy?
You get the idea, and of course we’re getting carried away, but still: what AI could be capable of in a few years is, quite frankly, potentially mind-boggling.
Naturally, we’ll see the cream of what Sora is capable of in these teasers, and some buggy and strange efforts have been aired as well. (Just like when ChatGPT and other AI chatbots first came on the scene, we saw AI hallucinations and general unhinged behavior and responses).
Perhaps the broader concern with Sora, however, is how this could ultimately displace rather than help content creators. But that’s a fear to chew on for another day – not to mention the potential for abuse with AI-created videos which we recently discussed in more depth here.