OpenAI’s new Sora text-to-video model can create shockingly realistic content
OpenAI is breaking new ground as the AI giant unveiled its first text-to-video model, called Sora, which can create shockingly realistic content.
We wondered when the company would finally release its own video engine, considering so many of its rivals, from Stability AI to Google, have beaten them to the punch. Maybe OpenAI wanted to get things just right before a real launch. At this rate, the quality of its products could eclipse those of its contemporaries. According to the official pageSora can generate “realistic and imaginative scenes” from a single text prompt; just like other text-to-video AI models. The difference with this engine is the technology behind it.
Lifelike content
Open AI claims its artificial intelligence can understand how people and objects “exist in the physical world.” This gives Sora the ability to create scenes with multiple people, different types of movements, facial expressions, textures and objects with lots of details. Generated videos, for the most part, lack the plastic look or nightmarish shapes seen in other AI content, but more on that later.
Sora is also multi-modular. Users will reportedly be able to upload a still image that can serve as the basis for a video. The content in the image is animated, paying close attention to the small details. It can even take an already existing video “and extend it or fill in missing frames.”
Question: A litter of golden retriever puppies playing in the snow. Their heads pop out of the snow, covered with. pic.twitter.com/G1qhJRV9tgFebruary 15, 2024
You can find sample clips on OpenAI’s website and on X (the platform formerly known as Twitter). One of our favorites is a group of puppies playing in the snow. If you look closely, you’ll see that their fur and the snow on their snouts have a strikingly lifelike quality. Another great clip shows a Victoria crowned pigeon floating around like a real bird.
A work in progress
As impressive as these two videos are, Sora isn’t perfect. OpenAI admits that its “model has weaknesses.” It can be difficult to simulate the physics of an object, confuse left and right, and misunderstand ‘cases of cause and effect’. You can make an AI character bite into a cookie, but the cookie won’t have a bite mark.
It also makes a lot of strange mistakes. One of the funniest accidents It involves a group of archaeologists who unearth a large piece of paper which then turns into a chair and eventually ends up as a crumpled piece of plastic. The AI also seems to have trouble with words. “Otter” is misspelled as “Oter” and “Land Rover” is now “Danover”.
even the Sora mistakes are fascinating pic.twitter.com/OvPSbaa0L9February 15, 2024
Going forward, the company will work with its “red teamers,” a group of industry experts “to assess critical areas for damage or risk.” They want to ensure that Sora does not generate false information, hateful content or have any bias. Additionally, OpenAI will implement a text classifier to reject prompts that violate their policies. This includes requests for sexual content, violent videos and celebrity likenesses.
No word on when Sora will officially launch. We’ve reached out for information about the release. This story will be updated at a later date. In the meantime, check out Ny Breaking’s list of the best AI video editors for 2024.