Stability AI, the developer behind Stable Diffusion, previews a new generative AI that can create short videos with a text prompt.
Aptly named Stable video distribution, it consists of two AI models (known as SVD and SVD-XT) and can create clips with a resolution of 576 x 1,024 pixels. Users can adjust the frame rate so that it runs between three and 30 FPS. The length of the videos depends on which of the twin models is chosen. Selecting SVD will play the content for 14 frames, while SVD-XT expands that a bit to 25 frames. The length doesn’t matter much, as rendered clips only play for about four seconds before ending, according to the official listing on Hugging Face.
The company posted a video to its YouTube channel showing off what Stable Video Diffusion is capable of, and the content is surprisingly high quality. They’re certainly not the nightmare fuel you see on other AI-like devices Meta’s Make-A-Video. The most impressive in our opinion is the Ice Dragon demo. You can see a lot of detail in the dragon scales and the mountains in the back look like something out of a painting. Animation, as you can imagine, is quite limited because the subject can only move its head slowly. The same can be seen in other demos. It’s a stiff walk cycle or a slow panning shot.
In the early stages
Restrictions don’t stop there. Stable video diffusion reportedly cannot achieve “perfect photorealism,” it cannot generate “readable text,” and it has difficulty with faces. Another demonstration on Stability AI’s website shows that the model is able to render a man’s face without any strange flaws, so this can be considered on a case-by-case basis.
Please note that this project is still in its early stages. Obviously the model isn’t ready for a wide release yet, and there are no plans to do so. Stability AI emphasizes that stable video diffusion is not intended “for real-world or commercial applications” at this time. In fact, it is currently “intended for research purposes only.” We are not surprised that the developer is very careful with its technology. Last year there was an incident where Stability Diffusion’s model leaked onlinewhich led to bad actors using it to create deep fake images.
Availability
If you’d like to try Stable Video Diffusion, you can enter a waitlist by calling a form on the company website. It’s unknown when people will be allowed in, but the preview will include a text-to-video interface. In the meantime, you can take a look at the The AI White Paper and read more about all the details behind the project.
One thing we found interesting after reviewing the document is that it mentions the use of “publicly accessible video datasets” as part of the training materials. Again, it’s not surprising to hear this, given that Getty Images has sued Stability AI following allegations of data scraping earlier this year. It seems like the team is aiming to be more careful so it doesn’t make more enemies.
No word on when Stable Video Diffusion will launch. Fortunately, there are other options. Check out Ny Breaking’s list of the best AI creators for 2023.