Adobe’s next big project is an AI that can upscale low-resolution video to 8x its original quality
A group Adobe researchers recently published an article about a new generative AI model called VideoGigaGAN and we believe this could be launched on a future product. What it does is upscale low-quality videos to eight times their original resolution, without sacrificing stability or important aspects of the source material. Various demo clips can be found on the website project website show his capabilities. It can convert a blurry 128 x 128 pixel resolution video of a waterfall into 1,024 x 1,024 pixel resolution footage.
Message from @luokai
View on Discussions
What’s remarkable about the AI is that it doesn’t skimp on the finer details. Skin texture, wrinkles, hair strands and more are visible on the faces of human subjects. The other demos also have a similar level of quality. Thanks to this technology, you can better distinguish a swan swimming in a pond and the blossom on a tree. It may seem bizarre to pay so much attention to skin wrinkles or feathers. However, it is this level of detail that companies like Adobe must capture if they want to implement image-enhancing AI at scale.
Improve AI
You probably have a few questions about the platform’s latest project, like how does it work? Well, it’s complicated.
The “GAN” in VideoGigaGAN stands for generative adversarial network, a type of AI that can create realistic images. Adobe’s version is specifically based on GigaGAN which specializes in scaling both generated content and real photos. The problem with this technology, like The edge highlights is that it cannot improve the quality of videos without showing up multiple issues such as strange artifacts. To solve this problem, Adobe researchers used several techniques.
The research paper explains the entire process. You can read it yourself to get the full picture, even though it is dense material. In short, they introduced a “flow-guided propagation module” to ensure consistency between video frames, anti-aliasing to reduce artifacts, and a “high-frequency feature shuttle” to compensate for sudden drops in detail. There’s more to VideoGigaGAN than what we’ve just described, but that’s the gist of it.
Potential inclusion
Will we see this in an upcoming Adobe product or roll out as a standalone app? Most likely – or so we think.
Over the past year, the company has focused heavily on implementing artificial intelligence into its software, from the launch of Firefly to Acrobat’s new assistant. A few months ago, during Adobe MAX 2023, a video upscaler was mentioned Project Res Up was previewed at the event and its performance is similar to what we see in the VideoGigaGAN demos. An old film from the 1940s goes from an image resolution of 480 x 360 to a sharp 1,280 x 960. Blurry images of an elephant in a river become crystal clear. The presenter even mentions how the software can upscale a clip to four times the original quality.
Admittedly, this is conjecture, but it is entirely possible that VideoGigaGAN is the driving force behind Res-Up. Adobe’s future product could give people a way to upscale old family videos or low-quality footage into the movie we imagine in our minds. Perhaps the recent preview is a hint at an upcoming release.
VideoGigaGAN is still in development, so it is unknown if and when it will be released. There are several obstacles on the way. The AI can’t properly handle videos larger than 200 frames or render small objects, but we’ll definitely keep an eye on it.
In the meantime, check out Ny Breaking’s list of the best AI image upscalers for 2024.