Using artificial intelligence to produce videos has become increasingly popular thanks to tools like Sora, Runway and Pika. But a new AI-augmented cinematic camera has taken the idea to the next level.
Creative technology agency SpecialGuestX and mixed-media production house 1stAveMachine have launched the CMR-M1 to achieve that goal. The result is a camera that doesn’t just film the world around it; it enhances it using a generative AI video-to-video model. The digital film is sent to the cloud, where a model based on Stable Diffusion transforms it. What’s more, the AI can also connect to other APIs to further play with what it can do with the film.
“We believe that filmmaking is inherently physical. Filmmakers are used to using cameras, lenses, tripods, and lighting, and that AI should be a new tool that augments creativity,” said Miguel Espada, co-founder and Executive Creative Technologist of SpecialGuestX. “In the spirit of creative experimentation, we designed a camera that serves as a physical interface for AI models.”
The camera has a wheel on the side of the body, allowing operators to smoothly switch between real and AI-generated overlays. There’s even a slot for a style card with an NFC chip that connects to the user’s model trained with their images and personalized prompts.
While it’s not a real-time setup due to latency, models in development like StreamDiffusion could make that a reality. There are currently five different filters the film runs through, including the jungle-infused “Blooming Nature” and the tuxedo and gold of “Old Money.” Below is an example of “Old Money.”
Film AI
The CMR-M1 is currently a prototype and not intended for commercial release. It is part of SpecialGuestX’s research into creating physical interfaces for generative AI. However, the design of the CMR-M1 was developed with scalability in mind, allowing for potential commercial production in the future.
CMR-M1, which stands for the somewhat basic name Camera Model 1, deliberately tries to evoke the dawn of film with its appearance. The camera has a design that is reminiscent of the Cine-Kodak camera from 1924, one of the first and most popular portable 16mm cameras.
“Sometimes you have to prototype to imagine what the future might look like. This is especially true when it comes to filmmaking, creative technologies, and the creative community,” said Aaron Duffy, founder and executive creative director of SpecialGuest. “The CMR-M1 is a prototype for how hands-on creators will use AI instead of sitting behind a keyboard.”
Whether CMR M1 will have an impact on the larger film world is hard to say. The more immediate question is whether it will open up a new competitive battle between AI video model developers like Runway and its new Gen-3 Alpha, OpenAI and its Sora model, and the efforts of Pika and Luma Labs.
The inclusion of open-source models from Stability AI suggests that this kind of access could play an important role in the evolution of AI videos. What ultimately happens in the competition, however, is still a cliffhanger.