After sneaks and peeks, Adobe pulled the official trigger on its Firefly Video Model, launching it into limited public beta, as the model follows the path to full public availability that the company followed for its other Firefly Models. Firefly-powered video tools such as Generative Extend can be found in Premiere Pro (beta).
We’ve been expecting this for the last few months and have been hearing about it for about a year now. The “it” refers to the Firefly Video Model, which Adobe has just launched into limit public beta, as announced at the Adobe Max conference this week.
The Firefly Video Model (beta) adds to Adobe’s growing family of creative generative AI models, including Image Model (introduced in beta in March 2023), Vector Model, and Design Model, providing AI-enhanced tools and features to the company’s production offerings within these industry segments. According to Adobe, Firefly has resulted in 4´ faster image generation, as well as new capabilities integrated into Photoshop, Illustrator, Adobe Express, and now Premiere Pro.
The Firefly Video Model follows the adoption pattern established early on by Adobe, starting with a limited public beta, which enables the company to collect and analyze feedback, before refining the model and offering it into beta for a wider assessment so bugs and other fixes can be made prior to public release.
“Video was a big ask from our customers,” said Alexandru Costin, vice president of generative AI at Adobe.
Costin noted that video is now a very prevalent medium for content creation, and Adobe has been working on its Video Model for a while. But, video has been difficult when it comes to generative AI, and is now happy with the results and quality it is able to deliver. And, like all of Adobe’s Firefly models, the Video Model is designed to be safe for commercial use, as it is trained only on content the company has permission to use, which is Adobe Stock data and public domain data. In fact, when Adobe first began engaging with the community as to what they needed in the model, this issue was of a major concern.
Also, the community indicated they favored editing functionality over pure generation, Costin said. That is not so surprising, he explained, given that with Firefly Image Model, the most usage is within Photoshop with Generative Fill, which uses a creator’s own imagery. So, for the initial Firefly-powered video editing workflows powered by the new Firefly Video Model, Adobe focused on generative editing capabilities, he said, helping creators extend video clips they already have in post. This mostly involves assembling clips and making sure the soundtrack matches the actual clips—addressing a common but complex issue in video editing.
To address that issue, Adobe has introduced new Firefly-powered tools like Generative Extend (beta) in Premiere Pro (beta) for seamless clip extensions to cover gaps in audio or video footage by making shots a little bit longer through generation of entirely new frames at the beginning or end of a clip. It can also be used to extend audio clips to create ambient “room tone” to smooth out audio edits, as well as hold shots longer for better timed edits.
To help video professionals quickly communicate their creative intent, create B-roll to fill gaps, or to add new elements to a shot, Adobe also released the new Generative Video (beta) module. Powered by the Firefly Video Model and likewise in limited public beta in the Firefly Web app is new Text to Video and Image to Video for generating content from text or still images. These capabilities enable editors to generate video from text prompts, access a variety of camera controls such as angle, motion, and zoom to fine-tune videos, and seamlessly fill gaps in a video timeline.
Image to Video capabilities allow creators to bring still shots or illustrations to life by transforming them into live-action clips. Here, the video is generated with the prompt:
Users can join the waitlist for the Firefly Video Model limited public beta here. During this limited public beta, generations are free. Adobe said it will share more information about Firefly video generation offers and pricing when the Firefly Video Model moves out of this stage.