Adobe Firefly rolls out significant updates to video and audio capabilities

Today Adobe's Firefly is rolling out significant updates to its video capabilities, aiming to give creators more control and flexibility in their video projects. The enhancements include improvements to the core Firefly Video Model, new generative AI partner integrations, and advanced workflow tools designed to streamline the video creation process from start to finish.

The Firefly Video Model now offers improved motion fidelity, resulting in more natural and lifelike movements in generated videos. This is particularly noticeable in areas like dynamic landscapes, animal motion, and atmospheric effects. These refinements are intended to produce smoother transitions and greater accuracy in video generations.

We tested Firefly with text to video and this was conjured up within 15 seconds (we had it set to low-rez output). It's meant to be a video of a photography magazine editor writing a story about Adobe Firefly. My only issue with the video is this guy is a lot better looking than I am. 

Expanding creative options with partner models

Firefly is expanding its range of integrated generative AI models, allowing users to choose the best tool for their specific creative needs. Recently added partners include Runway's Gen-4 Video and Google Veo3 with Audio within Firefly Boards and Generate Video.

Further integrations are on the horizon. Topaz Labs' Image and Video Upscalers and Moonvalley's Marey will soon be available in Firefly Boards. Additionally, Luma AI's Ray 2 and Pika 2.2, currently accessible in Boards, will be integrated into Generate Video.

This wider selection aims to provide creators with a comprehensive suite of AI models, enabling them to maintain creative consistency across various stages of their workflow, from storyboarding to final video output.

Directing every detail with advanced video controls

This release introduces new features that offer more precise control over video content. It's always been annoying shooting for vertical and horizontal for YouTube / Instagram / Facebook / TikTok videos. Creators can now easily select vertical, horizontal, or square aspect ratios to match their intended output format, such as mobile, widescreen, or social media, without additional editing.

Other notable features designed to enhance creative output and save time include:

• Composition Reference for Video: This tool allows users to upload a reference video and describe their vision. Firefly will then generate a new video that adopts the composition of the original, helping to maintain visual flow across scenes or repurpose existing content with a new look.
  
  
• Style Presets: Apply a distinct visual style, such as claymation, anime, or line art, with a single click. These presets help establish a consistent tone, whether for pitching concepts, building creative briefs, or finalizing projects.
  
  
• Keyframe Cropping: For efficient framing, users can upload their first and last frames, select how the image will be cropped, and describe their scene. Firefly will generate a video that fits the specified format, aiming to reduce the need for external tools and maintain creative flow.

Bringing stories to life with sound and avatars

Firefly is also introducing new features to enhance storytelling through audio and virtual avatars.

• Generate Sound Effects (beta): This feature allows users to create custom sound effects using a text prompt or their voice. Whether it's a lion's roar or ambient nature sounds, the tool aims to add emotion and depth to videos.
  A voice-guided option listens to the energy and rhythm of the user's voice to precisely place sound effects, synchronising them with the video's action. Finished videos can be exported directly to Adobe Express for social media or Premiere Pro for further editing.
  
  
• Text to Avatar (beta): This feature enables users to transform scripts into avatar-led videos. Users can choose from a library of diverse avatars, customize backgrounds, and select accents. This tool is useful for creating engaging video lessons, turning blog posts into scalable video content, or developing internal training materials with a human touch.

Streamlining the creation process

To help creators work more efficiently, Firefly is introducing Enhance Prompt within the Generate Video module on the Firefly web app. This new feature is designed to improve results by taking the user's initial input and adding language that helps Firefly better understand the creative intent. The goal is to reduce ambiguity, provide clearer direction, and speed up the workflow, allowing creators to focus more on their vision and less on prompt refinement.

You can test out Firefly for free, and find out more on the Adobe Firefly homepage.