AI-based scene analysis: optimizing editing with Adobe Premiere Pro

AI tools have been integrated into Premiere Pro for some time now, saving editors time and helping ensure quick turnarounds. 

Much of the conversation around Adobe’s foray into AI has been focused on the generative AI tools produced by its Firefly model. However, they have had many useful AI tools integrated within Premiere Pro for some time now. These tools significantly reduce the tedious work that editors are often required to do and allow for a more efficient workflow. Premiere Pro’s tools, such as Morph Cut, Auto Reframe, Color Match, and Scene Edit Detection, can save editors a lot of time and ensure quicker turnarounds. Let’s break down these tools and how to use them.

Morph Cut

The Morph Cut video transition effect matches two shots from different points within one long video clip. It is often used to mask jump cuts during talking head shots in interviews and podcasts. Premiere Pro’s in-built AI generates a few extra frames to blend and hide the jump cut, creating a seamless transition. While working on interviews and podcasts, editors must often smooth out disjointed cuts and awkward pauses. Instead of finding an elaborate workaround, the morph cut offers a one-click solution, speeding up the editing process.

When using the morph cut effect, ensure that both shots are from the same source footage. It works best when the camera is static, and the background is still. To use this effect, go to the Effects tab, and navigate to the Video Transitions folder. Here, under the Dissolve folder, you will find the Morph Cut effect. Simply drag this effect over to the cut you want to smooth over. Premiere Pro will take a few seconds to analyze the scene and generate the necessary frames to create a seamless transition. Adjust the length of the transition as required.

Auto Reframe

Auto reframe uses Premiere Pro’s inbuilt AI to analyse your footage. It highlights key elements within the frame and creates a new sequence with your desired aspect ratio. With the growing need for digital content to be published across multiple platforms, editors are often tasked with creating the same video in different aspect ratio formats. Manually reworking the video effects to fit a new aspect ratio is very time-consuming. However, Premiere Pro’s auto reframe tool will analyse the critical points of action and ensure they remain the focus in a new sequence while saving the editor a lot of time.

The best practice is to edit the video in a horizontal format first, then use Auto Reframe to create the vertical videos. To use Auto Reframe, navigate to the Project folder and select the sequence. Right-click and choose the Auto Reframe Sequence option. Choose the desired aspect ratio. Choose an option based on the camera movement in the footage (if unsure, use the Default option). Finally, click Create. Premiere Pro will create a new sequence in the Project folder. Editors will still have to rework any graphics or effect layers to fit the new sequence, so it isn’t a one-stop solution. However, it does speed up the process and helps optimize the editing workflow.

Color Match

Color Match is a Lumetri Color tool allowing editors to match the color scheme between two different shots. Premiere Pro’s in-built AI can analyze the color scheme from the reference clip and apply it to the desired clip. Instead of trying to match the color tones by eye, Premiere Pro takes the guesswork out of an editor’s workflow. The Color Match tool is handy when adding new footage or pick-ups to an already colored timeline. Editors can achieve precise consistency across their scenes without spending as much time and effort.

First, navigate to the Color tab at the top of your window. Select the desired clip, and in the Lumetri Color panel, choose the Color Wheels & Match tab. If the footage features actors, use the Face Detection option, as Premiere Pro’s AI will try to keep the skin tones consistent. When ready, click Comparison View. Scrub through the timeline using the scroll in the Program panel and choose the ideal reference clip. Finally, click Apply Match. The desired clip will update to match the color tones of the reference clip. (Note that you will still have to do basic color correction to match the contrast, highlights, etc.)

Scene Edit Detection

Scene Edit Detection is a Premiere Pro tool that can analyze an already edited video clip and break it down into smaller clips based on the edit points in the footage. Editing jobs are always complicated. Projects change hands, and sometimes, editors need to recut an already-edited video. Manually finding each edit point and cutting is a tedious process many editors have endured. But now, thanks to the Scene Edit Detection tool, Premiere Pro reduces the time spent organizing clips for a recut.

Using the Scene Edit Detection feature is very simple. Just select your edited video clip in the timeline. Right-click and choose the Scene Edit Detection option. Premiere Pro will analyze the footage for a few seconds and output a series of clips with cuts at each edit point. This straightforward yet effective tool can save editors an immeasurable amount of time.

All the tools mentioned above follow a similar pattern. Adobe Premiere Pro’s AI tools can analyze existing footage and speed up the tedious work editors often face. These AI tools make the workflow much more efficient and allow editors to focus on the creative aspect of their job. Time is the most significant value gained by using Premiere Pro’s AI tools, and they are essential for the modern editor dealing with tight turnarounds.