New Content-Aware Fill for video in AE

Ahead of NAB 2019, Adobe has announced product updates and new innovations for video and audio tools in Adobe Creative Cloud. These include some much anticipated things such Content-Aware Fill for video, and some lesser high profile, yet still key announcements, such as external GPUs and Twitch Trigger support.

Content-Aware Fill for Video

Users can now use Content-Aware Fill to remove unwanted objects from footage in After Effect. Things such as signs, boom mics, logos, and even people, can be removed without flicker or complex roto.

The process uses Adobe’s Sensei, artificial intelligence engine (AI) and machine learning technology to remove and replace unwanted objects. While it is not possible to always know what was being obscured, the program plausibility fills in the holes created as objects are removed. If it can, the program is informed by the frames surrounding the current frame, if not, it uses the same approach as the static content-aware fill, but significantly it does this without flicker. It is this temporal cohesion that makes it so powerful.

Formerly known as Project Cloak, the AE content-aware fill only requires a rough roto of the items that you want to remove. The user only needs to set up a garbage mask and not a detailed roto of the item to be removed. While the processing is not real time, the computation can be done in the background while you continue to work in the foreground. The process also adjusts the lighting to match the changing tones of the filled area.

It is also possible to paint example frames to guide the program in a particular direction of how to tackle the fill, thus directing the engine.

Since this technology was first shown as a tech demo (2017, see our coverage here), there has been a huge demand and expectation on Adobe to get the video version of content-aware fill in the hands of artists.

Source: FX Guide