Testing CoTracker2 for VFX inside my Silhouette RotoFury interface. The results are very promising for speeding up everyday VFX tasks such as tracking, roto, and matchmove.

It is one of the best ML algorithms currently available for this kind of work, with immediate practical applications.

Credit: compositing supervisor: Boundary Visual Effects

We completed more than 150 VFX compositing shots for the film. The work was split between the live-action sequences and the CG-toy sequences.

In the live-action work, we removed modern elements from sets, including security cameras, satellite dishes, and electrical wires. We also created winterisation and added snow to dressed sets.

For the CG work, the toys come to life. This involved preparing live-action plates for the CG toys through cleanup, compositing the multi-pass CG renders, fixing 3D-render issues and eye details, and integrating CG with the live-action footage.

I am really impressed by the quality of the SDXL inpainting model. This result used only an alpha mask and the prompt: “a bridge in the foreground, buildings and trees in the background.”

No ControlNets, pure inpainting.