Behind the Scenes of Black Widow

Digital Domain on how a team of artists tackled some of Black Widow's most memorable moments.

Digital Domain has worked on Marvel films in the past—“Avengers Endgame” and “Thor Ragnarok”—but handling the visual effects behind the cataclysmic ending of “Black Widow” was an enormous undertaking.

Working under the direction of VFX Supervisor David Hodgins and DFX Supervisor Hanzhi Tang, Digital Domain’s team of 250 artists used Houdini, Maya, Redshift, Substance Painter, V-Ray and more to construct and blow up the aerial Red Room, create hero debris and digital doubles to place in the falling wreckage, and orchestrate the air battle where the characters fall back to Earth.

We talked with Ryan Duhaime, one of Digital Domain’s CG supervisors on “Black Widow” about how the team handled the 320 shots they created for the film. Here’s what he had to say.

Tell us about how your team of artists worked together on this project.

Duhaime: For “Black Widow,” Digital Domain had artists working across several sites, including Los Angeles, Vancouver, Montreal, and Hyderabad. We were responsible for a few different sequences within the film, and we split the work across sites to be able to turn around shots quickly and efficiently.

The Vancouver team dealt with the FX heavy sequences of the Red Room exploding and the aftermath of the free fall towards Earth. Our Montreal team handled the sequences on the ground, the remnants from the explosion and the action from above.

The Hyderabad team was instrumental with our plate prep, tracking, match-moves, and integration while the Los Angeles team encompassed management, supervision and artists working in a variety of disciplines to help with finalizing shots and asset development. Collaboration was key to construct the complex visual effects shots required to successfully achieve Marvel’s vision.‍

How was the project described to you from the start, and did it grow from there?

Duhaime: We started the project by working with the art department to develop a look for the Red Room. They were able to provide us with various concept art from different angles, as well as a previz model that indicated where things would generally be located. With that in mind, we were able to extrapolate the scale of the tower floors, runways, catwalks and other elements and build out the remaining structure for a more complex look.

Over the course of the show, the sequence and edits evolved into the end product. We knew the heroes needed to land on the ground and be relatively unscathed. To do that, we had to figure out how to reduce our heroine’s terminal velocity by maneuvering through the debris field and evading the methodical villain in pursuit.  

We adjusted the action during the fall over time, but the key to identifying where she was going and where she came from was to have the same pieces of debris and destruction continuously flying around her. That helped to identify her trajectory and lead us from one shot to the next without getting too disoriented.

At one point, we needed to expand upon the Red Room’s engines and turbines to allow for some closeup shots in order to see the initial destruction in action. Our model was not quite as intricate as it needed to be for those hero angles underneath, so the team had to work diligently to give it more detail and magnitude.

From the beginning, we tried to make sure our assets would hold up to a variety of angles and closeups. We wanted them to have the detail required if something ended up changing after a reshoot, or an action needed to be improved upon in CG to make it more dynamic than what could be captured on set.  

Walk us through your process for the Red Room.

Duhaime: Digital Domain constructed the Red Room by working with art department concepts, previz models and real-world structures. It was designed to be both intimidating and functional while having a style that echoes Soviet-era architecture.

The structure features several arms connected to a massive central tower lined with catwalks and propelled by numerous engines below. The arms house airstrips, fuel modules, solar panels and cargo. Details like ladders, doorways and railings were added to maintain a sense of scale. We also created two hero arms that required high-res geometry to integrate seamlessly with the live-action footage by matching LiDAR scans for the physical set piece runways, hallways and confinement cells.

We started by modeling the building blocks of the Red Room, and we used instancing of individual assets, like beams, supports, scaffolding and flooring, to assemble as much as possible within a single layout. Our main exterior layouts consisted of over 350 assets and over 17,000 instances to make up the massive structure.

If you take into account all of the additional damaged pieces, interior confinement cells, surgical corridor and hallways, we generated over 1,000 assets that were used throughout our sequences to help sell the structure's complexity.

How did you get such a vast number of elements to match so seamlessly?

Duhaime: For such an intricate model, we needed to set up simplified shading networks to allow look dev to match from one renderer to another without any adjustments or color corrections. That helped establish a baseline regardless of the renderer, but it relied heavily on our texture team and their setups within Substance Painter and Mari.

We used Redshift for look development of the hard-surface objects and V-Ray for our digital double work. That combination allowed us to utilize both GPU and CPU rendering when needed.

What were some of the challenges you dealt with?

Duhaime: For shot work, and dealing with the Red Room and debris, we had to overcome a variety of issues and complexities. We approached the destruction by combining rigid body solves of instanced geometry and detailed hero fracturing and debris creation for custom sections. Those were published to lighting as Redshift proxies and layouts.

We also made use of Redshift proxies for our skydiving shots, which had several layers of falling debris that were all fractured assets from the initial Red Room arms. Our Houdini pipeline was set up to render similar look dev as final shot lighting, which allowed us to get FX Redshift renders that nearly matched the final render. Using Redshift proxies allowed us to package up the destruction geo, shaders and textures within one publish and pass it off to our Lighting team.

Because we built the Red Room in a very modular way, we were able to get fantastic simulation detail using straightforward rigid body sims. The heavy lifting was in the setup of constraints for the thousands of connected pieces, so when we finally ran the simulation, it broke apart in a realistic and believable way. If we needed hero bending and breaking, we would promote those pieces to a hero sim. That approach helped us simplify and keep the entire structure light enough to quickly iterate on.

Talk a bit about some of Natasha Romanoff’s action shots.

Duhaime: The section of the film where Natasha (Scarlett Johansson) runs down the hallway in the Red Room was another great series of shots. We recreated the entire hallway and added in laboratory equipment and test tubes behind the glass cabinets. It helped to create some dramatic moments when we shattered them in the shot.

The plates provided great references to match key components but, in the end, we needed to reconstruct everything in CG to be able to collapse, crumble and explode the ceiling and walls around her.

For her skydiving shots, a lot of inspiration came from live-action plates with stunt performers falling and flipping around. We tried to capture as much of the stunt actors' performances as we could while also maintaining the camera movements. To keep track of Natasha and her heroic journey to the ground, we needed a way to foreshadow her moments and maintain a sense of spatial awareness.

So we made sure that you would see similar pieces of debris, like the solar panels, and bent and crumbled sections of Red Room arms falling from one shot to the next. At the same time, lab equipment and broken pieces of a Russian Quinje are falling alongside her.

Did you learn anything new while working on this film?

Duhaime: This project was quite the undertaking from a personal standpoint, but also on a facility level. The quality of work we were able to achieve is a true testament to the artists that worked tirelessly to complete the shots. Personally, I learned so much about managing and organizing all of the moving pieces, and I certainly would not have been able to do it without the help of the talented team of artists and production working alongside me.

The team at Digital Domain worked incredibly hard to be able to develop all of the assets and sequences into what was shown on screen. Everyone should be incredibly proud of the quality of work produced and what they accomplished during such a demanding and complex project.

Meleah Maynard is a writer and editor in Minneapolis, Minnesota.

‍