Behind the Scenes of Dune

An interview with Oscar-winner Paul Lambert and VFX Supervisor Patrick Heinen on their work for DUNE (2021)

Stills courtesy of Warner Bros. Pictures

The creators of the latest take on the science-fiction epic, “Dune,” dealt with immense scale as they filmed massive exteriors with yawning deserts and gigantic sand worms. While Vancouver-based DNEG and Director Denis Villeneuve led the production, Oscar-winner Paul Lambert served as Overall VFX supervisor and brought on WylieCo to work on post-viz. 

Lambert knew that DNEG was already committed to creating a substantial portion of the 1,700 shots in “Dune,” so rather than having them stop work on the more complex effects he worked with Patrick Heinen, WylieCo’s VFX Supervisor, to put together temp versions of every composite for the director’s edit. “We were able to produce really fast turnarounds of fully lit and rendered scenes of some complex shots using Redshift,” Lambert recalls. 

WylieCo’s team worked hand in hand with editorial to help shape the movie in the early stages of the edit. They also helped facilitate storytelling by providing temp versions that informed, not only what was happening in a shot, but also the subtleties of feelings. 

Taking things a step further than usual, they provided photoreal renders to express the vast scale, look and feel of the Dune universe. Lambert made sure that WylieCo rendered the visualizations with proper lighting for the director. “Being able to render out the huge architecture while having proper physical lighting was crucial,” Heinen explains.

“And it was really beneficial to have renders that were pretty close to the look of the final movie. Instead of a technical render with grey boxes, we could render out almost a final frame view of the scene.”

At one point, WylieCo’s compositors were just a few doors away from the director, generating quick renders of shots they could show to him for instant feedback. With the work Wylie did being so close to exactly what Villeneuve wanted, it was a logical decision to have them take a few of the sequences all the way to final picture.

“I got WylieCo to take them to the final,” Lambert recalls, “and there were two sequences Wylie did themselves, the cemetery scene and the Hunter Seeker scene where Timothée Chalamet’s character hides inside a hologram.”‍

The Cemetery and Hologram Scenes‍

For the cemetery scene, which was shot on location in landlocked Hungary, Heinen’s WylieCo team used background footage Lambert shot of hills and ocean in Norway to create set extensions that made the oceanside scene believable. 

The sequence, in which the film’s heroes stroll the graveyard as they prepare to leave their home planet, included a substantial amount of 2D work, as well as additional tombstones. “I believe we had about six practical tombstones,” Heinen recalls, explaining that after taking a lot of tombstone photos, they used photogrammetry to multiply them and rebuild others.

The challenge lay in integrating the tombstones and set extensions in knee-high grass that was moving in the wind with actors crossing in front of it. Lambert had used gray screens on set to facilitate the extraction of the grass and weeds. 

But to achieve the same parallax on the extensions that went behind those gray screens, the artists had to add multiple layers of artificial grass and weeds in depth. To accomplish that, Heinen’s team used a variety of additional grass and weed plates that had been shot on set in front of gray screens, and they them on cards in Nuke’s 3D space.

WylieCo’s work on the scene involving an intruder (a bug known as a hunter-seeker) and a holographic tree was much more involved and has been nominated for Best Compositing and Lighting at the 2022 VES Awards. In the scene, Chalamet’s character (Paul) is in his room reading a book and looking at a hologram when the hunter-seeker comes in through the headboard on his bed.

Scared, he hides from the hunter-seeker inside the branches of the hologram. Having done a lot of digital human work on previous projects, Lambert knew that believably recreating light interaction with skin was very challenging and wanted to investigate other avenues.

Mag Sarnowska, one of the on-set, in-house artists originally played with the idea of visualizing the hologram being as thick slices. Though the director didn’t like that strategy, the idea inspired the team to project light slices onto Chalamet.

“Basically, the idea was to cut the CG bush into hundreds of cross-sectional slices and use an actual projector to project one slice at a time onto Timothée, depending on where he was in the room,” Lambert explains. James Bird of DNEG London oversaw the development of the real-time onset tracking solution that drove the projector with the relevant CG bush slice. 

“That created the illusion of Timothée intersecting with the branches as he moved through the scene,” Lambert continues. And, as the strategy was practical rather than virtual, it allowed Cinematographer Greig Fraser to adapt his camera, which in turn gave Chalamet the cue to change position.

With the lighting interaction of the hologram captured in-camera, the challenge for WylieCo was to match the computer-generated tree to the light spots on Chalamet’s face and body. First, the team tracked and perfectly rotomated Chalamet’s body in order to have a true representation of the scene in the computer. 

Then, starting with the actual model of the bush that was sliced and projected on set, the team began matching up branches to the light spots. To help, they projected the footage onto the rotomated body per frame and extruded the light spots along the motion of the body. 

That approach gave the team a three-dimensional representation of where the branches had been on set and allowed the CG branches to precisely line up with the light spots.

While the subtleties of animation of the hunter-seeker scene had been worked out by WylieCo during postviz, the look of the hologram was not locked down until later. Heinen knew that the shallow depth of field combined with the semi-transparency of the hologram would be very challenging to recreate with defocus in compositing. 

So he and CG Supervisor TJ Burke decided to create most of the look of the silver holographic tree in Maya with rendering of the defocus and bokeh in Redshift. 

Burke spearheaded the look of the tree using very distinct defocus kernel in Redshift to achieve the ephemeral look that Villeneuve was after. That also provided compositors with a foundation to refine the optical look of the hologram and integrate the branches with the plate.

“Using a practical approach to a digital technique worked out really well for this sequence,” Lambert says. “So well that it has been nominated for a VES awards and I want to congratulate everybody involved.”‍

‍

Paul Hellard is a writer/editor in Melbourne, Australia. 

‍