The Mill recently collaborated with award winning directing duo Dom&Nic to bring Chemical Brothers’ brand new video ‘Wide Open’ to life.
The enthralling promo, which uses a range of highly technical visual effects techniques, has since gone viral, prompting the question; Just how did The Mill turn world renowned dancer Sonoya Mizuno see-through?
Dave Fleet - Head of 3D + Co 3D Lead Artist, Suraj ‘Sid’ Harrington-Odedra - Co 3D Lead Artist and Fergal Hendrick - 2D Lead Artist take us through the key components of this complex project:
The team behind 'Wide Open'
One Continuous Shot:
Dave: Dom&Nic always wanted to shoot this as a single 4.5 minute long take. We had discussed the potential of bolting multiple takes together, but it became clear that the constant natural lighting changes would make that difficult.
The subtle transitions were something that Dom&Nic had in mind from the start, so we knew when they wanted the transitions to take place and had planned for it. To help us out, we shot opengate on the Arri Alexa, giving us more pixels top and bottom. This allowed us to re-rack the frame within the crop afterwards. The rest came down to well choreographed dance moves coupled with great steadicam work.
We recommended ARRI’s LDS lenses as The Lens Data System captures highly accurate metadata about iris, focus and zoom. We managed to pipe the focus data into our 3D camera, enabling us to render in depth of field, using real values from the actual lens.
Teaming up with Efficacy4D, we captured a LIDAR scan of the set and then used the scan to ensure that we had a very accurate track. Matching the dancer would’ve been a whole lot harder if we couldn't trust the camera track. We used the scan to project HDRI images for accurate lighting, which was also incredibly useful for making clean plates.
Environmental Lidar scan
Dave: Even though Sonoya is a great dancer, we obviously couldn't expect every take to be identical, which made a motion capture solution somewhat limited. However, we figured that if we were to use a portable motion-capture suit and get Sonoya to run through the choreography a few times on the day, then at least we’d have some animation to start with.
We enlisted the help of Ivo Slivkov from Timeaware, a company that specialises in remote motion capture. The mo-cap suit allowed Sonoya to perform her dance whilst the many sensors captured her in real time.
Even with Ivo’s help, the rest was good old-fashioned 3D animation, a lot of it! With the help of Peanut FX, we mounted 11 ‘witness’ GoPro cameras around the set to record the performance from every angle, with the idea of providing as much reference for the animation team as possible.
Sonoya in her mo-cap suit
Dave: By far the most labour intensive part of the project was the 3D tracking (matchmoving). Our team of animators had to position each limb so that it exactly matched Sonoya’s pose on all 6,798 frames. This is one of the most complicated things that we do in VFX and is a highly intricate and skilled job. How did we overcome this? Patience, lots of patience.
Not only was this the longest shot we have ever tracked, we incorporated a range of new procedural modelling techniques, human body scanning, lidar scanning, motion capture, an extremely complex rig, animated lighting, cloth simulation and a huge amount of cleanup work. These combined added up to an extremely complex blend of techniques.
Dave: Not only did we need to paint out whole parts of Sonoya’s body to reveal a clean background, but also create clean parts of her own body for when her limbs occlude parts of herself.
Cleaning Sonoya out of the backplate traditionally, by hand painting each frame, would have been very time consuming indeed, so we looked into ways of achieving this automatically. Co-3D Lead Suraj (Sid) Harrington-Odedra developed a custom tool that was able to automatically erase given sections of the live action plates, as long as there was an accurate camera track and detailed 3D geometry. This clean plate pass ended up being a great starting point for the overall cleanup.
Sid: There are moments in the video that Sonoya’s ‘real’ limbs and clothes are behind a mesh limb. For these parts of the image, we rendered full CG elements of her arms, legs, head and clothing so they could be used as in areas where traditional 2D painting might be too difficult.
In addition to this, we had to consider the fact that sunlight could travel through the mesh limb and then hit other areas of her body. In order to achieve the correct lighting, we create a light/shadow pass, which the compositors could use to grade up the live action shadow, to make it appear as if light had travelled through the mesh limb.
Fergal: The sheer volume of cleanup to do with removing Sonoya’s limbs from the background meant that Sid’s tool was invaluable. Time saved removing her from the background was then used in the very time consuming and meticulous task of removing her arms when they cross her t-shirt and face.
As she interacts with the t-shirt so much in the image sequence, this required a lot of 2D paintwork, tracking, warping and rebuilding of shadows.