CG MAKING
July 2015 [CG]
The role of the DF team on set was to be the go-between between the CG production side and the set, taking measurements of the sets inside and outside, gaining an understanding of the lighting situation, and conveying this to the CG production side. It was also to direct filming and production at the motion capture studio. We also worked to align performances with the actors to ensure that no awkwardness or conflict arose in combining the images with the CG characters and frequently proposed different actions for the actors to take while consulting with the director.
The Site Filming Director, Atsushi Doi, and Production Manager, Noriko Sato, joined us to discuss choreography, location shooting and motion capture work. Mr. Doi works as a CG Supervisor in film production.
Differences in Filming Speed from Film Production
Methods to Ease the Team's Work While Keeping the Director's Vision in Mind
There is some lenience with respect to requests and changes from the CG production side that is particular to television and these tend to be accepted. Sometimes, there isn't enough time. For example, we ask to have flying scenes minimized.
Then there are Ryuk's favorite apples (laughs). When Ryuk eats apples, he increases our work time. The amount of work required to model the shape of the apple each time he takes a bite is unwieldy. As the assets increase, it becomes tough on Unreal as well, so reducing the frequency of his biting is one way to reduce this volume. In other words, we had the acting changed to have Ryuk swallow the apple after one bite. Also, the CG characters are huge at over 2 meters, so we have to get the acting to take place such that the actors aren't blocked out by the animation when they are interacting with the star and pay attention to position so that they aren't lost in the background. This might be a detail, but we negotiate minimizing the quantity of on-site material for which we are responsible in order to reduce the number of work processes for CG production. We also have to discuss camera angles with the cameramen a lot. We ask them to record with the cameras fixed as much as possible in order to reduce the cuts in which the camera is moving.
On site, we make suggestions to the director for doing things our way. They can be unique to Ryuk on TV. For example, the director said that he wanted Ryuk to do a moon walk and when we proposed that he would step forward and then back up, that was used (laughs). There are occasions in which demands for choreography have to be conveyed to the location team considering the limits of the CG production site.
Efforts in Motion Capture
Basically, if the motion capture schedule is not aligned, post-processing work is late. Initially, we developed a process in which motion capture was done simultaneously with filming, but we couldn't keep up with the pace of filming on set, so we switched to doing it at the motion capture studio after we receive the offline footage.
We did the first 9 cuts at a temporary motion capture set next door to the Nippon Television Ikuta studio. The actors' and studio's schedule meant that filming of the first four episodes had to be done by the end of June, so we abandoned synchronous recording for motion capture with the studio set recording. The recorded material is given to DF the following day. Offline footage arrives the next day or the morning of the day after that. So what we do is we have actors perform while watching the offline footage on a monitor and motion capture that.
There is one actor each Shinigami character, and actors do a run through prior to the test recording and then the test and actual recording takes place. At a later date, we have them perform while playing the offline footage at the motion capture studio, so performances (especially for Ryuk's role) have to be repeated and that's quite onerous.
Actors have to take part in the motion capture in addition to the actual recording on the same day, so sometimes we are able to complete the work quickly within 1 to 2 hours and with no discrepancies in acting compared to the actual recording. Typically it takes a full day.
The facial motion capture system, Dynamixyz*4, doesn't involve attaching markers to the face as do conventional systems. Instead, a head mounted camera is used to film the face and capture facial movement.
*1 360°Omnidirectional camera
*2 3D coordinate measuring machine that acquires coordinate information of structures without contact using laser light
*3 Software that creates high-accuracy 3D models from photographs
*4 Video-based facial capture system developed by France's Dynamixyz
DOI
One of the biggest differences between television programs and movies is the lack of base lighting used to create intimacy with CG. Also, in filming on studio sets for television programs, 6 cameras run at the same time. Simultaneous recording takes place without any cuts and a switcher is used to make cuts, so this process feels 3 times faster than when filming a movie. Ryuk appeared often, so the number of cuts we had to handle was surprisingly large (laughs). It was a much more comical performance than when the movie was filmed. The greater the involvement of CG characters, the more fun it is for those watching.
There weren't any major differences in terms of the contents of the work we did on site. Flares were recorded using silver and white balls for compositing and raw material for HDRI (High Dynamic Range Images) was recorded. During the production of the movie, regular lighting was used and HDRI was not used. Unreal allows HDRI to be used, so we were grateful for that help. HDRI recording typically uses single-lens reflex, but we recorded with Ricoh's THETA*1 to emphasize speed and simplify things. We try to avoid complicating the work when we leave the set and convert the material into HDRI in order to keep up the television program production pace as much as possible. Also, instead of measuring the background, we use FARO 3D laser scan*2 and Acute 3D Smart 3D Capture *3 for efficiency. In this case, the number of cuts in which CG play a role is large and the number of sets is large, so measuring everything by the regular methods would increase the number of steps and the time tremendously. In terms of background modeling, we have about 20 to 30 scenes at this point. Every time lighting changes, we have to re-measure, but thanks to this equipment and software, we only have to measure some things, which helps us model and match as we go.