Weta Digital’s Joe Letteri talks HFR 3D and Smaug

hobbit-desolation-smaugStudioDaily.com has an interview with WETA Digital head Joe Letteri where he explains how making The Hobbit in HFR 3D affected WETA’s VFX process:

Studio Daily: What was the impact of 48 fps on post-production?

Joe Letteri: In a way, it was as simple as twice as many frames, so we had to do more work. It did allow us more creativity with animation. When you have 48 frames for every second, you can handle quick changes of motion better. You can see that in Gollum. At 48, you can really define those micro expressions. At 24 fps, the expressions are softer. We capture at 60 frames per second, so we could use more of the motion-capture data.

That WETA is already using 60 fps capture adds to the likelihood that future blockbusters they work on will be made at 60 fps.  I expect announcements of more high frame rate movies soon.  If I were to bet, I’d guess that the next announced HFR movie (beyond what has already been 100% confirmed) will be X-Men: Days of Future Past.  Bryan Singer has been making a string of exciting casting and plot related announcements on his Twitter, and he previously said he had “frame rate envy” after seeing The Hobbit in HFR. So I definitely expect to see this mutant epic in HFR 3D.

Letteri mentions realism as representing the future of visual effects:

Studio Daily: Leaving the business trends aside, what trends do you see technically and artistically in visual effects?

Joe Letteri: Generally, I see more of this trend toward realism. In a way, that’s what we’ve always done. But now, there’s more acknowledging that it is what we do. There’s more of a focus on understanding and trying to apply realism. Even though it’s more complex, it gives you the ability to standardize around a known quantity. There is less guesswork when you measure the real world.

What are you excited about now?

Smaug. He’s our next big character. You just got a few glimpses of him in the first film. I love the Riddles in the Dark, and I love Smaug. Seeing what we can do with Smaug is the next thing.

As I’ve been saying for a while now, I can’t wait to see Smaug.  His reveal at the end of An Unexpected Journey was perfect: from the thrush’s leisurely flight to the The Lonely Mountain, to it knocking the seed on the wall, then the camera taking us into the huge treasure chambers where we hear the amplified echoes of the thrush’s activity as we track over the hills of gold and treasure, leading into the final push-in on the dragon’s eye as we discover that Smaug had been sleeping under the gold coins all this time. I got chills.

Letteri’s love for Smaug and his excitement regarding Smaug as WETA’s “next big character” is very encouraging, as is the implicit promise of new techniques being used to bring Smaug and his environment to life.  We’ve never had a great talking dragon in the movies, not to mention a dragon whose belly is encrusted in dazzling golden coins and gems. Can’t wait to see how WETA plays with the lighting effects.

Even if An Unexpected Journey doesn’t win the best Visual Effects award tonight, I’d bet Smaug will win it for them in 2014.

Check out the StudioDaily.com interview for a lot more fascinating VFX-related discussion from Letteri.

WETA head Joe Letteri talks reactions to HFR…and what frame rate will the industry settle on?

Director of WETA Digital Joe Letteri spoke with Movieline.com about HFR 3D, comparing those decrying HFR 3D with the people who lashed out at color films when color technology first arrived:

If you grew up seeing films in black and white and suddenly start seeing films in color, some people are going to have the reaction ‘Wow, that’s great!’ and other people are going to have the reaction, ‘That’s not moviemaking! Films should be made in black and white! You’re losing the mystery of how to deal with tonality, you’re sacrificing that to deal with color!’ But if you grew up with only seeing color, you don’t know that. Just talking to the people that have seen [HFR 3D] so far, and obviously that’s been a very limited audience, the younger ones that I’ve spoken with don’t really have an issue with it because they’re not so ingrained with what 24 frames mean. To them they’re just watching a movie.

Letteri emphasizes the significant benefits high frame rates bestow upon stereoscopic 3D movies:

I think it’s beneficial with stereo. One of the artifacts of stereo — for example, if you look at something that’s out of focus, maybe an over the shoulder shot, this is where stereo differs from the real world. In the real world, wherever your eye focuses that object snaps to focus. So if you’re looking at a big screen and your eye wanders to something that is out of focus, your eye expects it to go into focus. It can’t. So you’re sitting there focusing on an out of focus object. That’s one of the things that causes your brain [to fritz]. The same thing also happens temporally. If you’ve got an object moving across the screen in the real world your eye wants to be able to track that and your eye wants to see it in focus. But because you’ve already photographed that with this motion blur, your eye cannot focus on something in space that’s blurry. Again, in the real world you never see that. It’s one of those other things that in 3-D your brain says something’s not right here. Well, if you go to high frame right, it is in focus. Your eye can focus on these fast moving objects or even slow moving ones and the detail’s always there. So your brain can make sense of it.

His point regarding the ability of HFR to mimic the eye’s tendency/ability to track and bring into focus fast-moving objects is interesting…not only does HFR result in smooth motion during action sequences (especially helping with the choppy motion found in 24 fps 3D movies), but it also reduces eye strain by allowing the eye to do what it would do during real-life fast movement: track objects on the screen. The primary cause of eyestrain in stereoscopic 3D is the eye’s inability to bring blurred objects into focus. While HFR 3D doesn’t solve the issue of the inability to focus on background objects during a shallow focus shot of, say, a character’s face, it does allow the viewer to focus on fast moving objects. And if the viewer chooses not to focus his/her eyes on, for example, an arrow flying across the screen, the brain will create its own motion blur just like it does in real life.

I think 48 fps still isn’t quite high enough to 100% fix the issue of blur during movement. Douglass Trumbull, Letteri and others seem to have reached a consensus that at 60 fps+ (especially 64 fps+) movements on screen are interpreted by the brain as real:

Jim [Cameron] is considering 60 fps [for Avatar 2]. That’s closer to where persistence of vision almost disappears. In fact, these discussions came out of when we noticed the effect of that in Avatar. And we were brainstorming with Jim on how to fix it — well, this is inherent in the photography and the only thing you can do is go shorter shutter, but that introduces strobing, or you can go higher frame rate. We started experimenting with higher frame rate [from a standpoint of] how do we solve the problem? It looks like something happening live.

I really want to know the exact number the industry will settle upon for HFR. Will it be 60 fps, which is perhaps a few frames per second shy of the point where increases in frame rate become unnoticeable by most? Will it be 72 fps or 96, which are multiples of 24, thus allowing for accurate downconversion to 24 fps? Or will it be 120 (a multiple of both 24 and 60), allowing for easy downconversion to both 60 fps (for TVs) and 24 fps (for film projection)? I don’t see it going higher than 120 fps per eye any time in the next decade. At that high of a frame rate even the most hardcore videophiles would be very unlikely to notice further increases.

In any case I think it will benefit the future of high frame rates as a format if a standard is chosen relatively soon.