Weta Digital’s Joe Letteri talks HFR 3D and Smaug

hobbit-desolation-smaugStudioDaily.com has an interview with WETA Digital head Joe Letteri where he explains how making The Hobbit in HFR 3D affected WETA’s VFX process:

Studio Daily: What was the impact of 48 fps on post-production?

Joe Letteri: In a way, it was as simple as twice as many frames, so we had to do more work. It did allow us more creativity with animation. When you have 48 frames for every second, you can handle quick changes of motion better. You can see that in Gollum. At 48, you can really define those micro expressions. At 24 fps, the expressions are softer. We capture at 60 frames per second, so we could use more of the motion-capture data.

That WETA is already using 60 fps capture adds to the likelihood that future blockbusters they work on will be made at 60 fps.  I expect announcements of more high frame rate movies soon.  If I were to bet, I’d guess that the next announced HFR movie (beyond what has already been 100% confirmed) will be X-Men: Days of Future Past.  Bryan Singer has been making a string of exciting casting and plot related announcements on his Twitter, and he previously said he had “frame rate envy” after seeing The Hobbit in HFR. So I definitely expect to see this mutant epic in HFR 3D.

Letteri mentions realism as representing the future of visual effects:

Studio Daily: Leaving the business trends aside, what trends do you see technically and artistically in visual effects?

Joe Letteri: Generally, I see more of this trend toward realism. In a way, that’s what we’ve always done. But now, there’s more acknowledging that it is what we do. There’s more of a focus on understanding and trying to apply realism. Even though it’s more complex, it gives you the ability to standardize around a known quantity. There is less guesswork when you measure the real world.

What are you excited about now?

Smaug. He’s our next big character. You just got a few glimpses of him in the first film. I love the Riddles in the Dark, and I love Smaug. Seeing what we can do with Smaug is the next thing.

As I’ve been saying for a while now, I can’t wait to see Smaug.  His reveal at the end of An Unexpected Journey was perfect: from the thrush’s leisurely flight to the The Lonely Mountain, to it knocking the seed on the wall, then the camera taking us into the huge treasure chambers where we hear the amplified echoes of the thrush’s activity as we track over the hills of gold and treasure, leading into the final push-in on the dragon’s eye as we discover that Smaug had been sleeping under the gold coins all this time. I got chills.

Letteri’s love for Smaug and his excitement regarding Smaug as WETA’s “next big character” is very encouraging, as is the implicit promise of new techniques being used to bring Smaug and his environment to life.  We’ve never had a great talking dragon in the movies, not to mention a dragon whose belly is encrusted in dazzling golden coins and gems. Can’t wait to see how WETA plays with the lighting effects.

Even if An Unexpected Journey doesn’t win the best Visual Effects award tonight, I’d bet Smaug will win it for them in 2014.

Check out the StudioDaily.com interview for a lot more fascinating VFX-related discussion from Letteri.

2 more Hobbit clips; VFX supervisor Joe Letteri talks HFR 3D

On the eve of the US release of The Hobbit: An Unexpected Journey we have two new clips. First up is a conversation between Bilbo and Gandalf, via Comingsoon.net/iTunes Movie Trailers:

And Peter Jackson has posted a clip from the classic ‘Riddles in the Dark’ chapter on his Facebook. Check it out:

I can’t wait for this scene: I’m blown away by the quality of WETA’s work here, and Andy Serkis has reportedly (and from what I can see in the above piece) turned in another brilliant performance.  This sequence will also serve as a great showcase for HFR 3D: as Serkis has said, HFR 3D takes the reality of CG characters to a whole new level:

“The wonderful thing about 48 fps is [how it handles] the integration of live action and CG elements; that is something I learned from The Hobbit.  We are so used to 24 fps and the romance of celluloid … but at 48 fps, you cannot deny the existence of these CG creations in the same time frame and space and environment as the live action. It works incredibly well.”

Comingsoon.net has a very interesting interview with ‘Hobbit’ VFX Supervisor Joe Letteri. They get into the topic of HFR 3D and preparing their pipeline for 48 fps:

CS: The other big factor was the fact Peter shot in 48 frames per second, so is all the software your animators use capable of doing 48? 

Letteri: It wasn’t two years ago but by the time we started, we made sure that it was. We talked with everyone that needed to be involved, both for in-house software and anything we were working with externally, like talking to Foundry, to let everyone know this was coming, just like we did with stereo when we did “Avatar.” When we started rigging everything up, we just started prototyping it and showing them what we did and we got them to add that into the released versions of the software, so the same thing happened here. Again, we were ready to go when the production started.

Letteri has some interesting thoughts on how the 48 fps will affect different types of moviegoers, admitting that some viewers may simply be too nostalgically attached to the “film look” (judder, strobing, motion blur) to truly accept and appreciate the 48 fps:

 CS: What’s the general reaction to 48 frames per second among the FX people you work with? Does that seem to be something that will continue into the future and does it help enhance the FX and 3D of it?

Letteri: Well, it solves one problem in 3D, which is motion blur. If you tend to be prone towards motion sickness when you see something really blurry in 3D, this really alleviates that problem, but it does change the look of the film because everything looks hyper-real. Yeah, as Peter mentioned, and I’m hearing the same thing too from the few screenings I’ve been to, when you talk to people, the more familiar you are with film and the more nostalgic you are for film, the harder it is to let go, and after a while, you can’t let go, and you take it for what it is. Anyone who I talked to who is younger who is used to seeing film in any number of different ways says it’s like really not even noticeable, not really a big deal.

Letteri tells us that James Cameron is still shooting for 60 frames per second for the Avatar sequels:

CS: Have you had any conversations with James Cameron yet on whether he might shoot on 48 as well? Has he seen “The Hobbit” yet?

Letteri: Jim’s still thinking 60, because it’s a persistence of vision artifacts and to quote Doug Trumbull and the studies he’s done, he says it tends to go away at around 64. I tend to agree with him from the tests we’ve done. We’ve gone to pretty high frame rates and after about 64, the returns are pretty minimal and to tell you the truth, the difference between 48 and 60 is not as noticeable as the difference between 24 and 48 so for practical reasons, we decided to go with 48 because asking the theaters to have projectors and servers that could handle 60 was not really practical in the time frame we had for “Hobbit” plus it would have meant even more work, double the amount of work, that we were already committed to at 48 frames.

CS: You also have two more years and movies for people to get used to it and it to become more available projection-wise.

Letteri: You have a choice. My recommendation would be that if you’re not certain that this is something you really want to see, see it in 24 frames, and if you want to see the film again, which I hope you do, then give it a try at 48, so that way you get the whole experience. Honestly, people have told me that seeing the 48-frame version of the movie, even though it might have taken them a few minutes to get used to it, when they went back and saw the 24 frame version, it felt like a step down. Even though initially they had the experience of “This is what I’m used to seeing” but I would say that if you’re curious, maybe go the other way, maybe go 24 and then 48.

CS: I might have to see it three or four times in every possible permutation to figure out which one I like.

Letteri: It’s interesting because I still like going back to… because we have to watch every version of it as well before it goes out the door, and it’s really interesting still looking at the film version of it and what’s interesting is that–as nostalgic as I am for film and a lot of people are–after you see the digital and you go back to look at the film, you realize how flickery it is and how jittery the projection is, stuff that when you watch film for 20 or 30 years, it looks fine, you never think about it. As soon as you compare it to what it could be, suddenly the deficiencies become glaringly obvious and I suspect that’s what people are seeing when they go from 48 to 24. Sure, it’s a different look, sure you’re seeing more detail then you may have expected going into it, but you’re seeing it like it’s happening.

I’d love for Cameron to make Avatar 2 & 3 at 60 fps or higher.  Cameron is a film technology pioneer, giving us the first major use of CG in The Abyss and creating the first stereoscopic 3D showcase with Avatar (along with numerous performance capture innovations), so creating the Avatar sequels at an unprecedentedy high frame rate would fit right into his pattern of innovation.  And I can’t think of a better flagship to show off the benefits of HFR 3D than the exotic landscapes, flora and fauna of Pandora.  Since Avatar is nearly 100% CG, the complaints of makeup and sets looking “bad” in HFR will be a nonissue (I’m sure they’ll develop techniques to improve the look of human actors and constructed sets when shot in HFR 3D in the next few years as well).

I do wonder though why Letteri said that making The Hobbit at 60 fps would have been “double the amount of work” of making it at 48 fps.  Wouldn’t double the work be 48 * 2, or 96 fps?  Anyway, the more interesting question about 60 frame per second movies is how a 24 fps version would be extracted to feed theaters that haven’t upgraded, since 60 isn’t a multiple of 24.  Maybe Cameron is planning on only showing Avatar 2 & 3 in HFR 3D?  Or perhaps he’ll decide to go for 72 fps (which would allow a 24 fps version to be created by taking every 3rd frame)? I’m hoping we’ll hear more from Cameron soon.