Jon Landau had a session / keynote at NAB on April 7 where he discussed his passions in filmmaking and the impact of technologies such as HFR and 3D. He also touched on the new underwater performance capture tech that will be used in Avatar 2 and 3 and the development of their new studio in Manhattan Beach. Watch the video of the session below:
The Hollywood Reporter has a good summary of Landau’s talk. But there’s still no firm start date(s) for the Avatar sequels’ motion capture and shooting, and the question of whether they will be 48 fps or 60 fps still hasn’t been answered definitively.
Q: Can you give us some scoop about your upcoming little film called “Avatar 2″
Cameron: “Oh believe me, it’s not that little! It’s not exactly a little, intimate drama. I’m working on ‘Avatar 2′ and ‘Avatar 3.’ I was talking the other day with Peter Jackson and said, ‘You had it easy dude. You had the books when you did the second and third ‘Lord of the Rings.’ I have to create my own books in my head and extract a script from it. I’m deep into it and I’m living in Pandora right now. There is that start up torque where you feel it’s coming to you. Then you build up momentum. That’s when it gets fun. The characters talk and it’s writing itself. I’m almost there right now. It’s building fast.”
Q: Tell us a little bit about your life. For example, what is your writing process?
Cameron: “As a writer, I need isolation. I’m calling you from New Zealand right now where I’m writing on a little farm. When you live in a special world like Pandora, you have to live in that world.”
Q: Do you ever feel the pressure of topping yourself? And do you have a release date you can share with us for “Avatar 2 and 3?”
A: “Pressure, no. It’s a little daunting because sequels are always tricky. You have to be surprising and stay ahead of audience anticipation. At the same time, you have to massage their feet with things that they know and love about the first film. I’ve walked that line in the past, so I’m not too worried about it. At the same time, I definitely have to deliver the goods…As for a release date that will be determined by when I get the script out. No pressure!”
Be sure to check out the full interview for talk about Cameron’s 3D Cirque Du Soleil movie (which was apparently shot in HFR 3D but never distributed in HFR), his life in New Zealand, what’s he’s learned since he was 18 and more.
Best Director Oscar winner Ang Lee talked with ScreenRant about HFR on the red carpet of the International 3D Society Awards:
People have mixed feelings right now. I wonder how much HFR is ahead of its time or people simply don’t like the look. It’s very hard to say. We associate it with “video” looks – which people associate with bad filmmaking. It doesn’t mean that the media itself is not good. It’s very hard to say but making Life of Pi I struggled with frame rate because you don’t want the 3D to be jittery and we’re constantly rocking in the ocean. And sometimes when things go too fast I could not see the eyes – so HFR might be a idea. But sometimes when I find out how people feel about it, I think it’s possible. We’re in the early stages of 3D filmmaking – so we have a lot to learn.
It seems that Lee is mostly worried about the apparently mixed reaction to the HFR 3D version of The Hobbit (amongst critics, at least…audience responses were significantly more positive) since he acknowledges that the judder-reducing HFR would’ve been particularly helpful for Life of Pi given the many scenes taking place on the rocking ocean.
Film traditionalists have certainly slowed the momentum of the HFR format, but I expect its popularity to surge quite soon – possibly as soon as The Desolation of Smaug. Now that HFR has already made its first splash there will be far fewer luddite hipster critics panning the movie for not being 24 fps. And when the Avatar sequels hit we’ll be getting HFR 3D at a whole new level of clarity: many filmgoers will consider them the first “proper” HFR movies, especially if 60 fps becomes the standard for a while.
Jon Landau is scheduled to deliver the keynote address at the NAB show’s Technology Summit on Cinema: Advances in Image and Sound on Sunday, April 7. It’s very likely Landau’s talk will explore the high frame rate technology / pipeline that he and James Cameron will be using for the Avatar sequels. Hopefully Landau will confirm that they will be making the sequels at 60 fps and that performance capture / shooting will begin soon.
Wendy Aylsworth, president of SMPTE, says of Landau:
“As one of the industry’s most successful producers and storytellers, Jon Landau is a hero to many within the NAB Show audience. He is a champion of employing the capabilities of technology to improve the telling of a story and has inspired many to push the envelope in movie-making.” (via BroadcastEngineering.com)
Besides Landau’s talk, the following sessions will include discussion of high frame rate: (via SMPTE’s press release):
1) ”‘Advancing Cameras for Cinema’ will discuss developments such as higher resolution and frame rates, as well as greater sensitivity, dynamic range, and color gamut, and their potential impact both on acquisition techniques and on human perception of the on-screen images.
2) “Two subsequent sessions will take a closer look at high frame rate (HFR) motion pictures, recent research on the psychophysical audience response to HFR, and how industry producers and directors are using 48fps and 60fps content to achieve a desired emotional audience response.”
Nabshow.com describes the the Technology Summit as providing “an in-depth global view of the new wave of technology coming soon to your local multiplex, with an eye toward how it might later affect the broader media ecosystem.”
Topics Include:
The latest work on higher-frame-rate cinema
Perceptual requirements for higher quality image and sound
New technologies for exhibition
Advantages and pitfalls of 3D film conversion
The 2013 NAB Technology Summit on Cinema will be held from 8:30 am Saturday, April 6 to 6 pm Sunday, April 7 at the Las Vegas Convention Center, South Hall Conference Room S222. The Summit is co-produced by the Society of Motion Picture and Television Engineers (SMPTE).
StudioDaily.com has an interview with WETA Digital head Joe Letteri where he explains how making The Hobbit in HFR 3D affected WETA’s VFX process:
Studio Daily: What was the impact of 48 fps on post-production?
Joe Letteri: In a way, it was as simple as twice as many frames, so we had to do more work. It did allow us more creativity with animation. When you have 48 frames for every second, you can handle quick changes of motion better. You can see that in Gollum. At 48, you can really define those micro expressions. At 24 fps, the expressions are softer. We capture at 60 frames per second, so we could use more of the motion-capture data.
That WETA is already using 60 fps capture adds to the likelihood that future blockbusters they work on will be made at 60 fps. I expect announcements of more high frame rate movies soon. If I were to bet, I’d guess that the next announced HFR movie (beyond what has already been 100% confirmed) will be X-Men: Days of Future Past. Bryan Singer has been making a string of exciting casting and plot related announcements on his Twitter, and he previously said he had “frame rate envy” after seeing The Hobbit in HFR. So I definitely expect to see this mutant epic in HFR 3D.
Letteri mentions realism as representing the future of visual effects:
Studio Daily: Leaving the business trends aside, what trends do you see technically and artistically in visual effects?
Joe Letteri: Generally, I see more of this trend toward realism. In a way, that’s what we’ve always done. But now, there’s more acknowledging that it is what we do. There’s more of a focus on understanding and trying to apply realism. Even though it’s more complex, it gives you the ability to standardize around a known quantity. There is less guesswork when you measure the real world.
What are you excited about now?
Smaug. He’s our next big character. You just got a few glimpses of him in the first film. I love the Riddles in the Dark, and I love Smaug. Seeing what we can do with Smaug is the next thing.
As I’ve been saying for a while now, I can’t wait to see Smaug. His reveal at the end of An Unexpected Journey was perfect: from the thrush’s leisurely flight to the The Lonely Mountain, to it knocking the seed on the wall, then the camera taking us into the huge treasure chambers where we hear the amplified echoes of the thrush’s activity as we track over the hills of gold and treasure, leading into the final push-in on the dragon’s eye as we discover that Smaug had been sleeping under the gold coins all this time. I got chills.
Letteri’s love for Smaug and his excitement regarding Smaug as WETA’s “next big character” is very encouraging, as is the implicit promise of new techniques being used to bring Smaug and his environment to life. We’ve never had a great talking dragon in the movies, not to mention a dragon whose belly is encrusted in dazzling golden coins and gems. Can’t wait to see how WETA plays with the lighting effects.
Even if An Unexpected Journey doesn’t win the best Visual Effects award tonight, I’d bet Smaug will win it for them in 2014.
It’s evident that 48 frames per second is only the beginning of the upcoming high frame rate revolution: “HFR” is a more accurate and encompassing reflection of the scope of the tech as well as a better fit in regards to what the movie studios and industry at large are beginning to call frame rates of 48 fps and higher.
Avatar 2 and Avatar 3 are likely to be made at 60 fps. And Douglas Trumbull’s upcoming sci-fi project is targeting 120 fps. Persistence of vision (the point at which the eye perceives distinct frames as perfectly smooth motion) doesn’t kick in until 60 fps at the absolute minimum (Trumbull has said it’s closer to 64 fps), so I see constant 48 fps as a standard that won’t last long: the future will be all about 60 fps and perhaps even higher. Or, quite possibly, variable frame rates (VFR): the utilization of multiple frame rates within a single movie, or even within a single shot. To enable VFR 3D, however, projectors must be capable of delivering HFR 3D content at 60fps+. So I am personally pushing for projector tech that can do 3D at 60fps+ per eye, and at 4K per eye as well. Keep in mind that 4K per eye is not yet possible even at 48 fps 3D due to bandwidth limitations.
So welcome to HFR Movies.com: I’ll be continuing to write about everything and anything relating to HFR movies and tech. If you have an idea for an article or a piece of news you’d like to send my way, please email me at mstat1@gmail.com.
Thanks, and let us all look forward to a world without awful judder and incomprehensible action scenes!
Director of WETA Digital Joe Letteri spoke with Movieline.com about HFR 3D, comparing those decrying HFR 3D with the people who lashed out at color films when color technology first arrived:
If you grew up seeing films in black and white and suddenly start seeing films in color, some people are going to have the reaction ‘Wow, that’s great!’ and other people are going to have the reaction, ‘That’s not moviemaking! Films should be made in black and white! You’re losing the mystery of how to deal with tonality, you’re sacrificing that to deal with color!’ But if you grew up with only seeing color, you don’t know that. Just talking to the people that have seen [HFR 3D] so far, and obviously that’s been a very limited audience, the younger ones that I’ve spoken with don’t really have an issue with it because they’re not so ingrained with what 24 frames mean. To them they’re just watching a movie.
Letteri emphasizes the significant benefits high frame rates bestow upon stereoscopic 3D movies:
I think it’s beneficial with stereo. One of the artifacts of stereo — for example, if you look at something that’s out of focus, maybe an over the shoulder shot, this is where stereo differs from the real world. In the real world, wherever your eye focuses that object snaps to focus. So if you’re looking at a big screen and your eye wanders to something that is out of focus, your eye expects it to go into focus. It can’t. So you’re sitting there focusing on an out of focus object. That’s one of the things that causes your brain [to fritz]. The same thing also happens temporally. If you’ve got an object moving across the screen in the real world your eye wants to be able to track that and your eye wants to see it in focus. But because you’ve already photographed that with this motion blur, your eye cannot focus on something in space that’s blurry. Again, in the real world you never see that. It’s one of those other things that in 3-D your brain says something’s not right here. Well, if you go to high frame right, it is in focus. Your eye can focus on these fast moving objects or even slow moving ones and the detail’s always there. So your brain can make sense of it.
His point regarding the ability of HFR to mimic the eye’s tendency/ability to track and bring into focus fast-moving objects is interesting…not only does HFR result in smooth motion during action sequences (especially helping with the choppy motion found in 24 fps 3D movies), but it also reduces eye strain by allowing the eye to do what it would do during real-life fast movement: track objects on the screen. The primary cause of eyestrain in stereoscopic 3D is the eye’s inability to bring blurred objects into focus. While HFR 3D doesn’t solve the issue of the inability to focus on background objects during a shallow focus shot of, say, a character’s face, it does allow the viewer to focus on fast moving objects. And if the viewer chooses not to focus his/her eyes on, for example, an arrow flying across the screen, the brain will create its own motion blur just like it does in real life.
I think 48 fps still isn’t quite high enough to 100% fix the issue of blur during movement. Douglass Trumbull, Letteri and others seem to have reached a consensus that at 60 fps+ (especially 64 fps+) movements on screen are interpreted by the brain as real:
Jim [Cameron] is considering 60 fps [for Avatar 2]. That’s closer to where persistence of vision almost disappears. In fact, these discussions came out of when we noticed the effect of that in Avatar. And we were brainstorming with Jim on how to fix it — well, this is inherent in the photography and the only thing you can do is go shorter shutter, but that introduces strobing, or you can go higher frame rate. We started experimenting with higher frame rate [from a standpoint of] how do we solve the problem? It looks like something happening live.
I really want to know the exact number the industry will settle upon for HFR. Will it be 60 fps, which is perhaps a few frames per second shy of the point where increases in frame rate become unnoticeable by most? Will it be 72 fps or 96, which are multiples of 24, thus allowing for accurate downconversion to 24 fps? Or will it be 120 (a multiple of both 24 and 60), allowing for easy downconversion to both 60 fps (for TVs) and 24 fps (for film projection)? I don’t see it going higher than 120 fps per eye any time in the next decade. At that high of a frame rate even the most hardcore videophiles would be very unlikely to notice further increases.
In any case I think it will benefit the future of high frame rates as a format if a standard is chosen relatively soon.
On the eve of the US release of The Hobbit: An Unexpected Journey we have two new clips. First up is a conversation between Bilbo and Gandalf, via Comingsoon.net/iTunes Movie Trailers:
And Peter Jackson has posted a clip from the classic ‘Riddles in the Dark’ chapter on his Facebook. Check it out:
I can’t wait for this scene: I’m blown away by the quality of WETA’s work here, and Andy Serkis has reportedly (and from what I can see in the above piece) turned in another brilliant performance. This sequence will also serve as a great showcase for HFR 3D: as Serkis has said, HFR 3D takes the reality of CG characters to a whole new level:
“The wonderful thing about 48 fps is [how it handles] the integration of live action and CG elements; that is something I learned from The Hobbit. We are so used to 24 fps and the romance of celluloid … but at 48 fps, you cannot deny the existence of these CG creations in the same time frame and space and environment as the live action. It works incredibly well.”
Comingsoon.net has a very interesting interview with ‘Hobbit’ VFX Supervisor Joe Letteri. They get into the topic of HFR 3D and preparing their pipeline for 48 fps:
CS: The other big factor was the fact Peter shot in 48 frames per second, so is all the software your animators use capable of doing 48?
Letteri: It wasn’t two years ago but by the time we started, we made sure that it was. We talked with everyone that needed to be involved, both for in-house software and anything we were working with externally, like talking to Foundry, to let everyone know this was coming, just like we did with stereo when we did “Avatar.” When we started rigging everything up, we just started prototyping it and showing them what we did and we got them to add that into the released versions of the software, so the same thing happened here. Again, we were ready to go when the production started.
Letteri has some interesting thoughts on how the 48 fps will affect different types of moviegoers, admitting that some viewers may simply be too nostalgically attached to the “film look” (judder, strobing, motion blur) to truly accept and appreciate the 48 fps:
CS: What’s the general reaction to 48 frames per second among the FX people you work with? Does that seem to be something that will continue into the future and does it help enhance the FX and 3D of it?
Letteri: Well, it solves one problem in 3D, which is motion blur. If you tend to be prone towards motion sickness when you see something really blurry in 3D, this really alleviates that problem, but it does change the look of the film because everything looks hyper-real. Yeah, as Peter mentioned, and I’m hearing the same thing too from the few screenings I’ve been to, when you talk to people, the more familiar you are with film and the more nostalgic you are for film, the harder it is to let go, and after a while, you can’t let go, and you take it for what it is. Anyone who I talked to who is younger who is used to seeing film in any number of different ways says it’s like really not even noticeable, not really a big deal.
Letteri tells us that James Cameron is still shooting for 60 frames per second for the Avatar sequels:
CS: Have you had any conversations with James Cameron yet on whether he might shoot on 48 as well? Has he seen “The Hobbit” yet?
Letteri: Jim’s still thinking 60, because it’s a persistence of vision artifacts and to quote Doug Trumbull and the studies he’s done, he says it tends to go away at around 64. I tend to agree with him from the tests we’ve done. We’ve gone to pretty high frame rates and after about 64, the returns are pretty minimal and to tell you the truth, the difference between 48 and 60 is not as noticeable as the difference between 24 and 48 so for practical reasons, we decided to go with 48 because asking the theaters to have projectors and servers that could handle 60 was not really practical in the time frame we had for “Hobbit” plus it would have meant even more work, double the amount of work, that we were already committed to at 48 frames.
CS: You also have two more years and movies for people to get used to it and it to become more available projection-wise.
Letteri: You have a choice. My recommendation would be that if you’re not certain that this is something you really want to see, see it in 24 frames, and if you want to see the film again, which I hope you do, then give it a try at 48, so that way you get the whole experience. Honestly, people have told me that seeing the 48-frame version of the movie, even though it might have taken them a few minutes to get used to it, when they went back and saw the 24 frame version, it felt like a step down. Even though initially they had the experience of “This is what I’m used to seeing” but I would say that if you’re curious, maybe go the other way, maybe go 24 and then 48.
CS: I might have to see it three or four times in every possible permutation to figure out which one I like.
Letteri: It’s interesting because I still like going back to… because we have to watch every version of it as well before it goes out the door, and it’s really interesting still looking at the film version of it and what’s interesting is that–as nostalgic as I am for film and a lot of people are–after you see the digital and you go back to look at the film, you realize how flickery it is and how jittery the projection is, stuff that when you watch film for 20 or 30 years, it looks fine, you never think about it. As soon as you compare it to what it could be, suddenly the deficiencies become glaringly obvious and I suspect that’s what people are seeing when they go from 48 to 24. Sure, it’s a different look, sure you’re seeing more detail then you may have expected going into it, but you’re seeing it like it’s happening.
I’d love for Cameron to make Avatar 2 & 3 at 60 fps or higher. Cameron is a film technology pioneer, giving us the first major use of CG in The Abyss and creating the first stereoscopic 3D showcase with Avatar (along with numerous performance capture innovations), so creating the Avatar sequels at an unprecedentedy high frame rate would fit right into his pattern of innovation. And I can’t think of a better flagship to show off the benefits of HFR 3D than the exotic landscapes, flora and fauna of Pandora. Since Avatar is nearly 100% CG, the complaints of makeup and sets looking “bad” in HFR will be a nonissue (I’m sure they’ll develop techniques to improve the look of human actors and constructed sets when shot in HFR 3D in the next few years as well).
I do wonder though why Letteri said that making The Hobbit at 60 fps would have been “double the amount of work” of making it at 48 fps. Wouldn’t double the work be 48 * 2, or 96 fps? Anyway, the more interesting question about 60 frame per second movies is how a 24 fps version would be extracted to feed theaters that haven’t upgraded, since 60 isn’t a multiple of 24. Maybe Cameron is planning on only showing Avatar 2 & 3 in HFR 3D? Or perhaps he’ll decide to go for 72 fps (which would allow a 24 fps version to be created by taking every 3rd frame)? I’m hoping we’ll hear more from Cameron soon.
Collider scored an exclusive interview with Titanic and Avatar producer Jon Landau in advance of Titanic’s blu-ray debut. It’s focused primarily on Titanic, but Landau shares his thoughts on 48 fps and 60 fps, The Hobbit‘s limited HFR release, as well as the status of Avatar 2 and 3:
What are your thoughts on 48 frames per second and 60 frames per second? Do you think that is the future of projection?
LANDAU: I think it’s a great way to go. I don’t think we have to choose 48 or 60. I think filmmakers should be able to choose. The technology is there. The projectors can play either one. I think filmmakers need to drive this. We’ll get rid of strobing, we’ll have brighter light levels, and we’ll see it evolve, down the road. But, I think it’s an evolutionary step, not a revolutionary step. So, I don’t think it’s going to happen overnight, but I think it’s something that will happen over time.
What are your thoughts on Warner Bros. seeming to have pulled back a bit from 48fps on The Hobbit, after the reaction from critics wasn’t as enthusiastic as they would have liked it to be?
LANDAU: I think that’s a misconception. I think what they are doing is managing expectations. If they come out there and say, “Hey, it’s at high frame rates,” but the theater in Des Moines, Iowa doesn’t have higher frame rates, people will say, “Wait, they promised it to me.” I think what they’re saying is, “Look, in this first initial pass, it’s going to be in selected theaters.” It can’t be everywhere, so I think they’re handling it very smartly.
The release dates for the second and third Avatar movies seem to be in flux a bit. Do you have a current status for the release dates and how far along those scripts are now?
LANDAU: The scripts are pretty far along. We have a team of people already working on the film at Manhattan Beach Studios. We’ve been working with WETA Digital on technologies. You know, Titanic was supposed to be a summer movie and it was a Christmas movie. We’re going to make the best possible version of these movies, and they will come out when they are exactly that.
I’m excited that a team is already working on what I’m guessing is prep work relating to the motion capture element of Avatar 2 and 3…on the first Avatar lots of early work was done simply figuring stuff out, i.e. how to mo-cap the banshee. I’d guess nothing (or at least not much) with the principal actors has happened yet. I think that would qualify as big news / the official start of production, and there’d likely be a press release from Fox / Lightstorm.
As to The Hobbit’s limited release, I think Landau is correct that it’s a good move for WB to manage expectations if HFR-enabled projectors aren’t going to be widespread by December. 48 fps screen count estimates over the past months have been very high, so it wouldn’t surprise me if the projector companies were simply too optimistic with their numbers, especially since required software upgrades for many projectors weren’t scheduled to become available until sometime in September.
Head on over to Collider for the full talk with Landau, including his thoughts on Titanic and details of the blu-ray release, other recent advances in technology he wishes to incorporate into future movies, and more.
