Hollywood is using Unity, eg. for Lion King 2019

[Peter77]

I was watching a video about what fancy technology Hollywood came up with to replace green/blue-screens (link).

In simple words, the green screen is replaced with a real-time rendered background, where the rendered content changes as the actual camera changes. The revolution according to the “documentary” is that real-life actors in-front of the virtual scene get correctly lit and reflections of real-life props are correctly as well, which is something you don’t get with green screens for free.

However, while they were showing their fancy tech, I though “Hey… I know that program, that looks like the Unity editor!”.
Screenshots

So I fired up google to figure out if really Unity or just my imagination. But yep, Hollywood is using Unity:

(…) building a virtual Serengeti using the Unity game engine, then adding the digital equivalent of real-world production elements like lights and dolly tracks. The filmmakers could put on Vive VR headsets to explore the world, use iPads to adjust the various elements and then shoot scenes using a virtual camera. (…)

'Lion King' director Jon Favreau explains why he's remaking an animated...

(…) It then developed an “asset management system” that could translate the scene into something that was compatible with Unity, a popular game engine.
(…)
The three-dimensional flight path was tracked and reflected inside the virtual world, allowing the team to ‘shoot’ the master scenes in Unity.
(…)
MPC’s task was to translate that camerawork, which had been shot in the Unity game engine, into a version of the movie that had final, production quality, assets and animation.
(…)

Inside the virtual production of ‘The Lion King’

Really impressive to see how Unity turned from that game engine for hobbyists to the next big thing in Hollywood.

[neginfinity]

I’m not sure if it is “the next big thing”, though. Likely one more tool in the toolbox.

Kinda reminds me how common lisp was used in creation of Gollum in Lord of the Rings.
(Mirai (software) - Wikipedia)

Normally unreal is the engine used for this kind of stuff.

[Kiwasi]

Yup. This has been going on for a while.

Unity is particularly useful to film makers in a couple of ways:

• It lets directors see motion capture data in real time as it’s captured. This reduces the need for retakes.
• Combined with VR it lets see and interact with objects that will be added later via special effects
• It lets multiple motion capture actors work together in the same virtual space, without interrupting motion capture equipment.

Unity is never going to be used to render movies. But it has a place in realtime rough renders.

[hippocoder]

Yes, Unity actually expects game dev to not be the most dominant use of Unity in the future. That might be taken by animation or even VFX. With HDRP Raytracing, and all the supporting tools, it’s a pretty great choice and insanely, insanely cost-effective.

[Murgilod]

@hippocoder

Yes, Unity actually expects game dev to not be the most dominant use of Unity in the future. That might be taken by animation or even VFX. With HDRP Raytracing, and all the supporting tools, it’s a pretty great choice and insanely, insanely cost-effective.

Business level AR is probably going to be huge too now that Microsoft has found their niche for Hololens.

[neginfinity]

@Murgilod

Business level AR is probably going to be huge too now that Microsoft has found their niche for Hololens.

Which niche is that?

[Murgilod]

@neginfinity

Which niche is that?

Recently got a job offer (largely misplaced) from a place in Toronto looking to use it for on-site visualization for factory setups/construction. Apparently there’s a bit of an AR push in technical areas like that since it reduces turnaround time.

[neginfinity]

@Murgilod

Recently got a job offer (largely misplaced) from a place in Toronto looking to use it for on-site visualization for factory setups/construction. Apparently there’s a bit of an AR push in technical areas like that since it reduces turnaround time.

You can do that with a phone, though?

[zombiegorilla]

@Murgilod

Recently got a job offer (largely misplaced) from a place in Toronto looking to use it for on-site visualization for factory setups/construction. Apparently there’s a bit of an AR push in technical areas like that since it reduces turnaround time.

Totally! That area is huge right now. I get contacts all the time about AR projects for construction/event/outdoor advertising / etc. it’s kinda crazy how much that has sprung up. Barely even heard about a year or so ago, now it’s all the time. I’ve politely declined offers, but it’s an interesting fallback.

[zombiegorilla]

@neginfinity

You can do that with a phone, though?

The contacts I got were exactly that. Either for client to use directly or a salesperson to take to a site on phone/pad. Most were more about a backend that allowed them to take cad models set them up physical locations so clients could view them (and switch/edit them a bit ) in place.

[Murgilod]

@zombiegorilla

The contacts I got were exactly that. Either for client to use directly or a salesperson to take to a site on phone/pad. Most were more about a backend that allowed them to take cad models set them up physical locations so clients could view them (and switch/edit them a bit ) in place.

Hololens is making inroads because while you CAN do it with phones, the tracking systems and visualization stuff in Hololens is really next level stuff. It’s also a bit more handy because you can just look where you need to.

edit: not just view tracking, but there’s some key advancements being made in glove-free hand tracking as well. It’s all super cool stuff, but it’s not much consumer application wise, obviously

[zombiegorilla]

@Murgilod

Hololens is making inroads because while you CAN do it with phones, the tracking systems and visualization stuff in Hololens is really next level stuff. It’s also a bit more handy because you can just look where you need to.

edit: not just view tracking, but there’s some key advancements being made in glove-free hand tracking as well. It’s all super cool stuff, but it’s not much consumer application wise, obviously

Indeed. In fact I was very tempted by one project because hololens was the prime platform. It’s just… marketing/advertising, I started my career there and don’t want to go back. Hololens games though… that could be fun.

[Murgilod]

@zombiegorilla

Indeed. In fact I was very tempted by one project because hololens was the prime platform. It’s just… marketing/advertising, I started my career there and don’t want to go back. Hololens games though… that could be fun.

Honestly, don’t think there’s going to be any real call for Hololens games until the thing gets a few more price drops like the last one. Gonna have to be sub-$500 and I don’t think the pipeline is there enough to drive that within the next decade.

[zombiegorilla]

True, even in that case it would likely be fringe/speciality. I am just hoping there will at least be enough interest in AR games for me to justify billing a client for one.

[neoshaman]

We need sub 300$ more like for widespread, and a killer app like pokemon something virtual pet

[neginfinity]

@zombiegorilla

The contacts I got were exactly that. Either for client to use directly or a salesperson to take to a site on phone/pad. Most were more about a backend that allowed them to take cad models set them up physical locations so clients could view them (and switch/edit them a bit ) in place.

I actually worked on something similar. It wasn’t cars, though, and I can’t tell details.

---

@Murgilod

Hololens is making inroads because while you CAN do it with phones, the tracking systems and visualization stuff in Hololens is really next level stuff. It’s also a bit more handy because you can just look where you need to.

edit: not just view tracking, but there’s some key advancements being made in glove-free hand tracking as well. It’s all super cool stuff, but it’s not much consumer application wise, obviously

Wasn’t google pushing their AR kit? Not all phones support it, but it had functions for lighting estimation and other things.

[koirat]

@hippocoder

Yes, Unity actually expects game dev to not be the most dominant use of Unity in the future.

If I allow myself to be a little salty.
It is to be expected if they don’t change the current way of things.

[hippocoder]

@koirat

If I allow myself to be a little salty.
It is to be expected if they don’t change the current way of things.

No, it’s the CEO’s announcement a couple of years ago.

[Murgilod]

@neginfinity

Wasn’t google pushing their AR kit? Not all phones support it, but it had functions for lighting estimation and other things.

Things that you see on phones are just dramatically outclassed by Hololens entirely. It’s like comparing the camera in a phone with an Arri Alexa.