Trying to Make a Movie using Unity

Unity Engine
1

My end goal is to use Unity to make movies (not just cut scenes or quick clips but movies). I do have some experience with Unity making games but this question is exclusive for making movies. Sorry for the length of this post but I want to be thorough.

I have gone through a bunch of tutorials and such for Timeline and Cinemachine… and I have seen people and organizations (including Unity) tout how Unity can be used to make movies… which I have seen the results and they look good… but I can’t get an answer to what I would think is a relatively easy question (for people who are experts with Unity and Cinema). About a year or so ago, I posted a similar question to this post in several different places… from Unity FB group to a Unity Forum post to a well known Unity Discord channel… and either received incomplete answers or the question was avoided. Let me try it again as I aim to get back into this.

For simplicity… this is in layman’s terms. In the “real world” ie. outside of unity… when making a movie… you go on a set… record your shots and move on to the next set. So, say you have one “set” or “shot” you need of someone on a mountain top during the day… you go there and shoot it. Then say you have another shot of a person on a boat… on a river… on an overcast rainy day… you go shoot that… then say you have an action car scene… at night… in a city… you go to the city… at night and shoot it. At some point, you have a bunch of “recordings” or “clips”… you put them in say Adobe Premier Pro… edit them together and now you have a movie. Yes, there is more to it than just this simple explanation (and yeah, you could do it in CG and not need to go anywhere) but this explanation is all that’s needed to give you a context for my overall base question.

I’m trying to sort out how we do this in unity… ie. have different sequences/shots from different locations… coming together to make a movie.

I don’t think you can have one giant “unity scene” with all 3 of the “locations” above located in the same “unity scene.” I guess you could do that but there has to be a better way as having all these difference “scene locations” on one giant unity scene seems like it would have all sorts of problems… starting with global lighting and day/night issues… and a whole bunch more challenges.

In terms of every tutorial video that I have seen trying to teach cinemachine and timeline… shows the shots/sequences being made from the same “unity scene”… ie. the same “map” or environment or “level.” Where there are just different shots/camera movements from the same “unity scene” and the same area/type of environment.

Using my example from the “real world” above… would you have a different “unity scene” for each location… ie. a “Unity scene” for the mountain top, a seperate “unity scene” for the city… a seperate “unity scene for the river sequence”… and then have a separate “Master timeline” “unity scene” where all the shots from those other scenes go? And then load all of those scenes in an iterative manner and somehow piece together the various timelines to the “Master timeline?”

The above may sound confusing or perhaps I’m not explaining it right… so, long story short… I’m trying to make a movie… not a cut scene… but an actual movie using Unity. Movies generally have shots, sequences, etc. being made from a ton of different “looks”, locales, sequences, day/night, etc. How can that be done using Unity in the context of the above question?

I’m not looking for a “how to use Unity” or “Learn c# coding in 7 days” or any of that type of hand-holding… I’m just trying to sort out the top level process using the example(s) above… where you have multiple move scenes, multiple movie sequences, multiple locations and such… all to make one movie… and I guess, ultimately on one timeline? All movies have these types of things (different locales, shots, etc.) so it’s not like it’s something special, I just can’t sort out how it’s done using Unity.

Thanks

PS - would be awesome if Unity or someone did a solid and detailed and narrated tutorial (using assets easily gotten from the Asset store) on making of say… a 10 minute short film… where the movie took place on different locales, locations, day/night, etc. All of the Unity Blog posts and articles about movie making with Unity that I have seen didn’t go into much detail on how they actually made the movie… those articles gave insight into plugins/assets… some of the technology process of say creating fur or skin tone or alembic imports, cloth dynamics and such… but not how they actually made the actual “movie”… at least not from a small indie POV and in the context above. Yes, I have seen some projects you can download but little to no details are shown about the project that helps with my original question and you just see the end results in the project that you download… which I don’t think is all that helpful for people less than at the Intermediate level in such things or wondering about core processes such as the questions I originally stated.

And not sure if the process changes any but I’m looking to do this using HDRP and Unity 2019.3 (or higher/more current).

1 Like
2

Most actual movies presented by unity, like Maria’s and Disney’s, export exr file to an external tools, unity only being a rt replacement of their old pipeline.

Imho Unity don’t actually know Internally much about movies I think, based on observations, they serve the need on a per project basis of external high profile team, which lead to randomized piecemeal implementation of cinematic features. It’s great what they are doing but expect to improvise around simple limitations to get full expressiveness, it’s a generic tool to help you, it’s not a specialized and dedicated software for movie. It’s fine as is though.

The day they add natively the possibility of doing a split diopter shot I’ll be really surprised. I hope it’s after screen space area lut and the addition of a flash line on the vector scope :stuck_out_tongue:

3

I have seen people mentioned the EXR export and then to an External Tool like Final Cut, Adobe Premiere Pro, DaVinci Resolve.

1 Like
4

@
If you’re planning feature length, I won’t recommend having it all run in realtime (oh my god) from 1 unity timeline. I think it’s more practical to approach this from the traditional way of doing cg films. Which is render out your scenes, in comp passes or, if you’re confident enough, in final look renders (including final lighting, foreground + background elements). But export them out in sequences and however much your hardware can bare without crashing or burning into flames (from overheat). I don’t know if there’s a cloud service for realtime rendering. Something to check out.

Best to do this in passes so you have flexibility in post. Plus Unity has the new AOV tool which lets you render in separate comp passes. Edit, color grade, sound mix using external tools.

Obviously, before using Unity, acquire your storyboards, animatics, pre-vis, concept art to help you schedule and efficiently pick and choose art assets (if planning to buy from asset store).

But if you want to be the hero and do it all natively in Unity…Good luck!

5

The plan is to start off with making 1 to 2 min short “films”… learn… get a process down… then start creating 5 to 10 min shorts… (hopefully) learn some more… and get a nice workflow / process going and then progress from there… with perhaps the “end goal” being to make full-length feature films.

Not trying to be a “hero”… just trying to sort out the best process… hence this post and the question(s).

Seems like the last 2 people who commented (which I appreciate) focused on some of the “side things” that I brought up / alluded to. I’m still hoping someone answers the actual core question that I had… crossing fingers for that - LOL.

6

I did this two test projects in Unity this past summer. I did all the assets except the mocap (from Unity assets) and sound fx. Didn’t bother polishing since they were just for tool and process testing. Edited externally. My best takeaway lesson: learn c# scripting :slight_smile: While you can skip scripting if you’re just doing videos I realized I need basic command of scripting to enable my use of deeper engine tools and features (it’s pretty deep I don’t need to list them here).

Good luck in your journey.

7

Thanks for taking the time but still looking for someone to actually address (answer) the specific main question that I have. Animations and C# Scripting and/or editing in Adobe Premier Pro (or another similar tool) isn’t the core question that I had.

Although I do agree that learning c# is helpful… your videos that you did wouldn’t really require scripting… you could do what you did with animation clips, a character controller, an AI asset and using timeline/cinemachine.

Your videos didn’t address my specific question which is (to sum it up)… if you had “scenes” from different locations/looks/environment/levels… ie. not the same “unity scene”… how can that be handled. Read my original post to get more details.

8

My guess, unless some unity insider pop up, is no.

All your environment should be in the same scene if you want seamless cut, BUT you could PROBABLY use layer to render them separately without interference.

Maybe another solution is to stream in and out stuff when needed.

9

And the more and more I look into this… and hopefully someone experienced in using Unity to create movies can confirm is this:

  1. Create a “Unity Scene” for the location/look/environment that you need… do the animations & action needed and use cinemachine/timeline to take the “shot.” Do all of the “shots” that you will need from that location/level/scene. ie. if it’s a Mountain top at night with a lot of volumetric volume and etc… do all of the shots you need from that “Unity scene”/location/environment… even if they aren’t in sequential order with the script. Export all of the clips/sequences out to the EXR format using the Unity Recorder and put them in a non-unity (outside of the specific unity project) folder.

  2. Then move on to the next “location” ie. Unity scene. Set up a new Unity scene… and say it’s for a series of shots in a city… setup the city, the look, the Unity post processing, the lighting, etc. for the look on that scene… and record all of the shots/sequences that you will need using timeline/cinemachine. Export those “clips” out to the EXR format and put those in a folder.

So on and so forth… for each “location/scene/environment/sequence” that you need until you have all of your shots/clips. Kind of how it’s done in “regular” movies… ie. in a non Unity movie.

Then bring all of those EXR export files into say Adobe Premier Pro or DaVinci Resolve… and sequence your shots and polish off/edit and finalize your movie within Adobe Premier Pro (or Davinci Resolve).

That seems to be the most reasonable and logical process from what I’m gathering… can someone who is actually experienced in such matters and movie making process confirm this is the best process of making a movie (not just a cutscene or 30 second trailer from the same Unity scene) using Unity that has different scene locations, looks and environments?

1 Like
10

How would you have all of your environment in the “Same scene”… say one sequence is night shot… another is day shot… another is a rainy shot… how can you logistically do all of that in one giant unity scene? The Global settings just for the skybox or the Global lighting (ie. the “sun”) wouldn’t work for every sequence/location… to name just one thing that would be a challenge by doing it the way you suggested.

How would you have a car race in one sequence of events… and then have another of a mountain shot done at night… all in one giant unity Scene? Could it be done… yeah… probably but is it a good solution… at first glance… I’m thinking no.

And most movies aren’t one seamless long running no cut movies… and that wasn’t a requirement that I brought up… or needed.

Remember, I’m not talking about making a game or a cutscene within a game or a trailer… I’m talking a short film or feature length movie.

11

I understood that. MY guess is on the fact that scene are container, basically it contain the data such as the timeline. Generally, unless you do a blend cut (temporal ellipse for example), all scene sequences are “contained” and coherent, so any adjustment, like lighting, can happen in between “during” the cut. Global setting or not, they can be changed on the fly, and that mean on a frame basis.

Now maybe a solution for you is to have a master scene, then load, unload subscene (like we do in game).

Also movie aren’t like game, true, you have full control over framing, unlike game, and generally you will design based on that. If you planned carefully ahead, you will just have what’s needed for the frame, with details harshly based on distance to the camera on an image basis. Which mean you can destroy/add object on the fly. Your scene can be way lighter than the equivalent game scene, for a denser visual quality.

Thing like day shot, night and weather shot, should be trivial, unless you do complex split screen with temporal mismatch, I’m not sure unity allow for easy split cinematography (hence my diopter comment), they are only mostly parameter change. Scene change are more complex, but it all depend on the actual complexity of the shot you want.

HOWEVER

I have been assuming you want something that allow you easily edit things out of the box.

BECAUSE

All of that is possible but you will have to touch deeply all aspects of unity, such asset management, shader, scene and object management, programming, render pipeline. Basically do your own custom solution.

The short answer is, well you have to learn all of unity now. In fact in theory you wouldn’t even need timeline, it’s just a convenience.

12

Actually…you can “shoot” it all and export frame/video assets from one file…since, if you took the time to watch Unite videos, the Cinemachine demo team occasionally show using multiple timelines in one master timeline via activation tracks.

It’s an unusual way of proceeding if your goal is an ambitious one like a feature length film (1 hour, 45 minutes minimum running time). I’d classify it more as brute force tech demo than a serious film project (in the context of using current versions and capabilities of Unity).

Maybe you’ll still get lots of youtube views for your resulting movie if say you create a massive terrain, using the terrain tools. Something like 4 kilometers by 4 kilometers. Situate your scenes in each “quadrant”. Localize lighting in each. Turn off lights and mesh renders in inactive “sets” in Inspector and turn all lights,post processing and cameras in sets or shots you’re currently working on. Your click bait would be a “1 timeline realtime feature film”.

It’s like a toy. :slight_smile:

13

I watched the Unite video you’re referencing and many other videos as well… and they didn’t exactly show or answer my original question in those videos. I did a lot of research before hoping on a thread like this one and posting the question… because posting on forums like this rarely (or at least not consistently) yields complete answers to potentially complex or technical questions. I’ve been successfully in the technology/entertainment fields for 20 years and usually wince at the thought of posting in a general forum like this. I’m about to buy the Pro level of unity just to get a final answer on it via their priority support access you get for being on the Pro plan.

And I don’t think… with the awards that the Unity platform is starting to rack up and the content coming out of it for movies… that Unity for movie making is a “toy”… and it’s more than capable now of creating “serious” film projects. The industry buzz and interest is high in the cinema world for Unity.

And from the additional info I have gathered elsewhere in the past couple of days… it seems it’s not being done by using a huge unity scene and using “quadrants”… but it’s being done how I described it a couple of posts ago (look above). I’m hoping someone with actual professional experience in the industry and actually has done it for more than just a “clickbait” youtube video… could hop on this thread and confirm the process above that I outlined.

14

[QUOTE="GameWorxStudios, post: 5088440, member: 550604 I’ve been successfully in the technology/entertainment fields for 20 years[/QUOTE]

:slight_smile: You’re not assuming or saying…that you’re the only one with 20 years of being successful in the tech-ent fields…in this thread…are you?

Do we have to post linkedIn’s for comparison and verification to validate replies. :stuck_out_tongue:

15

Didn’t assume or say that I’m the only one that does have that experience… (please copy and paste where I did)… but you assumed that I didn’t watch the unite videos… and you made some other comments which came across as snarky… and assumptive.

I don’t have a LinkedIn profile as I’ve been self employed for decades… and not looking for a job nor have a need to show my “resume” to anyone so LinkedIn isn’t really useful for me. People who need my services and expertise in various arena’s already know about me and seek me out. And lets keep it real… like with most social media… a person can make their profile page look however they need it to look. Most people use LinkedIn like a “virtual business” card… I don’t need one… as I’m not soliciting for business nor need a job.

You seem to be taking this personal now… me saying I have “X” amount of experience doesn’t mean or imply that you don’t… but on this specific topic… you seem to be guessing more than actually knowing the ways and processes of people who actually do what I’m asking about for a living.

PS - still wondering why you posted or some would say “crowbarred” your 3 videos into this thread, which didn’t really apply to what I was asking about… nor have your comments been all that helpful… just saying.

16

[QUOTE="GameWorxStudios, post: 5088488, member: 550604"People who need my services and expertise in various arena’s already know about me and seek me out.[/QUOTE]

:slight_smile:

1 Like
17

You use a “Control Track” in the timeline to cut between alternate timelines & if you wish scenes… You could even spawn in entire scenes / set dressing using Timeline "Signals & Receivers. Worth a little research…

The obvious reason for ditching Maya’s AOV’s & Render Passes is due to time. Imagine how long it would take to render out each shot when now Unity incorporates PBR materials / reflections / as seen in Adam…

I feel Maya will be obsolete once Unity adopts a kick ass Modeling package.

RENDER PASSES … Phew !! that’s so old school :slight_smile:

REAL TIME is where its at…

18

Hi @PixelPounder , I’m chiming in a little late here, hopefully you’re in a good place. Here’s some more info on your great question.

Yes, we work like you’ve described all the time. I’ve been involved in the Disney TVA ‘Baymax Dreams’ project and the internal ‘Sherman’ project - where you can download the project to see how it’s setup and a number of others, which use the ‘sets’ idea you described. We call them ‘beats’.

There’s a few ways to do it depending on what version of Unity you’re on. The most current method is to use Nested Prefabs for each beat or set. Have a master Timeline activate the nested prefab groups for each set/beat. Inside each beat you can have other Timelines - we usually break them up per discipline so one for Animation, one for Lighting, etc., that way multiple people can work on the same beat simultaneously.

You can also use Scenes for each beat and do everything else very much the same way. How you approach your lighting will need some thinking - are you baking lightmaps? Is it all dynamic? Switching skyboxes, baked GI, etc. You’ll need to switch out your scene settings as well in order to control all that.

Here’s a simple example, this video has a single car drive through all these different locations. A master timeline played the animations in a single scene with just the car, while activating/deactivating the different locations, as scenes.

See the scenes on the left in the Hierarchy, the activation tracks below in the timeline.

5167457--512717--upload_2019-11-12_9-55-32.jpg
5167457--512717--upload_2019-11-12_9-55-32.jpg1536×911 268 KB

As with many things in Unity, there’s more than one way to do something. Definitely do NOT put everything in one giant scene, look at prefabs an/or sub-scenes. Checkout that Sherman project download, there’s an example of a setup right there.

For an even simpler setup, checkout the Film Sample Project, it has a suuuper basic setup showing the beats / control track ideas.

Keep us posted

1200×712 104 KB

5 Likes
19

Thanks for the thorough response @Adam_Myhill … it would be awesome if Unity did a tutorial of making a short film… for example… how they did the car video. I kind of get what you’re talking about with the “Nested prefabs” and the other things you mentioned but still a bit confused. Would be a lot easier if Unity put together thorough tutorials for this.

I still have some questions… for example… is the “Master Timeline” in one “Unity Scene”… and the other animations and “environments/locations" that are from other Unity Scenes can be brought into the Master Timeline, which is itself in a separate Unity Scene? The nuances like that is confusing… and I think the word “scene” when people try to explain it is getting confused… as it can mean 2 different things… ie. a “Unity Scene” vrs. a “Scene” in the movie. A “unity scene” in the context that I’m talking about is the scene you build your “level” in a game… or environment if a movie… and is what you save and load, etc. A “scene” in a movie… is a part of the movie… ie. where characters, environment, etc. are and things are occurring… ie. a “movie scene.”

In your image of the Hierarchy above… like other things… I kind of get what you’re saying… but don’t fully grasp it… a 10 minute video tutorial that goes along with it would be awesome.

I have talked with a bunch of people who are interested in using Unity as a filmmaking tool and other folks who want to make great cut-scenes… and they have the same questions that I do. There are a lot of videos and tutorials out there on cinemachine or timeline… and videos on how to make a cut scene from the current “unity scene” level of the game… ie. making an animation or “clip” from the one specific “unity scene” that they are working on… but no current, thorough and high quality tutorial videos out there that shows someone making a movie… using multiple “unity scenes” and/or environments and brings it all together… it would be helpful if there were tutorials like this… even if it was a small 5 minute short movie you all make… and then have a 60 minute tutorial on how they made that 5 minute short… with the short having multiple scenes/environments, etc. like how most most movies are… and explain the techniques you mentioned in your post… in the tutorial. I don’t even need tips on lighting, effects or post processing… as there a ton of videos on that already… I would just need tutorials/insight on core fundamentals of making a movie with Unity in terms of the various things that I brought up in this thread… ie. construction of the timeline, combine scenes together from different locations/environments, best practices on exporting the final “product” so the movie looks great, etc.

I did check out the Sherman project… and “Film Sample Project” and didn’t really find them helpful in answering what I’m asking… it wasn’t that intuitive… and without a supporting video… to explain the various things in the project… doesn’t help someone new to making films with Unity. I do appreciate the effort by Unity with those though. But Sherman for example… it’s been awhile since I looked at it but if I remember correctly… I think the entire scene was just from one “Unity Scene”… ie. one environment… one of my questions would be… say you had a scene on an island… another scene was a night scene in a cave… ie. the “locations”… the environment… the entire scene was in a different “unity scene”… how would you bring in all of those “clips” from different “Unity Scenes”… different “environments”… different “locations” together… into one master timeline?

I see that Unity is trying to expand in the movie industry… but the tutorials are lacking on exactly how to use Unity to do that… and sample projects by themselves are not effective teaching tools for this in my opinion. They are good if you have videos with them… as you can use those sample projects as a reference to go with the video tutorial but as a standalone teaching tool… not so helpful. And when you have someone make these tutorial videos… have a couple of people who aren’t Unity experts… and/or new to making films with Unity watch them and get their feedback and see if they found the tutorial videos helpful.

I find that some of the Unity-made tutorials are hard to follow… (where other Unity videos made by non-Unity employed folks are much easier)… it’s like when you’re reading a “white paper” that tries to explain a technology concept… you can tell that the person who wrote it is more of a tech person than a writer… as often the documents are written by someone who fully understands the concepts they are explaining but written in a way that the “layman” or at times… even an intermediate level person can’t grasp it. They often write it assuming the person reading it knows as much as they do… it’s written at a level “above” the person who is likely to read it. Amazon is an example of a company that does this type of stuff… go read their “white docs” for the various AWS tools and technologies… and it’s often confusing and the technical aspects aren’t written in a way that is easy to follow and/or understand. There is an “art” to making technology documents/tutorials easy to follow… and to be able to write it so someone can easily learn from it. I find many technology companies have a hard time doing it. Thankfully, I have been in the IT space for 20 years… so I can usually grasp it at some point but it still can be a ton of wasted time… and that time could be saved if the writer or video tutorial maker was able to articulate their knowledge in a way that’s easier to follow… and a lot of people (I would imagine) that are coming to Unity… don’t have the overall experience that I do… which could be a “barrier to entry” for some… which I think should be a concern for Unity if you all want to grow and stay ahead of Unreal (and others).

At a minimum… if Unity can put together a video to go with the 2 sample projects… and answer in that video some of the things that I am asking about in this thread… that would go along way.

The things that I bring up aren’t nuances… or “strange” one off scenarios that I’m asking about… they are directly related to the “construction” of the film and how to make a multi-scene movie using Unity as the core film-making tool.

Another question that could probably be easily answered in this thread… without a tutorial video… at least from a “macro” POV… is say you made a nice timeline… it’s all set to be shown to the public… do you just use the Unity recorder and the resulting video file is your final product to show people… or would you export that file to say EXR or something… and “finish” it off and do the final edits it in something like Adobe Premier Pro? Long story short… what’s the best way to get the best looking movie from the creation that I made in timeline and cinemachine and export it out to show an audience?

And in terms of Unity versions that I would be using…lets assume at this time… Unity 2019.2.x or even 2019.3.x HDRP or even Unity 2020… ie. preferably something that is the most current production ready version… and if practical - beta versions of the latest and greatest Unity has… so in other words… something current (or in the relative near-future).

20

Don’t have much to contribute as I’m still beginner level in Unity but I am very interested in learning more about this topic so thank you for these questions! I’m finding Cinemachine to be a great tool and am also disappointend in the lack of tutorial content out there beyond the “set up your timeline” videos. I agree a longer video on how to create a 5-10 minute short would be extremely helpful for someone of my level. But I will try to go through the projects Adam shared more thoroughly as well and see what I can find.

3 Likes