Another thing I am not understanding the logic of is timelines for sequence assets and how to use them.
Sequences have timelines - makes logical sense. But what surprised me is Sequence Assets also have timelines. E.g. each character variant gets a timeline asset. If you add the same character variant to different sequences, the timeline is shared.
In my mind I was thinking sequences were the timelines, and adding characters (sequence assets in general) to a sequence would add a track to the sequence timeline. I had assumed I can have a character (e.g. Sam) and a variant of that character (e.g. Sam carrying bag). The variant allowed me to make adjustments to the settings for Sam.
But it appears that every time I want to use a sequence asset, I must create a variant to hold the animation of that asset. That implies every time I add a sequence asset to a scene, I should create a new variant of all the assets. Not doing so means I will be sharing the timeline. That means in practice I need to do bookkeeping - e.g. allocate a unique name to each sequence and create variants of each sequence asset for each sequence that uses it (and hopefully never have to allocate anything). So I don’t have “Sam” and “Sam with Bag” as variants - instead I have “Sam shot-23” I must remember to add a variant for every sequence the character appears in myself, otherwise I am going to mess things up.
Requiring a new variant per sequence asset works, but I cannot imagine a case where I want to reuse a timeline in different sequences when creating a cinematic. Each sequence is almost certainly going to have different acting and animation just for that sequence, or (e.g. for lights and sets) no animation.
Just checking I am understanding correctly (and providing feedback - it feels like added complexity for no benefit). Is there a reason sequences don’t animate character assets directly in their timelines?
What would I like? Remove timelines from sequence assets and add the animation of all sequence assets into Sequence timelines. Note that I do want to be able to add override tracks etc for sequence assets. E.g. I drop a “walk cycle” animation clip onto a character with an avatar mask so it controls the legs only, then I use a live recording to control the top part of the character using VSeeFace, and sometimes other override track for the facial expression control, left hand gestures, and right hand gestures.
I like the variants on characters. It helps me organize my projects. I can group all the variants of say Sam wearing different clothes etc. (Maybe that is just me, but for films it is logical. It is boring if the character never changes clothes, puts a hat on, etc.) So that part is good. It is just that I want to reuse a character variant in different scenes without reusing the same timeline.
In case of interest, here is an example timeline I created as an experiment (before this Sequences beta was available). It had a garage door open (which I ended up turning off), a character riding an electric scooter (so not as much body movement). I created animation clips for common head turns then drop and blend those clips into a timeline. (This one does not use a live recording - just static poses.) I line things up across tracks., with virtual camera swaps etc.
The above is the second page in Extra Ordinary Episode 1 - an experiment to see how it all fits together. I don’t need character prefab variants for different body poses as I have animation clip support already (or will record custom clips using VSeeFace etc). I am fine dropping those animation clips directly into the timeline - I don’t need a prefab per animation clip. I am not even sure how all the blending would work .
That is very interesting!
If I understand correctly your points:
You like being able to do variants on Character assets, but you don’t need the dedicated Timeline for them and you which to animate your Character directly in the Sequence timeline (and from there, being able to have different animation in different sequences for the same character asset variant).
You which that when adding a Character asset to a Sequence, it automatically creates an animation track for you in it.
Am I correct with this summary?
From there, I have some questions:
Does animation variant are an option as well? For example, “Sam walking normal” and “Sam walking weird”. Or you only have needs for character (“geometry” asset) variant?
Does your expectation scales to the other type of Sequence Asset, or is it a feedback specific for Character sequence asset?
About adding automatically an animation track, how would you see that precisely? Should it also automatically bind to something, and if yes what? If there is nothing to animate, should it creates a track anyway or not? What’s your dream workflow there basically?
And now some guidance (hopefully):
You actually don’t have to use the timeline and track created for you. You could do what you said (and shown in your second post). I.e. create a new Character Asset, add it to a sequence, remove the track that was automatically created (as you don’t need it in your case), create an animation track instead, animate your character variant as you which… repeat in other sequences!
It might sound tedious to do but it should at least allow you to do further investigation along this workflow you’re describing. Of course, if you have some other details or example to share along this line, please do!
Re summary: Your summary is accurate enough. I don’t mind creating an animation track myself in practice as for characters I frequently expect to have a few levels of override. Having the timeline assets created for me and linked up is the time saver (not creating tracks in the timeline). I frequently use multiple override tracks with avatar masks. This might not be typical for others, but I use facial capture for the facial movements and often want to combine with say a looped walk animation. Or combine upper body mocap tracking with a fixed sitting pose for the lower part of the body. So multiple layers of animation clips for me is common for characters. I cannot see doing this with a shared timeline object. (But I was asking about Unity’s thoughts here to see if I was missing something!)
Yes, I was thinking of deleting the reference to the character timeline like you suggested. I was just not sure if I was missing something in the approach being advocated. Going off track often causes problems later which I would like to avoid.
Trying to explain another way (possibly unnecessarily). The fundamental idea in terms of hierarchy is I want episodes / beats / sequences as a hierarchy (as done now). These are nicely scoped at present (put into sub-directories). Then I have sequence assets I share across sequences (characters, effects, lighting, sets, etc). Makes sense and nice. The funny part is I need to create a flat list of variants (at least one variant per sequence the asset is animated in) under “global” sequence assets. The nice sequence scoping just disappeared.
To add to the above I would rather record and store facial mocap et al animation clips in the sequence scope as it is only relevant to that sequence, but I also have animation clips per character for a character (e.g. a personalized walk sequence), as well as animation clips that are universal (e.g. sitting on a chair). I need to combine these different scopes of animation clips at the override track level for a single character.
I also worry about larger projects. Creating a character variant per sequence that animates the character feels like it could get long. (Or said the other way, each time you want a unique animation for a character, start by creating a new variant.) For myself, I plan (ha!) to create a series of short episodes. So it means over time I am going to have a lot of custom animations - using things like the iPhone ARKit support for mocap. I am going to make recordings, play them back, realize a short part of a longer segment did not come out well, re-record, patch it together, oh the eyes rolled a 200ms too early, splice and adjust, etc. I am worried about the global Structure panel for a character ending up with hundreds of variants making it a pain to navigate (but have not experienced it yet, so take this feedback with a grain of salt!) - I will need a good naming convention to keep it all in order.
HOWEVER… I think a critical unanswered question to me is scenes. How to flip between sets and make sure things like baked lighting is correct? (Or anything else that is scoped per scene.) I don’t understand how to use the current tool across multiple Unity scenes. I have a suspicion that each sequence that changes the Set should get its own scene so baked lighting can be done for that set (or else document never to use baked lighting with these sorts of projects??). A cut scene in a game would be different to creating a movie - the game is adding an animation to an existing scene (which will have already been baked etc), whereas creating a movie will have different lighting and colors (e.g. more yellow at sunrise). Baking is slow - I don’t want auto-baking to kick off every time I change to a new sequence to work on. I would suggest getting this problem resolved first as it may introduce another level of nesting in the scoping hierarchy (e.g. are variants per scene?).
So if I was to suggest a priority, I would get the story around scenes sorted out first, then come back to the scoping of animation management with sequence assets. Just my 2c!
Ah, made some good progress. (The pennies are dropping.) I had previously tried creating master sequences in empty scenes, but did not seem to work. However I tried again and it did work - so probably user error last time. That means I can create a new scene per location in an episode, create a master sequence in that, and use it. E.g. create “Episode1-Location1.scene” and then load Master Sequence “Episode1-Location1” into it (just give them the same name). Then I load the set on the master sequence.
However I also managed to get the scene activation approach working as well. Instead of adding a “set” to a sequence, load all that (probably lighting too) into a Unity scene, then use activation to load up that scene as the “set”. I can see how that would allow all the baking for lighting etc. The scene only holds the set, no sequences or animation - no characters or anything like that. But it means baking is only done once. That makes a lot more sense.
Then I also deleted the sequence asset track and added an animation track for my characters like I normally do, adding animation clips with override tracks etc. Not too bad. For myself that I think will work better due to the complexity of override tracks I think I will be doing (plus it avoids a variant per recording).
So good progress on several fronts - it is starting to fall into place.
Does this last message means that your worries on scenes in your previous post are “solved” (at least for now)?
Also, it’s cool that you’re trying this approach with Sequence Asset. It’s not a workflow that we heavily tested in the project we seen so far, so we’d be happy to hear any issues you may have (or success!).
I believe that all those conversation at least point that we are still missing documentations (this will definitely improve over time) and maybe some workflows could be clearer and more intuitive (or straightforward). If you have dream ideas concerning that, “I would love being able to…”, let us know!
Yes, I have a path forward to try a real project now - going to try a prototype episode based on what I have to see what happens. This might take a little while as I have to reorganize my project a bit, as well as sorting through asset management of mocap animation clips.
Yes, documentation would help. It depends on your audience. As a novice/intermediate Unity user it took a bit to understand the intent of the concepts. Note: the tutorial could be a video rather than written - show how to use the tools to create a real project, showing the flow between them. For example, explain why scene activation can be important. But with Unity, I am guessing lots of people will start blogging and creating videos of their own to fill those gaps.
The other missing docs for me is adding these tools to an existing project (but that may have been intentional since this is a beta). E.g. there is that workspace helper that I don’t have in my project and not sure how to get it. Also not sure what other packages I should add from the demo project to my own project. I only grabbed the sequences package.
Dropping some more feedback as I try and do a real (small) project.
I was thinking to create multiple master cameras and then pick the right one for a scene (e.g. with different focal length, different bloom settings, etc.). I am moving away from this position towards having one camera prefab with a script to control all the settings, making them easier to animate. The problem is when I flip to a different camera prefab, all the Cinemachine virtual cameras, which point at the master camera, lose their connections. Rewiring it is a pain. So instead of swapping between main cameras, I may instead swap between a main camera controller (or just animate it with animation clips) so the main camera never changes.
I am starting to wonder if I should do the same with characters to avoid “look at target” link problems. Swapping the prefab is a major change and breaks all the look at targets if they pointed at the character directly. Instead, I might beef up the character itself to add scripts etc to change the clothes as required. Then I just animate properties on a single character if I decide to change the clothes later. (Changing clothes at the moment has a major impact on the scene using my current plan.)
Cinemachine virtual cameras I am not doing with prefabs either. The main camera prefab I can share across scenes, but the virtual cameras tend to be specific to a scene. So there does not feel much benefit. (Note: I am not planning to use the timeline sequence per prefab variant as I want to line up animations across all the tracks on the one screen, rather than open up multiple timelines to try and line things up. So I am putting everything in the sequence level timeline.)
What does this mean? The sequence assembly seems useful to create a scene quickly, but once created changing anything in a scene breaks “look at” style target linkages badly (this is true for animation tracks of characters, virtual cameras controlling the main camera, different main camera settings, and so on).
An observation / discovery. I am using Magicacloth (3rd party Unity package) as well as a few other packages such as EVMC4U. These do not work in the “nested” scenes that get loaded. They have to be placed into the “root” scene that holds the sequences. This is probably because the character prefabs are in that scene and it won’t reference objects across scenes. That makes sense, just noting experiences using nested scenes.
I just wanted to acknowledge your feedback here and confirm one or two things you mentioned.
So yes, cross-reference is not supported in Unity (and it’s not more supported by Sequences), this include references across scenes, across prefabs and across scene+prefab.
We already knew it would cause problem with Cinemachine cameras… and apparently you also have other scripts that are in conflict with that. That’s good to know.
If I understand correctly though, you found a workaround to still achieve what you need, right?
Also, I realized later (and I think it is one of the thing you’re describing) that if you don’t use the Sequence Asset tracks in Timeline, then switching variants from the Sequence Assembly window is no that nice anymore as you’ll lose the Timeline animation tracks binding (if any). Is that correct?
This is not a use case we had in mind until now. Basically, we covered Sequence Asset animated or not, and if animated then it is done in the Sequence Asset timeline itself (and so through the Sequence Asset track). But I understand that you don’t need (and want) this extra nesting level in Timeline as it makes it hard for you to align the animations of the different cinematic components you have (like camera and characters).
I’ll log all this feedback on our side! Of course let me know if I miss-interpreted something or if you have more questions.
Also, someone in the team found some blog posts you made about Sequences and your project! That’s really cool! Thanks again for all the time and energy you’re giving to this, that’s really helpful for us!
Yes. Workaround = don’t change the variant! ;-). And if there are little changes, I plan to do them with animation clips more often than prefab variants, again so changing the variant is needed less often. I don’t think its a problem, just something to be aware of.
Err, I think so… but I think I have convinced myself “don’t switch variants if you can help it” so I just plan to not do it. Sequence Assembly is useful as a quick way to assemble a new shot - it provides all the main things you want to drop in without having to hunt around assets folders to find them. I do find this helpful as an accelerator.
Alignment across tracks is one reason. The other reason I don’t want it is I expect to end up with hundreds to thousands (see maths below) of prefab variants if I have to create one per animated section I want to record and that scares me having them in one flat list associated with a character. I like nested of timelines in Sequences for scoping. But I then have to create a prefab variant of a character per shooting sequence they are involved in. For the main character, that is a lot to show in a single drop down list.
Here is my latest plan in case interesting to give a bit more context (and standard disclaimer it is a plan!! … that may never happen)
I have 13 arcs of a story line mapped out
I am aiming for 5 episodes per arc - following Aristotle’s Heroic Journey structure from “The Poetics” (350BCE!!!)
Each episode will be around 50 shots of a few seconds each (it is a comic with lightly animated frames) - 2 to 3 minutes each total
I am using the “Web Stories” format (https://stories.google/), so each “frame/panel/shot/line of dialogue” will be a separate page in the web story - that is, a video clip per page (I don’t join them into a full animated movie - you watch a clip with superimposed speech bubble then tap to proceed to the next page)
I have around 20 characters so far (only half created so far, but I don’t need extra characters until later episodes)
I plan to create a “Set Scene” for sets (not use Set prefab variants) - then I can use baked lighting when needed
I plan to create a “Episode Scene” for each episode where the timeline and characters etc for that episode will go into
I plan to create a “Episode Sequence” (top level) corresponding to each Episode Scene
I plan to create a “Location Sequence” (“Set Sequence”?) per location used in the episode - the Scene Activation track will go here for the Set Scene for that Location
I plan to create a “Position Sequence” (need a better name, but “Shot Sequence” is not quite correct) under Location Sequences
I plan to add all the characters and props under Position Sequences, add all the animation clips at this level too
I will add a recorder track at this level, with a recorder clip per “shot” - I will often take multiple shots in the one Position Sequence (e.g. two people talking with the camera jumping between them as they say a line of dialogue per shot)
Frames in the comic will normally be created from one shot, but sometimes two shots will be joined (e.g. for a longer introduction shot)
So, if I created a prefab variant per animated sequence where my main character is in 25% of the shots, that means 13 x 5 x 50 x 25% = 812.5 prefab variants in a flat list to hold all the animation sequences. SCARY! That, in addition to wanting to line things across animation tracks, is why I don’t want to create a character variant per animated sequence they are in.
The reason I think I need the Position Sequence layer is because I want to use root motion in animation clips. I cannot find a way to say “move character to this position” easily ready for a shot without disabling root motion of animation clips. The best way seems to use “Scene Offsets” for the track, then manually position the character correctly in the scene, then play the animation sequence from there. As soon as I need to move something, start a new Position Sequence. I tried animating the position of the character using an animation track, moving the character to the starting point, but as soon as I did that it locked the position and root motion would not move the character any more. So each time the next shot needs characters to be moved, I start a new Position Sequence and make a new copy of all the characters and props (and cameras and lights… sigh).
Question: Is there a better way than the above? I tried trimming the animation clip where I recorded the position of the character, but the track just held the last value no matter what. I do not want to manually animate the position during animation clips (usually). I want to walk and have the character move naturally as defined by the recorded animation, not have to work out the destination point, get the timing/speed right, lining it all up by hand. (Well, I may do some of both to be honest).
Current feature wish list items while I am typing:
I would love a mode where characters etc added to a sequence via the Sequence Assembly window added Animation Tracks for the children of the Sequence Asset prefabs. I don’t want to animate the sequence asset, I want to animate the character inside the sequence asset because that is what my existing animation clips work on - e.g. a humanoid character I have created in a 3rd party tool. The enclosing Sequence Asset is just a container, not the thing I want to animate. I could image, for example, having a flag on a Sequence Asset (or a different type of sequence asset?) to say “when you drop me in a scene, create an animation track for my contents instead of adding my timeline”. That would save me a LOT of time.
I would love to be able to record in an animation track (or a new “Reposition” track type?) the position of a character so I can record their position by dragging them in the scene, but not block the root motion of animation clips that move the character. I want to move/rotate them multiple times in the same animation track to a new position ready for the next shot (breaking the law of physics as required to make the camera shots look good!) - I would line the movement/rotation up with a change of camera so you don’t notice. I want to add to root motion rather than take over from it.
Storyboard clips - add a text box so I can put dialogue very quickly in place without having to create an image per page (I love the drawings too, but just throwing the dialogue up first before creating images allows me to imagine and check the overall timing) - I hacked up some code for this already (added a “Text” field to the existing storyboard clips) but a standard solution would be nice!
Ability to open the scene the top level sequence was put in (it gets added to a scene - it would be handy to be able to reopen the correct scene easily)
I am not going to admit how long this took to create just this few pages, because a lot of learning still going on. But making progress! Episode 1, Friendship (works on desktop, but designed for phone). Tap right edge to go to next page, left edge to go back.
One time waster was realizing that I had to start a new sequence because I needed to reposition things - so had to redo all the clips a second time. Need to plan for that more carefully.
Hey akent! Feedback noted!! Really cool to have your plan, it gives an idea that is precious!
And I’m happy to read that you use the Storyboard track as well! I took note of the feedback on it!
For the animation part, I’m no expert… it sounds like you do it right but I don’t have enough knowledge to say if it is THE right way to do it or not and if there are other simpler ways. I can also check if someone around me could help a bit with that (at least maybe direct you to the right resources).
My suggestion to solve your root motion issue is the following:
Create a prefab of each of your character with an empty game object as the parent/root for them
This will allow you to move around / reposition your characters animation at will without any need for another layer of animation.
These prefabs will also be the place where you can adjust or fix any settings on your characters
I also advise you create an original Sequence asset per shot instead of a variant. Drop in it all of your character prefabs. This will allow you to have all character animation at the same level and place in timeline, minimizing back and forward
Think of variant as a possibility for trying something new or different for your shot and not another way of managing your prefabs. You should only use them for this purpose.
We didn’t do that for our previous project and we realized late that it was a mistake, painting us in a corner.
Thanks for the container game object suggestion - I have done that one before with some success. But the second idea is particularly intriguing and I had a follow up question. (If I do create a new sequence per shot the positioning is less of a problem - I have to reposition per shot anyway!)
I was using Sequence Assembly to quickly drop things in each shot sequence. Then I deleted what was added to the timeline (it added a reference to the sequence asset’s nested timeline) and dropped my prefab in directly (creating an animation track per prefab).
What you are saying I think is create a new Sequence Asset per shot, add all the prefabs I want into that, then they all appear in the sequence asset’s timeline automatically? I assume then you don’t use Sequence Assembly to add prefabs to the Sequence Asset? I am worried about creating a large number of Sequence Assets that appear in a flat list in Asset Collection or similar (I am going to have, hopefully, over a thousand shots!). Or do you create Sequence Assets directly in a Sequence (so they are nested in the sequence directory) somehow? Just curious about your workflow, how to use the provided tools to most efficiently do what you are saying.
(I was suggesting sequence assets have a flag to say “when you drop me into a sequence timeline via Sequence Assembly, drop my contents directly in with animation tracks to the current sequence timeline (instead of a reference to my nested sequence asset timeline)”. So instead of using the sequence asset timeline, I just use the current sequence timeline - with everything in the one timeline like you describe.)
Yes, my suggestion is to create a new Sequence Character Asset per shot.
Then drop all the character prefabs you need in the Sequence Asset prefab of your shot via the hierarchy view.
You will have to drop all root motion of your prefabs in the timeline of your Sequence Character Asset, this will create an animation track for each of them (add an animator Component on your prefab first, this will skip the pop up telling you it will create one)
For the huge list of Sequence Asset you will end up with, a good naming convention will help a lot
Something like Sq010_Sh0010, leaving padding give you room in case you need to insert a shot in between two.
I will also advise you to create a project per episode, Isolating them will help with clutter and is an added security.
I can see the logic in this (I have all the locations and assets in one project at present and it is many gigabytes in size - which definitely makes Unity more sluggish), but I will also have to learn how to share customized scripts and prefabs etc between projects better. I would need automated project builder tools as well as improved scene building tools!
I will probably try and do a few episodes first in the one project. Nothing like pushing something through end-to-end to discover hidden gotchas. Then review and see what changes will help.
In case of interest, I have done a first rough cut of part 1 of 3 for an episode of a lightly animated comic. Uses the new Sequences support. VRoid Studio characters animated in Unity. Click on the right edge to go to the next page. Episode 1, Friendship
Nice work! I can honestly say that this format was not a use case we had in mind when we created Sequences, and that’s awesome! Thanks for all of your feedback!
I just got bitten by lack of isolation in terms of performance. I went back to an old (complex) location after the project grew organically. Unity now takes 3 seconds to respond to a mouse down event . (My laptop is older, not a high powered machine.) So my next exciting phase is going to be creating a “common” project for all the characters and props etc that are shared across locations (reading docs I think I can then use “Assets->Export Package” to create my own “common” package), then creating a project per location which imports the common package. Just sharing to help others avoid the same pain.
