[Feedback] Editing live recordings with Sequences support

[akent99]

I thought I would share some experiences to see if there is an easier workflow to adjust animation clips created from live recordings after they are captured when usingSequences (and if not, here is a feature request! ;^).

I record an animation clip. I have it in the timeline etc. But the Sequences hierarchy means I have preallocated a length of time for the sequence (I leave some buffer, but it is still fixed). That means if I move the playhead too far forward, it jumps to the next sequence.

Challenge 1: I have extra content at the start and end of the animation clip just recorded which I need to trim off. Because the current sequence is only say 5 to 10 seconds long, if the recording has too much messing around (e.g. multiple attempts to get the acting right) I have to trim a few seconds off the front, drag it to the start of the sequence, trim a few more seconds off, drag to the start of the sequence again, repeat until the front is found. Painful!

It would be easier if there was a way to may ripple insert like 20 seconds (then remove it later), or disable everything else following, so the current sequence gets more time until I am finished tweaking. I need to keep it in the hierarchy as parent sequence layers are doing things like activating scenes etc.

Trimming the end is easier - usually - as I can position the playhead and right click to trim based on position of the playhead. (Note that I cannot zoom out to find the end of the animation clip (it did not work) and trim it with the mouse. But I found that I can trim it using the playhead and right click menu items to “trim to playhead”. So I solved that one.)

Challenge 2: Once I have the animation clip trimmed in an Animation track in the timeline, I find that I want to adjust the animation clip a bit. I have to line up different characters, movements, etc. So I double click on the animation clip in Timeline window and it opens the Animation window with that clip. I can edit it, but I cannot click to move the playhead - I have to return to the Timeline panel every time I want to preview a change. (It is like the timeline has taken control over the playhead because I opened the animation clip from Timeline.) So every little edit requires flipping backwards and forwards. I also have to read the frame numbers off the Timeline window to work out where it is in the animation window. (I normally have timeline and animation as tabs in the same position - I am going to try a different layout to see if it helps. But they are both very wide so I prefer them going across the screen.)

Can I move the playhead while in the Animation window somehow? That would save a lot of time.

Here is an example timeline. I double click on the animation clip (highlighted below - ep1-20-030-tom)

When the animation clip is opened, I cannot move the playhead

Challenge 3: Rendering. I often want to render one sequence in the middle of the overall hierarchy. At present I do this by playing the whole sequence hierarchy again from the start. Sometimes I mess around and reorder the sequences to get the one I want to the start (out of order) so I can “stop” it playing - but I have to make sure it is not half way through rendering some other sequence or I might mess up the other recording. Is there a way to tell “Play” mode with “activate on start” to only do one nested sequence? That is, skip forward to the current sequence start, play (doing recorder clips etc), then stop playing? I need it in the hierarchy for parent sequence set activation etc (I tried just setting “play on start” for the nested sequence, but it did not seem to get all the context right). Why? Because preview mode does not always come out the same as the final recording in play mode. So I need to check the final result.

I could imagine in the “Sequences” hierarchy window right clicking on a sequence and saying “Play this one only and stop”.

And sharing for fun:

[akent99]

Challenge 4: (I do not think relevant, but FYI I am using EVMC2U to capture VMC protocol from VSeeFace (not the Unity face capture etc). It does upper body and face tracking.) The important aspect is it will only capture in “Play” mode and selection changes when you enter and exit play mode (this is the issue I wanted to report for consideration, not the capture itself).

I start of with a shot sequence selected (I use episode / location / shot sequence nesting with Sequences package).

I then hit Play to I can start doing live capture to create an animation clip. Notice the Shot sequence deselects (I have the lock icon on the timeline window). This also means the scene activation (in the Location 20 sequence) goes away, so the scene that the characters are in disappears.

So I click on “Shot 50” again to try and regain focus on the current shot (to get the Scene Activation to fire etc). It is highlighted, but scene activation etc still does not kick in. It feels like it thinks it already has shot 50 open, so it does not set it again.

I click another shot (40). It loads up shot 40 correctly.

I click shot 50. It loads up shot 50. Now I can position the playhead into the timeline to activate all the characters etc for the shot sequence (including scene activation). That allows me to start recording.

Can it be made so when you enter play mode it keeps the current sequence active? It saves all the clicking around to reactivate it.

Note: What I think I need to investigate is whether I can adjust EVMC4U to run without entering Play mode. I have to disable the “Play on Awake” setting each time to do a recording, then turn it back on each time I want to render.

[akent99]

Another similar issue with focus that is hopefully easy to fix but time wasting.

I have a Shot (3rd level in Sequence hierarchy) selected, you have to click to position the play head appropriately in the Timeline window (I do lots of moving forwards and backwards checking a shot, or just selecting it again as you have to set the playhead for focus to be correct). E.g. successful click:

The problem is if you click a few pixels too high (the name just above the time markers) it selects the top level sequence (Episode 1 Timeline in my case).

The Timeline window is now showing the top level sequence in the hierarchy, but the Hierarchy window still has focus on the nested Shot 20 sequence (it did not realize it changed).

If you click on Shot 20 in the hierarchy, the Timeline window stays on the top level sequence (because it thinks its already where it should be).

You have to click on say Shot 30 then back on Shot 20 to get the Timeline window to refresh and get everything back in sync.

Its easy to work around, just annoying as I find I often click a few pixels too high when scrubbing the playhead, then I cannot quickly get focus back without going to a different shot and back again.

Proposed fix: If you click on the Episode 1 bar just above the time codes, at least change the focus in the scene hierarchy so its quicker to change back to where you should be, rather than forcing to jump to another shot and back again.

[lalefi]

Hi, thanks for the solution I had the same problem.

[ellka]

Heyo!

Alooooors, let me try to clarify a bit your challenges. I read your posts twice, I think that some part are not in my competences but I can try, and at least I may be more able to find someone who will have the answers.

Challenge 1: what I understand for this one is that editing clips in Timeline may be tedious in your case, but I’m not sure to actually understand your use case.

You seem to already be aware of the right click on clip > editing > Trim start or Trim end based on the playhead position.

You’re also speaking about “ripple”, there is a ripple mode in Timeline (see screenshot) but I’m not sure this is what you have in mind what you speak about “ripple insert like 20 seconds”.

The three editing mode are: blend, ripple (the one in red), replace

Would you be able to maybe provide a video of what you want to do and what you need to do to achieve it? And maybe detail precisely what you “would like” to be able to do (workflow wise)?

Challenge 2: This one I think I don’t have enough knowledge to repro what you are describing. On my side I can move the playhead in the Animation window without problem. So probably that my repro is to simple to cover what you’re describing. Could you provide step by step instruction? I can also try to find someone who will be more knowledgeable than me on that

Challenge 3: When you say “Rendering” you mean rendering with Recorder right? It is a known limitation that Recorder always play the Timeline from 0 even if the start frame for the recording is 1001 or whatever :/. If I remember correctly it’s for determinism reason (as Timeline can drive things like simulation). I’ll log your feedback!

Challenge 4: Ok, am I understanding this one correctly: basically you’d like that when going into Playmode, you keep the focus on the current sequence already focused, right?

I’m sorry to not help more than that. I hope we’ll be able to clarify things for you (or “at worst” log your feature requests :)).

Thanks!

[ellka]

Oups, I forgot to answer at you’re last feedback. I actually have not much to add to it right now, it’s clear enough for now, I’ll log it on our side though :).

[akent99]

Challenge 4: There are two issues with focus. Yes, things lose focus too often. Annoying but no idea how easy to solve. Worse however is you cannot just click back on it to regain focus. You have to click on something different then click back to the one you want to get focus on. That feels like a bug. It seems to think “the sequence the user clicked on already has focus, so I don’t need to do anything” - but the sequence had lost focus. Clicking on another sequence then clicking back again gets it working again. (The extra post was another circumstances when this occurs too.) So annoying, but a workaround exists.

Challenge 3: Ah, determinism. I can see that. In my case I don’t actually need determinism (I want scenes independent). Its annoying, but not a show stopper. (I can just re-render the whole sequence.)

Challenge 1: An exercise for you to repeat it:

• Create a sequence with two nested sequences
• Make both nested sequence 5 seconds long each
• In the parent sequence, leave a 5 second gap between the two sequences as buffer space (this is what I do in practice so its easy to extend the first sequence a bit without having to move the second sequence)
• Add a 5 minute animation clip and drop it into the first nested sequence so it extends way off the right side of the timeline window [ Note: you can drop any animation clip in, then set the Speed Multiplier to 0.01 and the clip will be 100 times longer in the timeline ]
• Trim the first 4 minutes off the animation clip (and tell me what magic you did to do it)

I cannot zoom the timeline window out beyond 10 seconds in the nested sequence - that is the cause of the problem I think. The horizontal scroll bar will not let me zoom out to see the whole animation clip. So I can only see the first 10 seconds of the animation clip in the window. I have to trim off 10 seconds at a time (the Timeline window width). So to trim 1 minute it takes 6 trims and sliding of the animation clip to the start of the track, 4 minutes is 24 trim-and-slide of the animation track. Painful!

I am finding this to happen somewhat often with live recordings. Normal animation clips are generally clean and short. Live recordings it is more efficient to record a few attempts in one recording session, play them back, and pick the best one - so the clips are much longer with more cruft.

There are different possible solutions - let me drag the animation clip off the left edge (it wont move past the start at present - I have to trim it); let me zoom the timeline window out much further (this might help, but is problematic as the sequence is 10 seconds long - if you move the playhead into the time allocated to the next sequence things lose focus etc and you cannot see what is going on.

It is the combination of nested sequences with a following sequence, timeline, etc all working together where it gets painful. Sequences I think puts limits on the Timeline window to stop zooming out further for example (if I understand correctly).

Regarding ripple, it is not ripple only in the current timeline - it is ripple that affects all the parent sequences as well. The parent sequence has all the child sequences packed in together (I add some buffer). To make the timeline bigger for a nested sequence, I have to go the parent sequence and move everything after it, which requires me to go to the sequence above that and shuffle things along too. It is a ripple insert/delete that affects all parent sequences up the hierarchy that would be useful.

Challenge 2: If I go to the Timeline window and double click on a recording dropped in for a character it also opens the Animation window on that clip. However it is locked - I cannot move the playhead. I can only move the playhead by returning to the Timeline window. (Just checked it again.)

If I double click on an animation clip asset from the Project window, it opens the Animation window and I can move the playhead around. Moving the playhead does not update the Game window, but I can see the character in isolation in the Inspector window.

I can right click in timeline, find the asset, open the asset and edit it in isolation…

Well, that is weird…

I was just repeating the steps to be sure of the details. When I started typing this response it would not move the playhead in the Animation window (as before) when I double clicked a live recording from the timeline. (I verified this was still happening to be sure.) Then I opened an animation clip directly from the Project window and it would move the playhead (as expected). After returning to the Timeline and double clicking again, suddenly now it DOES move the playhead??? Restarting Unity everything is working fine too! Opening other projects it is working there too! So confused… but Challenge 2 might have be a corruption or strange setting - it is working now!

BUT! it is still painful. When I move the animation clip playhead towards the end of the animation clip, because the playhead is outside the time range for the current sequence, the current scene assets go inactive and I cannot see the animation clip any more.

Looks like I need to do a first rough cut pass editing the animation clip directly through the Project window, watch the character in isolation, shorten the clip, then drag the shortened clip into the sequence for finer grain editing in context with the location scene, props, lighting, and other characters etc… I can get the project done, but the workflow is quite painful in places.