Feedbacks for experimental feature: GameObjectRecorder (2017.1)

Hi!

In the 2017.1 comes a new editor feature: GameObjectRecorder. It allows to record any properties on a GameObject and its children.

Please keep in mind that this feature is experimental. We know there is lots of room for improvement, but we would first like to get feedback from you, our user, in order to identify use cases that are meaningful/useful to you. This feedback will really help us decide how to evolve this feature, moving forward.

In the example below, there is one root GameObject and all the other GameObject are children of this “Scene Root”. On this Scene Root there is a script using a GameObjectRecorder. This GameObjectRecorder is set up to record the transform changes on the Scene Root and all its children. Once the user stops the recording, it saves everything that’s been recorded into an animation clip.

Click on this link to see the result:

Here is the code used to produce this example:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEditor;
using UnityEditor.Experimental.Animations;

public class HierarchyRecorder : MonoBehaviour
{
    // The clip the recording is going to be saved to.
    public AnimationClip clip;

    // Checkbox to start/stop the recording.
    public bool record = false;

    // The main feature: the actual recorder.
    private GameObjectRecorder m_Recorder;

    void Start()
    {
        // Create the GameObjectRecorder.
        m_Recorder = new GameObjectRecorder();
        m_Recorder.root = gameObject;

        // Set it up to record the transforms recursively.
        m_Recorder.BindComponent<Transform>(gameObject, true);
    }

    // The recording needs to be done in LateUpdate in order
    // to be done once everything has been updated
    // (animations, physics, scripts, etc.).
    void LateUpdate()
    {
        if (clip == null)
            return;
     
        if (record)
        {
            // As long as "record" is on: take a snapshot.
            m_Recorder.TakeSnapshot(Time.deltaTime);
        }
        else if (m_Recorder.isRecording)
        {
            // "record" is off, but we were recording:
            // save to clip and clear recording.
            m_Recorder.SaveToClip(clip);
            m_Recorder.ResetRecording();
        }
    }
}

Once saved in a clip, to play it back you can duplicate your scene object, remove any physics components there might be on the duplicated objects (to prevent conflicts with the animation), and drag and drop the clip on the scene object.

Features:

  • /!\ Only work in Editor scripts for now
  • Can record all the properties of a GameObject
  • Can record the properties of one component of a GameObject
  • Can record one specific property of a GameObject
  • Can record recursively on all the GameObject’s children
  • Save the recording in an animation clip (.anim)
  • Compute the appropriate tangents
  • Reduce the keys

Here is the link to the API: Unity - Scripting API: GameObjectRecorder

3 Likes

Sounds interesting, this is for recording cut scenes or similar videos?

For instance, yes.
You can record stuff for a video or a future in-game cinematics.

Or you can record a sequence of animations to show in the tutorials of your games.

Or maybe something else that we didn’t think of yet :wink:

I can see this being really useful for recording complex physics simulations and then playing them back. Say having a house collapse - you can set it up on a powerful computer and add a ton of forces and other complex stuff to make the thing fall apart in a natural looking way, record the details, and then play them back cheaply, so you can get as good looking stuff on cheap hardware.

Like how DK Country got away with looking way too good for the snes.

For things like that, it would be nice if there was an easy way to specify a scaling of the recording. Like:

Time.timeScale = .1f;

//record something that only runs smoothly at 60fps if we're recording at .1 times the normal speed

m_Recorder.SaveToClip(clip, .1f); //this would generate the clip at 10 times the recorded clip

This can be solved by just playing the clip at 10x speed, or by manually multiplying all of the keyframe’s time by .1, but having this directly in the API would probably make that smoother (and more obvious to users!)

2 Likes

Interesting idea @Baste !

Out of curiosity, I guess you’d like to increase the play speed of a physics because collapsing a huge scene, although physically true, tends to be slower than what we see in the movies?

No, I was thinking purely from a performance standpoint.

Like, I’m doing something that’s going to make the game run at 5 fps, so I want to record at 1/12th of the normal speed and then speed that recording up by 12 times so it looks like everything’s happening smoothly.

I’m not sure if fiddling with the timescale fixes that. I know that some AAA companies (not to be named) record teaser trailers by stepping through frame-by-frame and then playing those frames back at 60fps to make it look like their game runs smoothly.

That’s the obvious use of a recorder for me. I mean, record something that’s way too expensive and then play it back cheaply, not lying to my customers.

Out of curiosity, how much is it viable to store? I know that eg. Braid did a ton of compression to make it possible to rewind time a lot, so if you’re doing something like that, just naively making keyframes every second is not going to give good results. Thoughts on that?

The Super Meat Boy scenario looks like it’d probably just work straight out of the box, which is nice. Ghosts like that are also nice to have in racing games, but that will require the system to handle very long animations. Is there any upper limit to AnimationClips in Unity?

1 Like

About the too expensive scene baked at 60 fps, that’s basically what you’ll see when it is said “rendered in engine” (compared to “in real-time”). It means that the engine is capable of very impressive stuff, but when all the features are put together, it would need a monster rig to run it :wink:

I haven’t tried it yet, but I think it would be possible to update the scene at a fixed time using Unity - Scripting API: Time.captureFramerate. So you can record at 60 fps with the GameObjectRecorder, and play it back at runtime using way less resources. It might be better than using a time scale since you will have an exact control over the resulting FPS.

About the Braid game (which I love), note that it is done at runtime, which is impossible (yet) with the GameObjectRecorder that works in Editor only (the problem being about creating assets at runtime, we’re looking at it).

That being said, the GameObjectRecorder is already compressing the resulting animation. Right now the compression is light and the compression settings aren’t exposed in the GameObjectRecorder API, but they could be if requested.

For the go-back-in-time feature, be aware that the game has to be thought that way from the very beginning; it would be very difficult to plug this kind of feature afterward because some events, effects, etc. aren’t “symmetric”: you can play it forward, but not backward. Think animation events for instance, it triggers something (like a GFX or a sound) over time when played forward, but you don’t want it to be triggered at the same time when played backward. Also you’ll have to be very careful about what you’ll record obviously. If you take all the properties of all the GameObjects of your scene, that’s going to use a lot of memory! :wink:
But, other than that, I think that recording every relevant GameObject would be the way to go (at least at first) for the go-back-in-time feature. Extra care would have to be given to GFX, like particules… I don’t know how “recordable” the particules are.

For the long recording sessions or the ghosting feature, I don’t have any figures, but it would be interesting to see how the GameObjectRecorder behaves when we stress-test it like that.

1 Like

Hey this sounds interesting, could it potentially be used to save gameplay replays?

For a simple video demo, I think screencasting might be more appropriate.
But if you want to do something more fancy – say editing videos of the same gameplay sequence but with different skins, terrain, etc… – it could indeed be useful.

This looks like a great feature! I wish it’d been available last year, would have saved me heaps of time.

Am I right in thinking this can save a hierarchy of transforms, i.e. a skeleton, easily?

Thanks! :wink:

If by “save” you mean “recording”, then yes, the GameObjectRecorder can record any hierarchy you want, it doesn’t support humanoid though (yet).

But if by “save” you mean “remove”, then no, we don’t touch the hierarchy of the recorded scene.

I mean as in “recording”, sorry that wasn’t very clear. I like the word “yet” in your response :slight_smile:

Out of curiosity, why would a humanoid rig need to be handled any differently than any other hierarchy of transforms?

Great job again!

The binding system (i.e. the system that binds the curves to the bones) is different for the humanoid. The problem is that there can be only one humanoid per animator. So if you record two characters in the same scene, you will get two humanoids in the same .anim and so you’ll have two humanoids for one animator which will probably fail to play in very strange ways.

But fear not, we’re thinking of a solution as we speak, but it might not come right away. One solution might be to be able to record several GameObject and create a Timeline with the several .anim files generated.

Great stuff. Thanks for the response and keep up the good work!

Can it be used to bake Cloth physics? example: Record a flag waving for a few seconds and save it as a clip so we could remove the Cloth component and save performance.

1 Like

Yes, that’s exactly the kind of use it is meant for!
If it doesn’t work, you have a golden ticket to the bug reporter :smile:

1 Like

For me this would be really useful if it worked in the editor-less Player :slight_smile: Because today I got the task of recording the motion (and spawning) of everything in our project, to be saved to disk. Then later we want to play back exactly what happened, fast forward to a specific part, and then let the project continue running.

It’s a research project, so the goal is to be able to play back how an event went down, change one part, and then see how that affects the outcome.

That’s an interesting project!

Though, even if the GameObjectRecorder was available in the standalone player, in it’s actual implementation, it would probably not be able to record a whole gaming session with all the objects in the scene. It represents a important amount of data and the current implementation keeps everything in memory until the end of the session where it optimize and saves everything in a .anim file.

But for now, the main problem is that we can’t create editor assets in the standalone player. So we would have to bring this big piece of code first if we want to have the GameObjectRecorder working in the standalone player. Depending on the popularity of this feature, it could happen in the future…

But in the meantime, for your project, I would advise you to save into your format (known or internal), because eventually, the serializer would probably not be the most important part of your project :wink:

1 Like

This looks very promising. Can’t wait to get my hands on this.

Question, would this be able to record blendshape values on a skinned mesh renderer component?

I implemented a very similar system that stores the movement and arbitrary values on components animation files. Yes, this could be made to record replays as long as you record the time that gameobjects are created and place the animations on them.

However, there is a problem. These replays get really big, really fast. I had a rather simple Asteroids game the replays would just be enormous. I even wrote an algorithm that simplifies the curves when saved but still, even then, they get very large. I wanted to record a number of 5-minute for an attract mode or something, but it was just generating too much data. But Asteroids is kind of a worst case scenario with this type of replay. There are so many little objects being created and tracked with many bullets, small asteroids, etc.