Why is the new input system so slow?

Hey @theinlein ,
How many input bindings are you currently using? We have seen performance issues where there are a lot of input bindings active at the same time. Have you tried turning off Script Debuggin, turning off Deep Profiling and building with IL2CPP ? As shown above, this helps dramatically improve input performance.

That being said, we have been working on some tooling around this area of the code to help track down the performance problems and make improvements but also to ensure future updates do not cause a performance regression. I hope to share more news soon.

We ripped it out of our project pretty quick, so no I don’t have this stuff on hand. However if you email us (hello@funktroniclabs.com) i think @kalineh will be able to get you more information, maybe check back farther in git to get the scripts you want to reference.

1 Like

Update: We’re (unfortunately) using the New Input System for the PC version of our Quest 2 game, still. We had reports along the lines of “after a few hours of playing, the game’s framerate becomes unbearably slow,” so of course I looked into it ASAP. What a surprise, once again it was the input system biting us in the butt. Here’s what I uncovered.

Take a look at this short video.

09d1x2

This is me scrubbing through a unity profile timeline of snapshots taken across the game over about 30 or 40 minutes of playtime. In a non deep profile in a debug build, things start out okay on PC. Roughly 0.1ms or so. Way too high, but you know, it’s manageable. Towards the end, we’re looking at well over 1ms over overhead. Something is up. After some debugging via modifying the input systems source (looks like we’re on version 1.5.0) - I noticed that the internal list of “maps” was growing in size. There were a LOT of “null” entries and a lot of duplicates of the VR HMD and controllers. So, I modified the following lines:

This seems to have resolved our “performance degrades over time” issue entirely, as long as I call DisableAllActions occasionally (loading screens). Yay! Insane that I had to do so much debugging to find this, but I’m glad. However, I’m still pretty upset that it’s eating up 0.1ms so I started debugging that a bit too.

What I found was that most of the time was spent in DeferBindingResolutionWrapper’s Dispose() function. It looks like every single action callback has a using(DeferBindingResolutionWrapper), which gets auto disposed. Unfortunately in that Dipose it seems to be handling input rebindings… every… time???

So I made the following change:

With this one change, the input system’s update is now ~0.05ms, which is still too slow but I’m a lot happier about.

8804089--1197331--upload_2023-2-13_17-22-18.png

I wish I didn’t have to spend so much time debugging this. I lost an entire day of work just to make a few changes; but hey I’m glad our game is going to run faster and longer for many players now.

15 Likes

Hey @funkyCoty ,
Thank you for finding this and posting all this info, we really do appreciate the effort and diagnostics you have done on these performance issues. I will say that this has been a hot topic the last couple of days here and we are getting all the right people involved to look into it. Could you please file an official bug report (in Editor go to Help > Report a Bug…) with your findings and anything else that might be helpful for us to reproduce on our end, such as logs or information related to scene loads or setup/teardown of devices. This will help escalate the issue and get it resolved sooner.

3 Likes

Hi @funkyCoty ,

Just wondering if you’d experimented with this: x.com

1 Like

All of my posted profiles were already using that.

1 Like

I’ll try to when I find a chance. Honestly we’re trying to ship a game (live this wednesday!!) on quest/ps5/pc all at the same time so we’re actually incredibly swamped at the studio right now. Maybe after the launch stuff calms down I might find a chance to do this. The only reason I dug so deep was because our PC beta testers were having serious issues.

1 Like

Thanks for the update. It looks like you guys made it over the line, congratulations! The game looks fantastic by the way, we’ve been excited about it over here.

As far as the bug you discovered is concerned, I have created an internal ticket and escalated this forum post to our Input team and the XR Foundations team, so both sides are taking a look at it, making sure we’re using the Input System APIs the right way from the XR side and fixing things from the Input System package side as well.

4 Likes

Congratulations on the launch! This looks really great. Reviews seem great and looks like an overall success. I hope the sales are what you hoped for :slight_smile:

3 Likes

Any news on this?

IMO the problem with the Input System is overengineering. Those call stacks are far too deep, with layers upon layers of unnecessary abstraction between reading the low level input and it finally informing the game about it. This is the kind of code that shows the weakness of IL2CPP: deeply nested virtual/delegate/interface calls prevent a lot of compiler optimizations due to the extra cruft needed by the dotnet execution model.

1 Like

Hi! I work on the Optimization team and I’ve recently been starting to investigate this. I haven’t yet found any projects using the input system that show this slowdown, and I haven’t been able to build a sample that shows the problem either. If anyone here has a sample project that shows this slowdown it would be really useful if you could share it (ideally by filing a bug with the project attached, or send me a DM) so that I can try and fix the problem.

I had a problem with the input system performance for VR and when I found this thread I thought it was related. I’ve just created a bug report (IN-42801). But while making the small demo sample it looks like a different issue. My problem seems to come from not disabling inputs when switching scenes, which resulted in a 17ms “PreUpdate.NewInputUpdate” after a few scenes loads :face_with_spiral_eyes:.
I post this here because I don’t understand enough about the input system to definitely say its a completely unrelated problem.

1 Like

This does sound like it could be related - we’ll try testing with your sample and see if we can work out what’s going on. Thank you for reporting it!

1 Like

After doing some more tests I found out that just having many components getting their own input in code destroys the performance. 50 components all getting position and rotation of the left and right controller would cost me ~1ms per frame, even without a scene reload and correctly disabling the input. While they should all get the exact same values (input mode is on dynamic, so I am not manually repolling the newest values from the device, as far as i know.)

I’ve added this as a comment under the Bug report, since its mostly the same setup.

Have you tried using the caching and fast path optimization flags that shipped with 1.5? The caching optimization in particular should alleviate the 50 component issue, but I’m curious to see how it affects your scene in practice.

The package manager says I am on version 1.5.1 of the Input system.

I’ve added a GameObject to the scene with a component with this code :

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.InputSystem;

public class ApplyPerformanceInhancerCodeForInputSystem : MonoBehaviour
{
    private void Awake()
    {
        Debug.Log("Optimise!!");

        InputSystem.settings.SetInternalFeatureFlag("USE_OPTIMIZED_CONTROLS", true);
        InputSystem.settings.SetInternalFeatureFlag("USE_READ_VALUE_CACHING", true);
        InputSystem.settings.SetInternalFeatureFlag("PARANOID_READ_VALUE_CACHING_CHECKS", true);
     
    }
}

From the changelog + twitter it seems like this can be run anywhere to activate… Or should I run it in the input script or at a special time? It looked like a global setting.

The “Optimise!!” from the debug line is shown in console, but nothing else.

The performance looks the same, not better not worse. Please let me know if I need to apply this code in some other way, since it does not seem to do anything now.

When I look in the “Optimized” column of the Device Debugger I do see + signs. (here is a screenshot)
The scene in the bug report has the input of the controllers Position and Rotation set to PointerPosition and Pointer Rotation. These have a ‘+’.
9055186--1251664--upload_2023-6-3_21-44-54.png

1 Like

I would remove the paranoid value check, I think that one slows things down for debugging.

Indeed if the + is there the control is optimized, it should work on all controls that don’t have any processors on them.

Thanks a lot for providing information.

@ericprovencher is there any chance we users could get a low level, absolutely minimal input system capable of optimal levels of low latency and MIDI input recording friendly rates of polling?

Upon this, all sorts of performance critical input could be managed.

1 Like

I’ve commented out the paranoid check. I’ve reimported the Input System (just in case). I’ve restarted Unity.

Performance is roughly the same, ~1ms per frame for input.

Sorry, I know this is not what anyone wants to hear when there is code in place that should make this case work. And it might still be something that I am doing wrong. So hopefully you (or someone at unity) can reproduce the bug from the bug report and fix this, or tell me what I am doing wrong. Really appreciate the direct communication.

1 Like