One thing that always feels ugly to me is that getting data into/out of a job seems incredibly expensive, almost to the point of eliminating any significant performance gains of putting the data into a job in the first place. This mostly appears to be due to populating native arrays prior to executing the job, and reading data out of native arrays after the job is complete.
Here I’m populating the array before executing the job:
Here I’m reading the array after executing the job:
This profiling is being done in a dev build with deep profiling enabled.
Is there something I can be doing to greatly reduce this overhead? I’m assuming it’s proportional to how much data is going into the array. (I’m currently putting a couple of Vector3s, a couple of bools, and some other floats.) I’m not sure I can really reduce how much data is going in.
I’m on Unity 2019.4. Is there any hope that this sort of thing is faster in later versions of Unity? Faster using il2cpp? Is there just some other approach I should be using to populating a native array for my jobs?
I’ve been working on optimizing my code, following some hunches, and so far I’ve gotten some really significant improvements in performance by ensuring that the structs in my NativeArrays are as small as they possibly can be. The get/set_item call performance is directly proportional to the size of the struct.
Previously, I was using one fairly large struct as the Input and Output of my job. That seemed efficient to me when I first authored that code. But reading that NativeArray after running the job was very expensive due to all of the “input” data in the struct.
So, I refactored my code to have two Native Arrays, one for input, and one for output. Basically my old struct was split into two structs. I’m finding that populating the input takes less time now (set_item has less work to do), and reading the output takes much less time, since get_item is only accessing a very small struct compared to what it used to be getting.
I’m just writing this up in case it helps anyone. My new “rule” for job performance will be to keep structs as small as possible, even if it means having multiple arrays. This approach of separating input from output has resulted in something like a 40% performance improvement for my code, given that such a large portion of the execution time was just get/set_item.
1 Like
Separating input from output is generally a good idea. The reason this is particularly slow when executed from MONO is that NativeArrays by default copy by value while arrays return by ref. We can’t return by ref by default in NativeArray since that means we don’t know if users will read or write to the data which is critical to know when writing safe multithreaded code.
For the time being it is possible to use unsafe code using var ptr = (MyStruct*)NativeArray.GetUnsafePtr();
and access the ptr directly, this operates on the values the same way as “by ref” by default.
We will also introduce a ref T ElementAt(int index); method to NativeArray and all other containers in future releases.
Generally speaking when using NativeArray, you should aim to have all the code that pushes large amounts of data in / out of it in fully bursted code. managed arrays are natively optimized by the JIT itself, the only way to beat that performance is using pointers directly in MONO. In bursted code the picture looks of course completely different and native array results in highly optimized code.
7 Likes
To my surprise, and a bit to my disappointment, I think the large NativeArray copy actually seems to have no impact on performance after all, in a release build. In a Dev build, with Deep Profiling enabled, I was seeing big improvements in both FPS and method call timing, as I reduced the data in my NativeArrays. That was exciting. But when comparing Release builds of the before and after, the FPS was identical. I have to assume that the performance differences I was seeing in the dev build are just optimized away in a release build. Pretty surprising, and makes me wonder how I can effectively profile a dev build if some of the code I’m profiling won’t even exist in a release build…
I guess I would try for that, if it weren’t that the inputs (and outputs) of my NativeArray have Physics dependencies. for example, I’m populating the NativeArray with Positions and Masses of rigidbodies, and computing forces which end up as the output, to be applied to the rigidbodies. I don’t think there’s any way to do that kind of thing in bursted code.
Yeah I feel like the only way to really profile Jobs/Burst stuff is to compare timings in release with your own instrumentation. The profiler isn’t yielding much.
I ended up on this thread after walking into the Editor Profiler and finding a big NativeArray.`1.set_Item() eating up time… but I am going to just say that’s likely not an issue in release.
Other than NativeList, I don’t see this method on other containers. Is it still on the plan or you have changed your thought?
You can use extension methods to implement it
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref T ElementAsRefUnsafe<T>(this NativeList<T> list, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(list.GetUnsafePtr(), index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this NativeList<T> list, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(list.GetUnsafeReadOnlyPtr(), index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref T ElementAsRefUnsafe<T>(this NativeArray<T> array, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafePtr(), index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this NativeArray<T> array, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafeReadOnlyPtr(), index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref T ElementAsRefUnsafe<T>(this UnsafeList<T> list, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(list.Ptr, index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this UnsafeList<T> list, int index) where T : unmanaged
{
return ref UnsafeUtility.ArrayElementAsRef<T>(list.Ptr, index);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this UnsafeList<T> list, int start, int length) where T : unmanaged
{
return new ReadOnlySpan<T>(list.Ptr + start, length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeList<T> list) where T : unmanaged
{
return new ReadOnlySpan<T>(list.GetUnsafeReadOnlyPtr(), list.Length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeList<T> list, int start, int length) where T : unmanaged
{
return new ReadOnlySpan<T>(list.GetUnsafeReadOnlyPtr() + start, length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this UnsafeList<T> list) where T : unmanaged
{
return new ReadOnlySpan<T>(list.Ptr, list.Length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeArray<T> array, int start, int length) where T : unmanaged
{
return new ReadOnlySpan<T>((T*)array.GetUnsafeReadOnlyPtr() + start, length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeArray<T> array) where T : unmanaged
{
return new ReadOnlySpan<T>(array.GetUnsafeReadOnlyPtr(), array.Length);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ref T ValueAsRefUnsafe<T>(this NativeReference<T> nativeRef) where T : unmanaged
{
return ref UnsafeUtility.AsRef<T>(nativeRef.GetUnsafePtr());
}
4 Likes
