Based on Parallel procedural mesh generation with Job System I would like to parallelize and jobify the Mesh and texture creation which is one of my main bottlenecks. I did want to post there because I already have all mesh data and don’t want to construct it and now have troubles getting anything to run with Burst.
Version:
- Unity 2020.3.34f1
"com.unity.jobs": "0.8.0-preview.23",
"com.unity.burst": "1.6.4",
"com.unity.entities": "0.17.0-preview.42",
"com.unity.collections": "0.15.0-preview.21",
I ran into the following problems:
- Unity complains that I may not use my BlobArray and BlobArray inside a job because it was constructed from UNSAFE POINTER. I use reinterpretation to reinterpret it to a NativeArray
- All MeshApi seems to be main thread only
Which essentially left no work left that I was able to run inside the job, as I continously had to move everything back to the main thread until it ran, which is where it is working now, but because I need to run it so often per frame I would like to parallelize and burst jobify as much as possible.
This is the data I am trying to construct a Mesh and Texture serialized blobs of the following types:
public struct RenderableBlob
{
public MeshBlob Mesh;
public TextureBlob Texture;
}
public struct MeshBlob
{
public AABB RenderBounds;
public BlobArray<VertexData> Vertices;
public BlobArray<int> Indices;
}
public struct TextureBlob
{
public TextureFormat Format;
public int MipmapCount;
public int Width;
public int Height;
public BlobArray<byte> TextureData;
}
public struct VertexData
{
public float3 Position;
public float3 Normal;
public float2 UV;
public static readonly VertexAttributeDescriptor[] Layout =
{
new VertexAttributeDescriptor { attribute = VertexAttribute.Position, format = VertexAttributeFormat.Float32, dimension = 3 },
new VertexAttributeDescriptor { attribute = VertexAttribute.Normal, format = VertexAttributeFormat.Float32, dimension = 3 },
new VertexAttributeDescriptor { attribute = VertexAttribute.TexCoord0, format = VertexAttributeFormat.Float32, dimension = 2 },
};
}
Which are deserialized as follows
ublic static unsafe BlobAssetReference<T> DeserializeBlob<T>(string blobFile) where T : struct
{
var cmds = new NativeArray<ReadCommand>(1, Allocator.Temp);
ReadCommand cmd;
cmd.Offset = 0;
using (var file = new FileStream(blobFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
// new FileInfo: Slow
// cmd.Size = new FileInfo(blobFile).Length;
cmd.Size = file.Length;
cmd.Buffer = (byte*) UnsafeUtility.Malloc(cmd.Size, 1024, Allocator.Temp);
cmds[0] = cmd;
var readHandle = AsyncReadManager.Read(blobFile, (ReadCommand*) cmds.GetUnsafePtr(), 1);
readHandle.JobHandle.Complete();
using (var br = new MemoryBinaryReader((byte*) cmd.Buffer))
{
var result = br.Read<T>();
UnsafeUtility.Free(cmd.Buffer, Allocator.Temp);
return result;
}
}
}
Where br.Read() is using the following implemenation for
MemoryBinaryReader : BinaryReader
public void ReadBytes(void* data, int bytes)
{
UnsafeUtility.MemCpy(data, content, bytes);
content += bytes;
}
After reading the data, I try to construct both mesh and texture. This works on the main thread, but I fail to jobify it without running into all the restrictions with burst and Native Collections. Whatever I tried to move into the job cause Unity to complain it cannot do so (UNKNOWN DATA; UNSAFE POINTER, …)
// FLAG
public const MeshUpdateFlags DoNotRecalculate = MeshUpdateFlags.DontRecalculateBounds
| MeshUpdateFlags.DontValidateIndices
| MeshUpdateFlags.DontResetBoneBounds;
// Main Thread
var ResultMesh = new Mesh();
// PART 1: Mesh Creation
Mesh.MeshDataArray meshDataArray = Mesh.AllocateWritableMeshData(1);
Mesh.MeshData data = meshDataArray[0];
NativeArray<VertexData> nativeInputVertices = renderable.Mesh.Vertices.ToNativeArray();
data.SetVertexBufferParams(nativeInputVertices.Length, VertexData.Layout);
NativeArray<VertexData> vertexData = data.GetVertexData<VertexData>();
vertexData.CopyFrom(nativeInputVertices);
NativeArray<int> nativeInputIndices = renderable.Mesh.Indices.ToNativeArray();
data.SetIndexBufferParams(nativeInputIndices.Length, IndexFormat.UInt32);
NativeArray<int> indexData = data.GetIndexData<int>();
indexData.CopyFrom(nativeInputIndices);
Mesh.ApplyAndDisposeWritableMeshData(meshDataArray,
ResultMesh,
DoNotRecalculate | MeshUpdateFlags.DontNotifyMeshUsers); // no notify
var bounds = renderable.Mesh.RenderBounds.ToBounds();
var subMeshDescriptor = new SubMeshDescriptor()
{
topology = MeshTopology.Triangles,
vertexCount = nativeInputVertices.Length,
indexCount = nativeInputIndices.Length,
bounds = bounds,
indexStart = 0,
baseVertex = 0,
firstVertex = 0
};
ResultMesh.subMeshCount = 1;
ResultMesh.SetSubMesh(0, subMeshDescriptor, DoNotRecalculate /* and do notify mesh users here */);
ResultMesh.bounds = bounds;
// PART 2: Texture Creation
var texture = new Texture2D((int) textureBlob.Width, (int) textureBlob.Height, textureBlob.Format, textureBlob.MipmapCount, false);
texture.LoadRawTextureData(textureBlob.TextureData.ToNativeArray());
if (compress)
texture.Compress(true);
texture.Apply();
Material m = renderable.Texture.Format switch
{
TextureFormat.Alpha8 => Hierarchy.Example.MaterialAlpha8,
TextureFormat.RGBA32 => Hierarchy.Example.MaterialRGBA,
TextureFormat.RGB24 => Hierarchy.Example.MaterialRGB,
TextureFormat.R8 => Hierarchy.Example.MaterialR8,
_ => throw new NotImplementedException()
};
var material = new Material(m);
material.SetTexture("_BaseMap", texture);
The slowest parts are:
-
“Part 1: Mesh Creation”:
-
readHandle.JobHandle.Complete(); // Reading via AsyncReadManager
-
MemoryBinaryReader.ReadBytes(); // Reading the into the struct
-
“Part 2: Texture Creation”: texture.LoadRawTextureData - and as this already is using the NativeArray I am not sure how this could be sped up.
Is it possible to parallize anything of this with burst compiled jobs or make it faster any other way?