Mathf.FloatToHalf very slow

AI Beta Sentis
1

Although Sentis does not support float16 as of yet. There may be times where you want to convert an array of float16’s to float32 or vice versa. (Perhaps you have some input data downloaded from Hugging face in float16 format. Or you want to output results to some compute shader on the GPU.)

It would be nice if there was a way to convert arrays of float16 (Halfs) to float32’s and vice versa. Perhaps implemented as a compute shader or using Burst. The built in Mathf.FloatToHalf is not suitable for this.

I have used Mathf.FloatToHalf when using Onnx Runtime and found it very slow so I ended up creating a small neural network on the GPU that just converted float arrays to half arrays.

dot net 7 has the Half type: Half Struct (System) | Microsoft Learn but not sure if you can convert arrays with it.

Sentis must be using something like this when you drag a float16 onnx file to the asset folder it gets converted to float32. So maybe there is this function already which can be exposed?

2

Yeah you are correct there is some in-efficiencies going on there. Luckily it’s only at compile time so run time is not affected, but it doesn’t justify that we should be doing that conversion more efficiently.
In the meantime if you wish to do it yourself, here is a Burst kernel that does it

    [BurstCompile]
    unsafe struct ConvertHalfToFloatJob : IJobParallelFor
    {
        public NativeArray<half> datah;
        public NativeArray<float> dataf;

        public void Execute(int i)
        {
            dataf[i] = (float)(datah[i]);
        }
    }
1 Like
3

Unfortunately Unity does not support .NET 7 yet, but we’re actively work on that support now, so I suspect that this will be possible in the future.