C# HasAFlag method extension: how to not create garbage allocation?

[FeastSC2]

I would like to have an extension method on my flags to know if they have a certain flag in them or not.
The problem is that the HasAFlagOf code allocates garbage and I just use flags too much to allow this. How can I not allocate garbage and use a method that works for my flags?

public static class EnumExt{
    public static bool HasAFlagOf<T>(this T value, T flag) where T : struct // allocates garbage
    {
        return (Convert.ToInt64(value) & Convert.ToInt64(flag)) != 0;
    }
    public static bool HasAFlagNoGarbage(this SH_UberFx.XY value, SH_UberFx.XY flags) // NO garbage allocated but I must create many different variations of these because it is type dependant
    {
        return (value & flags) != 0;
    }
    public static bool HasAFlagNoGarbage(this SH_UberFx.MainTex value, SH_UberFx.MainTex flags)// NO garbage allocated but I must create many different variations of these because it is type dependant
    {
        return (value & flags) != 0;
    }
}

Here’s how I create all my flags ( I just need the method to work with that kind of flag enum ):

    [System.Flags]
    public enum VegetationBitmaskEnum
    {
        Wind = 1 << 0,
        Ondulation = 1 << 1,
        Rotation = 1 << 2,
        Scale = 1 << 3,
        //All = Wind | Ondulation | Rotation | Scale
    }

[Gladyon]

I guess that the culprit is ‘Convert.ToInt64()’, did you tried just casting?

The way I used to manage the Enums without garbage is to dynamically create the cast methods:

DynamicMethod AreEqualMethod = new DynamicMethod("AreEqual", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, typeof(Boolean), new Type[] { TheType, TheType }, typeof(EnumEqualityComparer), true);
ILGenerator TheIlGenerator = AreEqualMethod.GetILGenerator();
TheIlGenerator.Emit(OpCodes.Ldarg_0);
TheIlGenerator.Emit(OpCodes.Ldarg_1);
TheIlGenerator.Emit(OpCodes.Ceq);
TheIlGenerator.Emit(OpCodes.Ret);
AreEqual = (Func<TEnum, TEnum, Boolean>)AreEqualMethod.CreateDelegate(typeof(Func<TEnum, TEnum, Boolean>));

DynamicMethod CastToInt32Method = new DynamicMethod("CastToInt32", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, typeof(Int32), new Type[] { TheType }, typeof(EnumEqualityComparer), true);
TheIlGenerator = CastToInt32Method.GetILGenerator();
TheIlGenerator.Emit(OpCodes.Ldarg_0);
TheIlGenerator.Emit(OpCodes.Ret);
CastToInt32 = (Func<TEnum, Int32>)CastToInt32Method.CreateDelegate(typeof(Func<TEnum, Int32>));

DynamicMethod CastToValueMethod = new DynamicMethod("CastToValue", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, TheType, new Type[] { typeof(Int32) }, typeof(EnumEqualityComparer), true);
TheIlGenerator = CastToValueMethod.GetILGenerator();
TheIlGenerator.Emit(OpCodes.Ldarg_0);
TheIlGenerator.Emit(OpCodes.Ret);
CastToValue = (Func<Int32, TEnum>)CastToValueMethod.CreateDelegate(typeof(Func<Int32, TEnum>));

The problem is that it won’t work on mobile, as I think it’s not possible to generate IL code at runtime on mobile.
It also probably won’t work with IL2CPP.

And to use it, I created a generic static class:

static public class EnumExt<TEnum>

That way I can ‘remember’ everything I need for each type of enum, including all the values and names, that way I can even have a ‘toString()’ which is garbage-less.

[FeastSC2]

@Gladyon

I guess that the culprit is ‘Convert.ToInt64()’, did you tried just casting?

Indeed it is, but casting doesn’t work for some reason.

I’m not familiar enough with your code to use it but when I’ll be better in programming I just might use it.

Could there be a simpler solution by any chance?

[Gladyon]

You’re right, just casting doesn’t work, which explain why I had to use the code in my previous post (it was long ago, I wasn’t sure if it was the reason or not).

I’m afraid that there’s not really any other way. In fact, it’s not a .NET limitation, it’s a C# one. Such a casting is allowed in IL code but not in C# because of a syntax limitation.

That said, there is another way, which is not very pretty, but which works quite well.
You can generate C# code with all the Enums you need.
I mean, you just have to write a program which writes the source code you need in plain C#, as if you wrote it yourself. It will simply write the same functions over and over again, just changing the type of the enum.
You name the created file as a ‘.cs’ file and you include it in the Unity asset folder as you would for any source file you write yourself.

The problem is that each time you need to have another type of enum, you need to to add it to the list of enums your code generator generates, and generate the code again.

Note that you can also use reflection in order to scan you source code to find all the enums you have, and automatically generate the source file for all the ‘HasAFlagNoGarbage()’ methods your source code requires.
That takes a bit more skills, but honestly not a lot. Reflection is a lot easier than the code generation I put in my previous post.
Take a look there:
https://stackoverflow.com/questions/4247410/c-sharp-or-vb-net-iterate-all-public-enums
It contains that method:

var query = Assembly.GetExecutingAssembly()
                    .GetTypes()
                    .Where(t => t.IsEnum && t.IsPublic);

foreach (Type t in query)
{
    Console.WriteLine(t);
}

That will find out all the Enum types in the currently executing assembly, and as you can see, it’s not that complex.
Instead of the ‘WriteLine()’ call, you’ll have to write in a text file the text corresponding to the content of your ‘HasAFlagNoGarbage()’ method (‘t.name’ will contain the name of the enum type).

[xVergilx]

There’s not much you can do.
About the allocation, I don’t think the Convert.ToInt64 does that.
It’s the boxing allocation that is performed on the value in order to convert it from generic to the object and then pass it to the convert method.

Writing the code manually or generating the code is the solution unfortunately.
As you can’t really apply constraints for the value types.

[Baste]

“The implementation of an allocation-free version of HasFlags is fairly straightforward:”

From “dissecting new generic constraints in C# 7.3”. The core change that enables this is the where T : Enum constraint introduced in C# 7.3

[lordofduct]

I’ve played with the same idea of having enum extension’s for flags and the sort.

And well… it just don’t work. You will get boxing all the time because the compiler doesn’t know what type it is. It’s just going to happen. It doesn’t help that when treated as System.Enum it gets boxed.

I gave up on the idea and just inline my flag checks into the code.

That or you convert them to long’s before passing them into the function you write. Or using newer versions of C# that do support the proper constraints, which I’m pretty sure aren’t in Unity yet.

[Baste]

C# 7.3 is supported in 2018.3 and 2019.1. You can get access in 2018.2 with the (deprecated) incremental compiler package.

[Gladyon]

@Baste

“The implementation of an allocation-free version of HasFlags is fairly straightforward:”

From “dissecting new generic constraints in C# 7.3”. The core change that enables this is the where T : Enum constraint introduced in C# 7.3

It’s not the ‘T : Enum’ which makes it garbage-free, it’s the IL code generated, which also work with .NET 3.5.
Unfortunately it has some limitation with mobiles and IL2CPP probably.

Also, it seem to be too complex for the OP, which is why I think that generating C# code in plain text instead of IL code is probably the best way to go.

Thanks for the link, it’s very interesting to know about the new constraints, that can avoid some runtime checks.

[FeastSC2]

That seems all very interesting, thanks for the many answers.

Where did you guys learn this?
Where would you go to learn C# online as thoroughly as possible nowadays?

[xVergilx]

@Baste

“The implementation of an allocation-free version of HasFlags is fairly straightforward:”

From “dissecting new generic constraints in C# 7.3”. The core change that enables this is the where T : Enum constraint introduced in C# 7.3

Good to know. Now I can finally avoid boxing on the generic enums.

[Gladyon]

@FeastSC2

That seems all very interesting, thanks for the many answers.

Where did you guys learn this?
Where would you go to learn C# online as thoroughly as possible nowadays?

I’m sorry, I can’t really help you on that one.
I started to code more about 35 years ago, even before I got access to a BBS with a 9600 bauds modem, before the Internet era… so I had to learn by myself, without any forum or friends who knew how to code.

Also I do not know C#, or any other language, thoroughly. There are always more to know, it’s an endless task.
In addition, knowing how the compiler work can be important in some situation, as can be how the CPU works, or the RAM, etc.

My best advice would be for you to be curious about everything.
When you see some bit of code which corresponds to your needs in a forum, do not just copy/paste it. You have to understand it, dissect every line until you’re confident you know the purpose of every line.
It takes time, as I said you’ll never stop learning things, but the best way to know things is to understand them completely.

In order to do that, I usually do not use copy/paste. I write it down character by character, line by line, and if there’s a line which has no meaning to me, I just do not copy it, to see what happen.
Usually it doesn’t work, so I look into it with the debugger, and very often the meaning of the line/instruction becomes clear, or at least some of the fog disappear.

[Suddoha]

Without the new enum constraint, one could try the following:

Enumerations in C# are always based on integer types.

So that one thing that needs the workaround is getting that underlying value of any enumeration type generically into an actual integer representation, so that we can benefit from all the operators.
As we figured out, there’s no way to use usual casting approaches without additional type checks, boxing etc, but thinking on a lower level as you’d do in C++, we can leverage some features that are available in C# as well, at least to some extent: Unions in C++, or explicit struct layouts in C#.

That is, we set the offset of an enumeration field to the same as a 64bit wide integer field, in order to read/write the values from/to the same location. (As the set of unsigned long values contains all the potential enumeration values, i.e. the enum’s set is much smaller, the unsigned long will only be writable privately - otherwise a consumer could quickly mess things up)

The result could be this:
(only added the stuff that gets you going, there’s potentially a lot more to add like bitwise OR, other extension methods etc):

using System;
using System.Runtime.InteropServices;

public static class EnumExtensions
{
    [StructLayout(LayoutKind.Explicit)]
    private struct EnumValue<TEnum> where TEnum : struct // could add some more interfaces to exclude more value types
    {
        static EnumValue()
        {
            if (!typeof(TEnum).IsEnum)
            {
                throw new Exception("Generic type argument must be an enum.");
            }
        }

        [FieldOffset(0)]
        private ulong _backingValue;

        [FieldOffset(0)]
        public TEnum Value;

        // this one is only ever going to be used internally, and that's within the overloaded operators
        private EnumValue(ulong value)
        {
            Value = default(TEnum);
            _backingValue = value;
        }

        public EnumValue(TEnum value)
        {
            _backingValue = 0;
            Value = value;
        }

        // example for the bitwise AND operator
        public static EnumValue<TEnum> operator &(EnumValue<TEnum> lhs, EnumValue<TEnum> rhs)
        {
            return new EnumValue<TEnum>(lhs._backingValue & rhs._backingValue);
        }

        public static implicit operator ulong(EnumValue<TEnum> rhs)
        {
            return rhs._backingValue;
        }
    }

    public static bool HasFlagOf<TValue>(this TValue value, TValue flag) where TValue : struct
    {
        var lhs = new EnumValue<TValue>() { Value = value };
        var rhs = new EnumValue<TValue>() { Value = flag };
        return (lhs & rhs) != 0ul;
    }
}

Note that this allows to write and possibly compile code with any value type (if you add some more interfaces as constraint you can reduce the number of types again).
The static initialization block will throw at runtime when an invalid generic type is used (you could extend this to support other numeric types if you wanted to).

I left out the other operators and potential extension methods, as that will be easy to add.
Might rename everything, but it’s just a quick example.

[MartinTilo]

@Suddoha You’re code compiles but throws runtime exceptions because generic types can’t have an Explicit layout.
This implementation is simpler, also GC.Free, should still be burst compilable, and when profiled in editor, 10x faster than .HasFlag:

using System;
using Unity.Collections.LowLevel.Unsafe;
static class EnumTools
{
    static uint NumericType<TEnum>(TEnum t) where TEnum : struct, Enum
    {
        unsafe
        {
            void* ptr = UnsafeUtility.AddressOf(ref t);
            return *((uint*)ptr);
        }
    }
    static ulong NumericTypeULong<TEnum>(TEnum t) where TEnum : struct, Enum
    {
        unsafe
        {
            void* ptr = UnsafeUtility.AddressOf(ref t);
            return *((ulong*)ptr);
        }
    }
    static byte NumericTypeByte<TEnum>(TEnum t) where TEnum : struct, Enum
    {
        unsafe
        {
            void* ptr = UnsafeUtility.AddressOf(ref t);
            return *((byte*)ptr);
        }
    }
    static int s_UIntSize = UnsafeUtility.SizeOf<uint>();
    static int s_ULongSize = UnsafeUtility.SizeOf<ulong>();
    static int s_ByteSize = UnsafeUtility.SizeOf<byte>();
    public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum : struct, Enum
    {
        int size = UnsafeUtility.SizeOf<TEnum>();
        if (size == s_UIntSize)
        {
            return (NumericType(lhs) & NumericType(rhs)) > 0;
        }
        else if (size == s_ULongSize)
        {
            return (NumericTypeULong(lhs) & NumericTypeULong(rhs)) > 0;
        }
        else if (size == s_ByteSize)
        {
            return (NumericTypeByte(lhs) & NumericTypeByte(rhs)) > 0;
        }
        throw new Exception("No matching conversion function found for an Enum of size: " + size);
    }
}

Note: you could get around the size checks by leaving it to the caller to specify the type. In my benchmark however, the size check cost is negligible so I’d go with the safe option.

Also Note: you don’t actually need the Enum constraints so if you’re on .Net 3.5 you can just remove it.

Thanks to @alexrvn for the unsafe magic

[Suddoha]

@MartinTilo

That’s weird, it compiles and runs just fine for me in 2018.1. I’ve ran it myself to ensure it works.

But yea, I get why it wouldn’t be allowed, just looked it up.

The unsafe one looks great. Thanks for sharing.

[Baste]

@Suddoha

That’s weird, it compiles and runs just fine for me in 2018.1. I’ve ran it myself to ensure it works.

Running on Mono? I’ve seen Mono have bugs where stuff that’s not permitted by the C# specs compiles and runs.

[MartinTilo]

@Suddoha Using 2018.3 with .Net 4.x I get:

Error while loading types from assembly Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null: System.Reflection.ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
  at (wrapper managed-to-native) System.Reflection.Assembly.GetTypes(System.Reflection.Assembly,bool)
  at System.Reflection.Assembly.GetTypes () [0x00000] in <ac823e2bb42b41bda67924a45a0173c3>:0
  at UnityEngine.Experimental.UIElements.VisualElementFactoryRegistry.DiscoverFactories () [0x0006d] in C:\buildslave\unity\build\Modules\UIElements\UXML\VisualElementFactoryRegistry.cs:54

EnumExtensions/EnumValue`1 : Generic class cannot have explicit layout.

UnityEditor.EditorAssemblies:ProcessInitializeOnLoadAttributes()

and

TypeLoadException: Generic Type Definition failed to init, due to: Generic class cannot have explicit layout. assembly:C:\Users\martinsch\Documents\Projects\AdventureGame\Library\ScriptAssemblies\Assembly-CSharp.dll type:EnumValue`1 member:(null) signature:<none>
EnumTestScript.Update () (at Assets/Scripts/EnumTestScript.cs:54)

it does work with .Net 3.5 and performance is quite similar to the unsafe solution.

[Suddoha]

Oh I see. Good catch both of you @Baste and @MartinTilo .

@Baste

Running on Mono? I’ve seen Mono have bugs where stuff that’s not permitted by the C# specs compiles and runs.

@MartinTilo

it does work with .Net 3.5

That might explain it. Once you told me it does throw at runtime, my first thought was that it could be the other way around.

Just re-checked the project I used and it’s got
Scripting Runtime Version: .Net 3.5 Equivalent
Scripting Backend: Mono

Always great to learn more about such specific cases and “features”. Thanks again.

[OndrejP]

Would it be possible to use UnsafeUtility to copy value from enum to ulong without branching?

EDIT:
It seems to me you should be able to use it like this:

ulong val = 0;
UnsafeUtility.CopyStructureToPtr(ref enumValue, &val);

Then you can easily do any bit operations and if needed write it back using similar method.
Even endianness does not matter as long as you need only bit operations.

[MartinTilo]

Yep that works.
But if we’re already getting rid of the branches, let’s also get rid of the copies

public static class EnumTool
{
    public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum :
#if CSHARP_7_3_OR_NEWER
    unmanaged, Enum
#else
    struct
#endif
    {
        unsafe
        {
#if CSHARP_7_3_OR_NEWER
            return (*(ulong*)(&lhs) & *(ulong*)(&rhs)) > 0;
#else
            ulong valLhs = 0;
            UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
            ulong valRhs = 0;
            UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
            return (valLhs & valRhs) > 0;
#endif
        }
    }
}

Edit: The initial snippet did not show that TEnum was constraint to unmanaged, or that this only works with C# >7.3