Fatal error in Unity CIL Linker

Hell i am doing some modifications to the assemblies just after the compilation and i get a strange error wheni try to build(2018 beta) :
Failed running C:\Program Files\Unity\Hub\Editor\2018.1.0b13\Editor\Data\il2cpp/build/UnityLinker.exe --api=NET_4_6 -out=“E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\tempStrip” -l=none -c=link --link-symbols -x=“C:\Program Files\Unity\Hub\Editor\2018.1.0b13\Editor\Data\PlaybackEngines\AndroidPlayer\Whitelists\Core.xml” -f=“C:\Program Files\Unity\Hub\Editor\2018.1.0b13\Editor\Data\il2cpp\LinkerDescriptors” -x “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed..\platform_native_link.xml” -x “C:\Users\Paul\AppData\Local\Temp\tmp567d5f35.tmp” -x “C:\Users\Paul\AppData\Local\Temp\tmp639bf969.tmp” -x “C:\Users\Paul\AppData\Local\Temp\tmp7099150d.tmp” -x “C:\Program Files\Unity\Hub\Editor\2018.1.0b13\Editor\Data\il2cpp\LinkerDescriptors\mscorlib45.xml” -x “C:\Program Files\Unity\Hub\Editor\2018.1.0b13\Editor\Data\il2cpp\LinkerDescriptors\System45.xml” -d “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\Assembly-CSharp.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\FastSerializer.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\Core.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\Modules.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\Extensions.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\FrameworkUtility.dll” -a “E:\Framework\Temp\StagingArea\assets\bin\Data\Managed\DOTween.dll”

stdout:
Fatal error in Unity CIL Linker
Mono.Linker.MarkException: Error processing method: ‘System.Collections.Generic.List1<System.Boolean> FastSerializer.Autogenerated.SerializerMethodsAndroid::Get_DataHolderBase__System_Boolean_values(Framework.Extensions.DataHolderBase1<System.Boolean>)’ in assembly: ‘Assembly-CSharp.dll’ —> Mono.Cecil.ResolutionException: Failed to resolve System.Collections.Generic.List1<System.Boolean> Framework.Extensions.DataHolderBase1<System.Boolean>::values
at Mono.Linker.Steps.MarkStep.MarkField(FieldReference reference)
at Mono.Linker.Steps.MarkStep.MarkInstruction(Instruction instruction)
at Mono.Linker.Steps.MarkStep.MarkMethodBody(MethodBody body)
at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)
at Unity.Linker.Steps.UnityMarkStep.ProcessMethod(MethodDefinition method)
at Mono.Linker.Steps.MarkStep.ProcessQueue()
— End of inner exception stack trace —
at Mono.Linker.Steps.MarkStep.ProcessQueue()
at Mono.Linker.Steps.MarkStep.ProcessEntireQueue()
at Mono.Linker.Steps.MarkStep.Process()
at Unity.Linker.Steps.UnityMarkStep.Process(LinkContext context)
at Mono.Linker.Pipeline.Process(LinkContext context)
at Unity.Linker.UnityDriver.Run()
at Unity.Linker.UnityDriver.RunDriver()
stderr:

UnityEngine.Debug:LogError(Object)
UnityEditorInternal.Runner:RunProgram(Program, String, String, String, CompilerOutputParserBase) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:128)
UnityEditorInternal.Runner:RunManagedProgram(String, String, String, CompilerOutputParserBase, Action1) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:73) UnityEditorInternal.AssemblyStripper:RunAssemblyLinker(IEnumerable1, String&, String&, String, String) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/AssemblyStripper.cs:89)
UnityEditorInternal.AssemblyStripper:StripAssembliesTo(String[ ], String[ ], String, String, String&, String&, String, IIl2CppPlatformProvider, IEnumerable1) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/AssemblyStripper.cs:82) UnityEditorInternal.AssemblyStripper:RunAssemblyStripper(IEnumerable, String, String[ ], String[ ], String, IIl2CppPlatformProvider, RuntimeClassRegistry) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/AssemblyStripper.cs:203) UnityEditorInternal.AssemblyStripper:StripAssemblies(String, IIl2CppPlatformProvider, RuntimeClassRegistry) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/AssemblyStripper.cs:113) UnityEditorInternal.IL2CPPBuilder:Run() (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/Il2Cpp/IL2CPPUtils.cs:154) UnityEditorInternal.IL2CPPUtils:RunIl2Cpp(String, String, IIl2CppPlatformProvider, Action1, RuntimeClassRegistry) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/Il2Cpp/IL2CPPUtils.cs:35)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

And here is the code:

using System;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using System.Threading;
using FastSerializer.CodeGeneration;
using Mono.Cecil;
using Mono.Cecil.Cil;
using UnityEditor;
using UnityEditor.Build;
using UnityEditor.Build.Reporting;
using UnityEditor.Compilation;
using UnityEngine;
using Assembly = System.Reflection.Assembly;

namespace FastSerializer.Editor
{

    /// <summary>
    /// This class will force unity to recompile all the scripts before building and to generate the serialization scripts
    /// </summary>
    public class SerializationBuildPreprocess : IPreprocessBuildWithReport, IPostprocessBuildWithReport
    {
        public int callbackOrder { get; } = 0;

        /// <summary>
        /// Checks if a <see cref="MethodDefinition"/> is marked with <see cref="PrivateFieldGetAttribute"/> and if so returns the <see cref="PrivateFieldGetAttribute.TargetType"> </see> and <see cref="PrivateFieldGetAttribute.FieldName"/>/>
        /// </summary>
        /// <returns></returns>
        private static Tuple<TypeReference, TypeReference, string> TryGetFieldGetAttribute(MethodDefinition methodDefinition)
        {
            foreach (CustomAttribute methodDefinitionCustomAttribute in methodDefinition.CustomAttributes)
            {
                if (methodDefinitionCustomAttribute.AttributeType.FullName == typeof(PrivateFieldGetAttribute).FullName)
                {
                    return new Tuple<TypeReference, TypeReference, string>(
                        (TypeReference)methodDefinitionCustomAttribute.ConstructorArguments[0].Value,
                        (TypeReference)methodDefinitionCustomAttribute.ConstructorArguments[1].Value,
                        (string)methodDefinitionCustomAttribute.ConstructorArguments[2].Value);
                }
            }

            return null;
        }

        /// <summary>
        /// Checks if a <see cref="MethodDefinition"/> is marked with <see cref="PrivateFieldSetAttribute"/> and if so returns the <see cref="PrivateFieldSetAttribute.TargetType"></see> and <see cref="PrivateFieldSetAttribute.FieldName"/>/>
        /// </summary>
        /// <returns></returns>
        private static Tuple<TypeReference, TypeReference, string> TryGetFieldSetAttribute(MethodDefinition methodDefinition)
        {
            foreach (CustomAttribute methodDefinitionCustomAttribute in methodDefinition.CustomAttributes)
            {
                if (methodDefinitionCustomAttribute.AttributeType.FullName == typeof(PrivateFieldSetAttribute).FullName)
                {
                    return new Tuple<TypeReference, TypeReference, string>(
                        (TypeReference)methodDefinitionCustomAttribute.ConstructorArguments[0].Value,
                        (TypeReference)methodDefinitionCustomAttribute.ConstructorArguments[1].Value,
                        (string)methodDefinitionCustomAttribute.ConstructorArguments[2].Value);
                }
            }

            return null;
        }


        /// <summary>
        /// Finds a field in a <see cref="TypeDefinition"/>
        /// </summary>
        /// <param name="containingType">the type</param>
        /// <param name="fieldName">the type</param>
        /// <returns></returns>
        private static FieldDefinition GetField(TypeDefinition containingType, string fieldName)
        {
            foreach (FieldDefinition variable in containingType.Fields)
            {
                if (variable.Name == fieldName)
                {
                    return variable;
                }
            }

            return null;
        }

        private class CustomResolver : BaseAssemblyResolver
        {
            private readonly DefaultAssemblyResolver _defaultResolver;

            public CustomResolver()
            {
                _defaultResolver = new DefaultAssemblyResolver();
            }

            public override AssemblyDefinition Resolve(AssemblyNameReference name)
            {
                AssemblyDefinition assembly;
                try
                {
                    assembly = _defaultResolver.Resolve(name);
                }
                catch (AssemblyResolutionException ex)
                {
                    foreach (System.Reflection.Assembly loaded in AppDomain.CurrentDomain.GetAssemblies())
                    {
                        if (loaded.IsDynamic)
                            continue;
                        if (loaded.FullName == name.FullName)
                        {
                            return AssemblyDefinition.ReadAssembly(loaded.Location);
                        }
                    }

                    throw ex;
                }

                return assembly;
            }
        }

        public static TypeReference ResolveGenericParameter(TypeReference type, TypeReference parent)
        {
            var genericParent = parent as GenericInstanceType;
            if (genericParent == null)
                return type;

            if (type.IsGenericParameter)
                return genericParent.GenericArguments[(type as GenericParameter).Position];

            if (type.IsArray)
            {
                var array = type as ArrayType;
                ResolveGenericParameter(array.ElementType, parent);
                return array;
            }

            if (!type.IsGenericInstance)
                return type;

            var inst = type as GenericInstanceType;
            for (var i = 0; i < inst.GenericArguments.Count; i++)
            {
                if (!inst.GenericArguments[i].IsGenericParameter)
                    continue;

                var param = inst.GenericArguments[i] as GenericParameter;
                inst.GenericArguments[i] = genericParent.GenericArguments[param.Position];
            }

            return inst;
        }

        public static void PrepareDll(string dllPath, bool readWriteSymbols)
        {
            if (File.Exists(dllPath) == false)
            {
                throw new System.IO.FileNotFoundException(nameof(dllPath));
            }

            AssemblyDefinition assemblyDefinition = AssemblyDefinition.ReadAssembly(dllPath,
                new ReaderParameters { ReadSymbols = readWriteSymbols, AssemblyResolver = new CustomResolver() });

            if (assemblyDefinition.HasCustomAttributes == false)
            {
                return;
            }

            ModuleDefinition moduleDefinition = assemblyDefinition.MainModule;

            foreach (TypeDefinition variable in moduleDefinition.Types)
            {
                foreach (MethodDefinition methodDefinition in variable.Methods)
                {
                    Tuple<TypeReference,TypeReference, string> customGetAttribute = TryGetFieldGetAttribute(methodDefinition);
                    if (customGetAttribute != null)
                    {
                        FieldReference fieldReference = new FieldReference(customGetAttribute.Item3, moduleDefinition.Import(customGetAttribute.Item2), moduleDefinition.Import(customGetAttribute.Item1));
                        methodDefinition.Body = new Mono.Cecil.Cil.MethodBody(methodDefinition);

                        ILProcessor ilProcessor = methodDefinition.Body.GetILProcessor();
                        ilProcessor.Emit(OpCodes.Ldarg_0);
                        ilProcessor.Emit(OpCodes.Ldfld, moduleDefinition.Import(fieldReference));
                        ilProcessor.Emit(OpCodes.Ret);
                    }
                    else
                    {
                        Tuple<TypeReference,TypeReference, string> customSetAttribute = TryGetFieldSetAttribute(methodDefinition);
                        if (customSetAttribute != null)
                        {
                            FieldReference fieldReference = new FieldReference(customSetAttribute.Item3, moduleDefinition.Import(customSetAttribute.Item2), moduleDefinition.Import(customSetAttribute.Item1));
                            methodDefinition.Body = new Mono.Cecil.Cil.MethodBody(methodDefinition);

                            ILProcessor ilProcessor = methodDefinition.Body.GetILProcessor();
                            ilProcessor.Emit(OpCodes.Ldarg_0);
                            ilProcessor.Emit(OpCodes.Ldarg_1);
                            ilProcessor.Emit(OpCodes.Stfld, moduleDefinition.Import(fieldReference));
                            ilProcessor.Emit(OpCodes.Ret);
                        }
                    }

                }
            }

            assemblyDefinition.Write(dllPath, new WriterParameters() { WriteSymbols = readWriteSymbols });
            Debug.Log(dllPath + "  was prepared for serialization.");
        }

        /// <summary>
        /// This class will be created in a sparate domain
        /// </summary>
        public class ProxyObject : MarshalByRefObject
        {
            private List<string> _names = new List<string>();
            private List<string> _paths = new List<string>();

            public void LoadAssembly(string assemblyPath)
            {
                AppDomain.CurrentDomain.Load(AssemblyName.GetAssemblyName(assemblyPath));
            }

            public ProxyObject()
            {
                AppDomain.CurrentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
                AppDomain.CurrentDomain.TypeResolve += CurrentDomain_TypeResolve;
                AppDomain.CurrentDomain.ResourceResolve += CurrentDomain_ResourceResolve;
                AppDomain.CurrentDomain.ReflectionOnlyAssemblyResolve += CurrentDomain_ReflectionOnlyAssemblyResolve;
            }

            public void Setup(List<string> names, List<string> paths)
            {
                _names = names;
                _paths = paths;
            }

            public void Generate(ModulesContainer container, string fileName, string className)
            {
                container.GenerateCsFile(fileName, className);
            }

            public Assembly CurrentDomain_ReflectionOnlyAssemblyResolve(object sender, ResolveEventArgs args)
            {
                return TryToResolve(args);
            }

            private Assembly TryToResolve(ResolveEventArgs args)
            {
                int index = _names.IndexOf(args.Name);

                if (index != -1)
                {
                    return Assembly.LoadFile(_paths[index]);
                }

                return null;
            }

            public Assembly CurrentDomain_ResourceResolve(object sender, ResolveEventArgs args)
            {
                return TryToResolve(args);
            }

            public Assembly CurrentDomain_TypeResolve(object sender, ResolveEventArgs args)
            {
                return TryToResolve(args);
            }

            public Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
            {
                return TryToResolve(args);
            }
        }

        private static void GenerateSerialization(ModulesContainer modulesContainer, string fileName, string className)
        {
            AppDomain domain = AppDomain.CreateDomain("TempDomain", null, new AppDomainSetup()
            {
                ShadowCopyFiles = "false",
            });

            try
            {

                Type type = typeof(ProxyObject);
                ProxyObject proxyObjectInstance = (ProxyObject)domain.CreateInstanceFromAndUnwrap(type.Assembly.Location, type.FullName);

                Assembly[] allAssemblies = AppDomain.CurrentDomain.GetAssemblies();

                List<string> pathsList = new List<string>();
                List<string> namesList = new List<string>();

                foreach (Assembly assembly in allAssemblies)
                {
                    if (assembly.IsDynamic)
                    {
                        continue;
                    }

                    string dest = Path.Combine(Path.GetTempPath(), Path.GetFileName(assembly.Location));

                    if (File.Exists(dest))
                    {
                        File.SetAttributes(dest, FileAttributes.Normal);
                    }
                    File.Copy(assembly.Location, dest, true);

                    pathsList.Add(dest);
                    namesList.Add(assembly.FullName);
                }

                proxyObjectInstance.Setup(namesList, pathsList);

                foreach (var path in pathsList)
                {
                    proxyObjectInstance.LoadAssembly(path);
                }

                proxyObjectInstance.Generate(modulesContainer, fileName, className);

            }
            catch (Exception e)
            {
                Debug.Log(e);
            }
            AppDomain.Unload(domain);

        }

        public void OnPreprocessBuild(BuildReport report)
        {
            EditorUtility.DisplayProgressBar("Compiling", "Force compile for target platform", 0);

            CompilationInterfaceWrapper.DirtyAllScripts();//mark all scripts as dirty, this will make shure the call to CompileScripts will be succesfull
            if (CompilationInterfaceWrapper.CompileScripts(CompilationInterfaceWrapper.EditorScriptCompilationOptions.BuildingEmpty, EditorUserBuildSettings.selectedBuildTargetGroup, EditorUserBuildSettings.activeBuildTarget))//ask unity to recompile the scripts
            {
                CompilationInterfaceWrapper.DirtyAllScripts();//mark all scripts as dirty, we need this in case the user cancels the build(if he does he won't have the wrong platform dlls cuz unity will recompile automaticaly)

                CompilationInterfaceWrapper.CompileStatus compileStatus = CompilationInterfaceWrapper.CompileStatus.Compiling;

                while (compileStatus == CompilationInterfaceWrapper.CompileStatus.Compiling)//wait until unity recompiles
                {
                    Thread.Sleep(25);
                    compileStatus = CompilationInterfaceWrapper.PollCompilation();
                }

                if (compileStatus == CompilationInterfaceWrapper.CompileStatus.CompilationComplete)//if no errors were found
                {
                    string outputDirectory = Path.Combine("Assets", "Serializers");
                    string outputFile = Path.Combine(outputDirectory, EditorUserBuildSettings.activeBuildTarget + "Serializers.cs");

                    if (Directory.Exists(outputDirectory) == false)
                    {
                        Directory.CreateDirectory(outputDirectory);
                    }
                    else
                    {
                        if (File.Exists(outputFile))
                        {
                            File.Delete(outputFile);
                        }
                    }

                    EditorUtility.DisplayProgressBar("Generating", "Generating serialization for target platform", 0);

                    GenerateSerialization(ModulesContainer.LoadDefault(), outputFile, EditorUserBuildSettings.activeBuildTarget.ToString());

                    if (File.Exists(outputFile) == true)
                    {
                        AssetDatabase.ImportAsset(outputFile);
                        EditorApplication.update += CompletedBuild;
                        CompilationPipeline.assemblyCompilationFinished += CompilationPipeline_assemblyCompilationFinished;
                    }
                    else
                    {
                        Debug.Log("Generation failed");
                    }

                }
            }

            EditorUtility.ClearProgressBar();
        }

        private void CompilationPipeline_assemblyCompilationFinished(string arg1, CompilerMessage[] arg2)
        {
            PrepareDll(arg1, true);
        }

        public void OnPostprocessBuild(BuildReport report)
        {
            CompletedBuild();
        }


        public void CompletedBuild()
        {
            EditorApplication.update -= CompletedBuild;//somethimes the build fails and on post process is not called ....
            CompilationPipeline.assemblyCompilationFinished -= CompilationPipeline_assemblyCompilationFinished;

            string outputFile = Path.Combine("Assets", "Serializers", EditorUserBuildSettings.activeBuildTarget + "Serializers.cs");

            if (File.Exists(outputFile) == true)
            {
                AssetDatabase.DeleteAsset(outputFile);
            }
        }

    }
}

This error means the method Get_DataHolderBase__System_Boolean_values in the class FastSerializer.Autogenerated.SerializerMethodsAndroid is referencing a field named values in the type Framework.Extensions.DataHolderBase.

Do you know where the type Framework.Extensions.DataHolderBase should be defined? It seems to be in an assembly this is not available to the Unity build process.

First of all sorry for the late response, and to answer to you’r question the DataHolderBase class is contained in a assembly definition marked with any platform (SerializerMethodsAndroid is a autogenerated file)

namespace Framework.Extensions
{

    /// <summary>
    /// Holds, saves and returns values. This is only the Base of DataHolders.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    public abstract class DataHolderBase<T> : MonoBehaviour
    {
        /// <summary>
        /// The saved value list.
        /// </summary>
        [SerializeField] protected List<T> values;
    }
}

P.S. The compilation works when not calling PrepareDll wich modifies the assemblies with cecil, the thing is that the modified assemblies look valid in ILSpy or dotPeek

It is probably worth using Peverify (Peverify.exe (PEVerify Tool) - .NET Framework | Microsoft Learn) on the assembly produced by the PrepareDll step. Peverify can let you know if there is anything seriously wrong with the modified assembly. Peverify doesn’t catch all invalid IL code, but it catches a good bit.

My suspicion is that the IL code in the assembly after PrepareDll is somehow invalid.

Peverify told me it cannot resolve the token for the field, and then i looked into my code and noticed that when i create a FieldReference cecil will fail to resolve the field (generic instance field), i am still searching for how to get that ‘good’ FieldReference for the generic instance field :frowning: any suggestion would be helpfull.

I think you will need to get the FieldReference from the generic instance type. You may be able to create it, but I’m not sure.

Ok i have finally found the problem, to find it i did a simple case and compared the il and i realised i missed something here is the test app il : and the generated il the difference is the emitted il was passing a certain type but instead it should have been of a generic type, just passed the generic type to the fieldType in the FieldReference ctor and worked :slight_smile:

1 Like