[SOLVED] Trying to write jobified texture generation code, getting a weird granularized result.

So I was trying to jobify my map generation code, I figured it was simple enough, but my result is coming back kinda right but like the pixels are going into not exactly the right places they’re supposed to go to.

Left side of the image is what my regular code produces, right what the jobified code produced:

One thing that I wasn’t so sure how to accomplish and might be the center of my problem is that I didn’t know how to connect the position of the world coordinates to the NativeArray of colors, so I created a NativeArray and did a nested for loop with x and y to input the coordinates into the nativearray, I figured that if both indexes are the same in the colors native array and the coordinates native array it should work. But I guess I’m wrong? How should I approach this?

If anyone has any ideas I’d be happy to hear them, not sure where to go from here ;( thanks

The job code:

public struct MapTexturePreviewJob : IJobParallelFor
        {
            public NativeArray<float2> coordinates;
            public NativeArray<Color> colors;

            public void Execute(int i)
            {
                var coords = coordinates[i];
                float xWorld = coords.x;
                float zWorld = coords.y;
                colors[i] = TerrainMaker.FindColor(xWorld, zWorld, true,1));
            }
        }


public static MapTexturePreviewJob CreateMapTexturePreviewJob(int width, int height, int midPosX,int midPosZ, float zoomOut)
        {
            var _coords = new NativeArray<float2>(width * height, Allocator.TempJob);
            var _cols = new NativeArray<Color>(width * height, Allocator.TempJob);

            float xOffset = ((midPosX) - (width * 0.5f));
            float yOffset = ((midPosZ) - (height * 0.5f));
            for (int y = 0; y < height; y++)
            {
                for (int x = 0; x < width; x++)
                {
                    float xWorld = (xOffset + x) * zoomOut;
                    float zWorld = (yOffset + y) * zoomOut;
                    _coords[x + y * width] = new float2(xWorld, zWorld);
                }
            }

            MapTexturePreviewJob m_PreviewMapJob = new MapTexturePreviewJob()
            {
                coordinates = _coords,
                colors = _cols
            };

            return m_PreviewMapJob;
        }

Without the job essentially the same thing, but produces correct result:

var _coords = new NativeArray<float2>(width * height, Allocator.TempJob);
            var _cols = new NativeArray<Color>(width * height, Allocator.TempJob);
           
            gen = TerraSettings.GetGeneratorForPosition(midPosX,midPosZ);
           
            float xOffset = ((midPosX) - (width * 0.5f));
            float yOffset = ((midPosZ) - (height * 0.5f));
            for (int y = 0; y < height; y++)
            {
                for (int x = 0; x < width; x++)
                {
                    float xWorld = (xOffset + x) * zoomOut;
                    float zWorld = (yOffset + y) * zoomOut;
                    _coords[x + y * width] = new float2(xWorld, zWorld);
                }
            }
           
            for (int i = 0; i < _coords.Length; i++)
            {
                var coords = _coords[i];
                float xWorld = coords.x;
                float zWorld = coords.y;

                colors[i] = TerrainMaker.FindColor(xWorld, zWorld, true,1));
            }
           
             var res = new Texture2D(width, height, format, false);
             Color[] colors = new Color[width * height];
             _cols.CopyTo(colors);
             res.SetPixels(colors, 0);
             res.Apply();
           
             _coords.Dispose();
             _cols.Dispose();

Where is your schedule for IJobParallelFor along with passed in properties, including size of the job?

Forgot about it :hushed: here it is.

Other interesting things I found out trying to fix it:

Every time I run it it gives me a slightly different result. The math behind it should be deterministic, so It sounds to me like it’s placing the pixels in the wrong order in the array.

I tried changing from a parallel job to a regular IJob instead and it had the weird behavior the first few times I ran but after 5th time or so it worked perfectly and didn’t stop working afterwards. No idea why.

using System;
using System.Collections;
using System.Collections.Generic;
using Sirenix.OdinInspector;
using Terra.CoherentNoise.Texturing;
using Unity.Jobs;
using UnityEngine;

public class PreviewMapJobTest : MonoBehaviour
{
    [PreviewField(128),HideLabel]
    public Texture2D previewDisplayMap;
    public Transform debugTransform;
    public float DisplayMapZoom;

    public Action endedCoroutine;

    public bool scheduleJob = false;
    public bool hasScheduledJob = false;
    private JobHandle jobHandle;
    private TextureMaker.MapTexturePreviewJob previewJob;

    [Range(16,1024)]
    public int size = 32;
    private bool hasAction = false;
    
    public void Update()
    {
        if (scheduleJob)
        {
            scheduleJob = false;
            hasAction = endedCoroutine != null;
            previewJob = TextureMaker.CreateMapTexturePreviewJob(size, size, debugTransform.position.x, debugTransform.position.z, DisplayMapZoom);
            jobHandle = previewJob.Schedule(previewJob.colors.Length, 64);
            hasScheduledJob = true;
        }
    }
    

    public void LateUpdate()
    {
        if (hasScheduledJob)
        {
            hasScheduledJob = false;
            jobHandle.Complete();
        
            previewDisplayMap = new Texture2D(size, size, TextureFormat.RGB24, false);
            Color[] colorsArray = new Color[previewJob.colors.Length];
            previewJob.colors.CopyTo(colorsArray);
            previewDisplayMap.SetPixels(colorsArray, 0);
            previewDisplayMap.Apply();
            
            if(hasAction)
                endedCoroutine.Invoke();
            
            previewJob.colors.Dispose();
            previewJob.coordinates.Dispose();
        }
    }
}

Don’t know if that going to help in this case, but try add dependency jobhandle to schedule as well. Keep your job in chain. I would also move stuff out of late update to update after job complete call. At least until you resolve the issue.

Thanks for taking the time to help me with this :slight_smile: really appreciated.

I’m not sure what add dependency jobhandle means, I tried searching but it sounds like something to do when I have 2 jobs going on in a row, in this case I only have the one, so not sure what to do with that.

I moved the stuff to be all in the update, didn’t work. I also tried only having one native array by using a float3 array instead of a float2 and writing the y result of the noise math to the .z value of the native array. Also same result.

Now that I have a proper position for my map on the native array my next test is going to be placing objects on each spot on my native array and seeing if it shows my map correctly in 3d space. If it does I guess it means that somehow my indices in the array got out of order. If not… I guess I have a different problem?

-Edit-
I have a different problem, the Y values that I’m getting are wrong and not deterministic… Weird.

Updated code:

public static MapTexturePreviewJob CreateMapTexturePreviewJob(int width, int height, float midPosX,float midPosZ, float zoomOut)
        {
            var _coords = new NativeArray<float3>(width * height, Allocator.TempJob);

            float xOffset = ((midPosX) - (width * 0.5f));
            float yOffset = ((midPosZ) - (height * 0.5f));
            for (int y = 0; y < height; y++)
            {
                for (int x = 0; x < width; x++)
                {
                    float xWorld = (xOffset + x) * zoomOut;
                    float zWorld = (yOffset + y) * zoomOut;
                    _coords[x + y * width] = new float3(xWorld, zWorld,0);
                }
            }

            MapTexturePreviewJob m_PreviewMapJob = new MapTexturePreviewJob() {
                coordinates = _coords
            };

            return m_PreviewMapJob;
        }


public struct MapTexturePreviewJob : IJobParallelFor
{
   public NativeArray<float3> coordinates;

   public void Execute(int i)
   {
      var coords = coordinates[i];
      coords.z = TerrainMaker.FindColor(coordinates[i].x, coordinates[i].y, true,1);
      coordinates[i] = coords;
   }
}

Test Monobehavior:

public class PreviewMapJobTest : MonoBehaviour
{
    [PreviewField(128),HideLabel]
    public Texture2D previewDisplayMap;
    public Transform debugTransform;
    public float DisplayMapZoom;

    public bool scheduleJob = false;

    private JobHandle jobHandle;
    private TextureMaker.MapTexturePreviewJob previewJob;
    Gradient gradientEval;
    [Range(16,1024)]
    public int size = 32;
   
    public void Update()
    {
        if (scheduleJob)
        {
            scheduleJob = false;

            previewJob = TextureMaker.CreateMapTexturePreviewJob(size, size, debugTransform.position.x, debugTransform.position.z, DisplayMapZoom);

            jobHandle = previewJob.Schedule(previewJob.coordinates.Length, 64);

            jobHandle.Complete();
       
            previewDisplayMap = new Texture2D(size, size, TextureFormat.RGB24, false);
            Color[] colorsArray = new Color[previewJob.coordinates.Length];

            for (int i = 0; i < colorsArray.Length; i++)
            {
                colorsArray[i] = gradientEval.Evaluate(previewJob.coordinates[i].z);
            }

            previewDisplayMap.SetPixels(colorsArray, 0);
            previewDisplayMap.Apply();
           
            previewJob.coordinates.Dispose();
        }
    }
   
}

I made the coordinates returned be displayed with DrawGizmos in 3d space in the world and that should give me a perfect 3D representation of my map even if the indices got out of order, but that’s not what’s happening.

I thought it was somehow getting indices out of order but it seems that it gets the wrong Y from my Noise function… Not super wrong cause it’s still keeping the island shape kinda there but wrong enough that it generates the artifacts, weird.

-Edit-

Definitely looks like a problem with my Noise generation code somehow, since when I replace it with a Mathf.PerlinNoise it works :confused: now I just need to figure out why my noise generator is returning a different value each time if it’s run in a ParallelJob but not when it’s run in main thread.

-Edit -

Mystery solved :confused: I was making a new random based on a seed from within each request to get a Y from the noise… It should always return the same since it’s based on a seed but it seems to work differently in a multithreaded job.

No probs.

If you having two or more jobs, which are depending on each other, they should be chained in dependency / jobHandle, so they don’t run in parallel, or different order. Also avoiding potential race conditions.

So for example
jobHandleA = MyParallelJobA
{

}.Schedule ( lengthA, batchSizeA, initialJobHandle )

jobHandleB = MyParallelJobB
{

}.Schedule ( lengthB, batchSizeB, jobHandleA )

Dependency, instead jobHandle is used, in case of using
class MySystem : SystemBase
{
}
Is just naming. But are the same thing JobHandle.

Regarding random and multi threading. This may be a little trap, for unaware.

When you pas random into a job, specially with parallel execution, you copy initial state of random into each job.
So if you have 1000 elements, spread over 10 jobs in parallel threading, that is ideally 100 elements per job.
But each job at initial state, takes copy of the random value, that you pass in. So multiple parallel jobs has exact same random sequence.

To mitigate that, you either initialize your ranom inside each job, based on some semi random value, i.e. entity index, job index and initial state as example. Giving enough impression for randomness.
Or output random values into a NativeArray before actual job and pass these random values into the threaded job. You can generate these random NativeArray values, using single threaded job. If that makes sense?

I made(stole the idea from someone here) this utility function a while ago so this is a good example of the above:

/// <summary>
/// Get NativeArray containing a Unity.Mathematics.Random for each thread.
/// </summary>
public static NativeArray<Random> RNGPerThread() {
    var randoms = new NativeArray<Random>(
        JobsUtility.MaxJobThreadCount, Allocator.TempJob, NativeArrayOptions.UninitializedMemory);
 
    var r = (uint)UnityEngine.Random.Range(int.MinValue, int.MaxValue);
    for(int i = 0; i < JobsUtility.MaxJobThreadCount; i++) {
        randoms[i] = new Random(seed:(r == 0 ? r + 1 : r));
    }
    return randoms;
}

Then to use this in a job you need to get the thread index, use the rng at that index, then write it to the same thread index when you’re done.

Turns out random wasn’t the only problem. I still had some problems after fixing the random.

Ended up finding out those problems were due to sampling animation curves within the job, solved it by using the Fast Blob Curve from this thread: A Fast BlobCurve

Now it’s all working perfectly fine :slight_smile: thanks everyone