We’re using SetPixels32 and the texture isn’t always updating. The first time it’s called, it works - but subsequent ones seem to fail.
If we close and reopen the App, we see the updated texture.
Is this a known issue? Is it related to MarkDirty for RenderTextures? There doesn’t seem to be an equivalent for Texture2D.
tex.SetPixels32(textureData);
tex.Apply();
The Apply function sets the texture dirty, which should notify us about the change. If you can send us a small repro project to see what you are doing we can take a look and see what might be up.
I believe the issue you are seeing has to do with how you are dealing with the material in combination with the texture. I don’t think just setting the texture as a shader property will work as intended as PolySpatial will never see that as being updated.
Here’s a modification of your code that does work:
using System.Collections;
using UnityEngine;
using Random = UnityEngine.Random;
public class textu : MonoBehaviour
{
Material baseUnityMaterial;
Material clonedUnityMaterial;
Texture2D baseTexture;
Texture2D clonedTexture;
bool modifyTextureCoroutineActive;
int xx = 0;
private Texture2D GenerateColorCheckerboardTexture()
{
Texture2D newTexture = new Texture2D(256, 256, TextureFormat.RGBA32, false);
var modifiedPixels = new Color[256 * 256];
var yMinColor = Random.ColorHSV(0, 0.25f, 0.2f, 1.0f, 0.2f, 1.0f);
var yMaxColor = Random.ColorHSV(0.25f, 0.5f, 0.2f, 1.0f, 0.2f, 1.0f);
var xMaxColor = Random.ColorHSV(0.5f, 1, 0.2f, 1.0f, 0.2f, 1.0f);
for (int pixelY = 0; pixelY < 256; ++pixelY)
{
float yAmount = pixelY / 256.0f;
var xMinColor = Color.Lerp(yMinColor, yMaxColor, yAmount);
for (int pixelX = 0; pixelX < 256; ++pixelX)
{
float xAmount = pixelX / 256.0f;
modifiedPixels[pixelY * 256 + pixelX] = Color.Lerp(xMinColor, xMaxColor, xAmount);
if (pixelX % 32 < 16 ^ pixelY % 32 < 16)
{
modifiedPixels[pixelY * 256 + pixelX] += new Color(0.5f, 0.5f, 0.5f);
}
}
}
newTexture.SetPixels(modifiedPixels);
newTexture.Apply();
return newTexture;
}
void Start()
{
if (clonedUnityMaterial == null)
{
if (TryGetComponent<MeshRenderer>(out var meshRenderer))
{
baseUnityMaterial = meshRenderer.sharedMaterial;
baseTexture = baseUnityMaterial.mainTexture as Texture2D;
if (baseTexture == null)
baseTexture = GenerateColorCheckerboardTexture();
clonedUnityMaterial = new Material(baseUnityMaterial);
clonedTexture = new Texture2D(baseTexture.width, baseTexture.height, TextureFormat.BGRA32, false);
clonedTexture.SetPixels(baseTexture.GetPixels());
clonedUnityMaterial.mainTexture = clonedTexture;
meshRenderer.sharedMaterial = clonedUnityMaterial;
}
}
if (!modifyTextureCoroutineActive)
{
if (TryGetComponent<MeshRenderer>(out var meshRenderer))
{
StartCoroutine(ModifyTexture());
}
modifyTextureCoroutineActive = true;
}
}
private IEnumerator ModifyTexture()
{
var waitForNextFrame = new WaitForEndOfFrame();
while (true)
{
if (!clonedTexture || !baseTexture)
break;
var pixels = baseTexture.GetPixels();
int index = 0;
for (int y = 0; y < clonedTexture.height; y++)
{
for (int x = 0; x < clonedTexture.width; x++)
{
bool connected = (x >= xx - 5 && x <= xx + 5);
pixels[index].r = connected ? (byte)255 : (byte) 0;
bool isPreview = false;
pixels[index].g = isPreview && !connected ? (byte)255 : (byte)0;
bool unpowered = false;
pixels[index].b = unpowered && !connected && !isPreview ? (byte)255 : (byte)0;
index++;
}
}
clonedTexture.SetPixels(pixels);
clonedTexture.Apply(true);
xx = (xx + 1) % clonedTexture.width;
yield return waitForNextFrame;
}
}
}
I actually think this might be a case of overloading RealityKit with texture data. I changed your repro case to update only every tenth frame, and it worked as expected in the simulator. It might help to use a smaller texture, a lower update rate, or to use a RenderTexture instead.
int step;
// Update is called once per frame
void Update()
{
if ((++step) % 10 == 0)
Refresh();
}
The sample is a bit exaggerated in how many and how much we update, in our project we have several visualisations that we switch between with each having their own texture, but they update when we first switch and then every update when the data has been marked as dirty (such as the player moving their “cursor” over something). one of them works fine but several others have this issue.
when you say about overloading realitykit with texture data, would other operations contribute towards that as well and affect textures, such as dynamic geometry? and is there anything else?
and when you say use a rendertexture instead, do you mean use a RT as the texture property and updating the texture then drawing it to the RT / draw the individual pixels on the RT? or reading back the pixels to the texture?
interestingly, if I recreate the Material after updating the texture then it appears to render as expected, in both our project and the sample.
the change being this at the end of the Refresh() method in the textu class
visualiseTexture.SetPixels32(textureData);
visualiseTexture.Apply();
xx = (xx + 1) % mapWidth;
Material.Destroy(material);
material = new Material(renderer.material);
material.SetTexture("_BaseMap", visualiseTexture);
renderer.sharedMaterial = material;
}
so I’m curious to what is meant by
is it the fact we’re using SetPixels32 instead of SetPixels? I noticed i don’t set the cloned material back onto the sharedMaterial, but I just tried that and it made no difference.
or are we meant to set the texture property everytime we change it’s data?
thanks
I’m not sure. It’s more a hunch rather than anything backed up by evidence. We haven’t seen this particular failure mode before, although we’ve seen things like reduced frame rates due to creating meshes/colliders every frame.
Yes; RenderTextures are better suited to textures that change every frame, because they use RealityKit’s DrawableQueue API. It’s enough of a difference that it’s likely to improve performance even if you have to use SetPixels/SetPixels32 and Graphics.Blit to copy a Texture2D into the RenderTexture (though updating the RenderTexture from the GPU, using a compute shader for instance, would be preferable).
That shouldn’t be necessary, AFAIK. The change to the texture should be picked up without having to re-set the material property for the texture.
The other thing that’s worth noting is that the ARGB32 format seems to be problematic, and is requiring conversion, which slows things down. Metal supports RGBA32 or BGRA32, but not ARGB32.
I tried the RenderTexture approach but I have the same issue. Yesterday I found that if I recreate the material then it would render with the new pixel data, so I tried doing the same but with RenderTexture being used and strangely it doesn’t
EDIT: My mistake, there were two places in code where I Blit to the RT, and I forgot to add a MarkDirty call to one of them. The RT works