Reflective materials appear black in RenderToCubeMap 2019.2 standard pipeline

[Bitshuffler]

I’m using Camera.RenderToCubeMap and RenderTexture.ConvertToEquiRect to create equirectangular panoramas.

This works well except it doesn’t seem to take account of Reflection Probes with reflective materials.

Top image is in the editor, bottom image is from the CubeMap.

I thought there might be something to do with culling masks but I don’t think that’s the case. Is this known behaviour and if so is there a way around it?

Sample source that I’m using to capture the png below…

Hoping someone can help out. Many Thanks

             RenderTexture cubeMap = new RenderTexture(cubemapSize, cubemapSize, 24);
             cubeMap.dimension = UnityEngine.Rendering.TextureDimension.Cube;
             mainCamera.RenderToCubemap(cubeMap);
             RenderTexture equiRect = new RenderTexture(2 * cubemapSize, cubemapSize, 24);
             equiRect.dimension = UnityEngine.Rendering.TextureDimension.Tex2D;
             cubeMap.ConvertToEquirect(equiRect);
             var tex2 = toTexture2D(equiRect);

[AcidArrow]

RenderToCubeMap keeps breaking from time to time, I don’t know why, so we ended up doing it manually.

Here’s what we’re sort of doing (this won’t work, I copy pasted various parts, so there’s stuff missing. And the whole “saving to a cubemap, and then saving to a png” is not really needed, but it’s done because we do something else in between, but anyway, it should give you an idea. PS: you may need to flip the final image upside down).

int layers = 0;
for (int i = 0; i < LayersToIgnore.Length; ++i)    layers |= (1 << LayersToIgnore[i]);

goCamera.cullingMask = ~layers;
goCamera.targetTexture = rt;
Texture2D tex2 = new Texture2D(rt.width, rt.height, TextureFormat.RGBAFloat, false);
RenderTexture.active = rt;

go.transform.LookAt(go.transform.position + Vector3.up);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.PositiveY);

go.transform.LookAt(go.transform.position + Vector3.down);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.NegativeY);

go.transform.LookAt(go.transform.position + Vector3.right);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.PositiveX);

go.transform.LookAt(go.transform.position + Vector3.left);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.NegativeX);

go.transform.LookAt(go.transform.position + Vector3.forward);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.PositiveZ);

go.transform.LookAt(go.transform.position + Vector3.back);
goCamera.Render();
tex2.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);

sourceCubemap.SetPixels(tex2.GetPixels(), CubemapFace.NegativeZ);

sourceCubemap.Apply();

for (int i = 0; i < 6; ++i)
{
    tex.SetPixels(sourceCubemap.GetPixels((CubemapFace)i));

    for (int x = 0; x < tex.width; x++)
    {

        for (int y = 0; y < tex.height; y++)
        {
            Color theColor = tex.GetPixel(x, y);

            texFull.SetPixel(x + i * width, y, theColor);
        }
    }
}

string path = "Assets/Images/Cubemaps/" + filename + ".png";

File.WriteAllBytes(path, texFull.EncodeToPNG());

[Bitshuffler]

Thanks AcidArrow, much appreciated. I’m trying this approach.

The issue is that it looks like it’s only RenderTexture has the built in ConvertToEquiRetangular() function. and I can’t seem to make a Cubemap (ie the unity cubemap class) believe that it’s a RenderTexture.

(Which it isn’t of course, but it seems like a pain that ConvertToEquiRectangular is only on the cubemap render texture, but not on the cube map.)

Does anyone know a way to convert the Cubemap to an equirectangular texture without coding it myself?

[Bitshuffler]

p.s. AcidArrow, you may be able to simplify and speed up your output.

        void SaveCubeMap( Cubemap c, string fileName)
        {
            var texFull = new Texture2D(c.width * 6, c.width, TextureFormat.RGB24, false);
            for (int i = 0; i < 6; ++i)
            {
                texFull.SetPixels(i*c.width,0,c.width,c.width,c.GetPixels((CubemapFace)i));
            }
            File.WriteAllBytes(fileName, texFull.EncodeToPNG());
            Destroy(texFull);
        }

[Bitshuffler]

So I’ve used this library GitHub - Mapiarz/CubemapToEquirectangular: Cubemap to equirectangular format converters for Unity3D as a base.

I have a cpu based algorithm and it works but is seriously aliased and extremely slow.

I have implemented the shader based version which is faster but is bleached out for some reason. I think it’s something to do with linear vs gamma colour space? The App is linear and I thought I read somewhere that the shader wants gamma?

I also purchased Evereal’s SkyboxCapture however it just turned out to be based on the same github library and shader, and gives the same washed out result.

All of these are slower than the built in method because the built in method all works on the rendertexture and so there’s no copying the cubemap textures back and forth to the card.

Really hoping there’s just something that I missed setting in Camera.RenderToCubeMap that will include reflected surfaces.

[Bitshuffler]

From Remy_Unity

I might be wrong here, but I suspect that Camera.RendertoCubeMap has some safeguard to avoid recursive rendering, that would explain why the reflection probes are missing in the capture.
Like AcidArrow pointed it in the thread, I would suggest to render a camera in 6 directions, as it basically what the original command does.
You should be able to keep it fast if you keep all data on the GPU.
If you absolutely want to have a cube render texture, and option would be to use a 2D render texture on the camera, and for each direction, render it an then use Graphics.CopyTexture to copy the data from the 2D render texture to a specific face of the cube render texture.
This should allow to have all data on GPU, and then you’ll be able to call ConvertToEquirect.

This was all I needed. Thank you Remy

[AcidArrow]

@Bitshuffler

p.s. AcidArrow, you may be able to simplify and speed up your output.

Yeaaaah, in retrospect rereading that, I think I used to flip the pixels upside down (don’t quite remember why it was needed) when writing them to the png, that’s why it was written like that and it took you saying that for me to notice it’s not longer needed.

So thanks for the heads up, it is noticeably faster now.