I’m creating a webgl texture in javascript, and trying to get back the handle pointer for the texture, so that I can use it within Texture2D.CreateExternalTexture.
Unfortunately I can’t find any examples on this online!
This is the js code:
GetCanvasTexturePointer: function () {
var canvas = document.querySelector('#iframecanvas').contentWindow.document.querySelector('canvas');
var texture = null;
if (canvas != null)
{
var canvasgl=document.querySelector("#glcanvas");
canvasgl.width=256;
canvasgl.height=256;
const gl = canvasgl.getContext("webgl");
// Generate a handle on the GPU
texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
// Upload the image data to the GPU
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, canvas);
}
return texture;
}
and this is my C# code:
[DllImport("__Internal")]
private static extern int GetCanvasTexturePointer();
int texint = GetCanvasTexturePointer();
if (texint > 0)
{
IntPtr intPtr = new IntPtr(texint);
Texture2D texture = Texture2D.CreateExternalTexture(256, 256, TextureFormat.RGBA32, false, false, intPtr);
}
Unfortunately I don’t really know how to convert the js “texture” object (which is a type WebGlTexture), into an IntPtr for Unity to use.
The problem here is that WebGL creates a texture object, and on the Unity side it uses texture IDs, as used by desktop OpenGL. Emscripten normally manages the mapping of texture IDs to WebGL texture objects, which it stores in the GL.textures array.
What you might be able to do is generate the texture ID yourself by adding the texture to the GL.textures array. Something like (untested):
GetCanvasTexturePointer: function () {
var canvas = document.querySelector('#iframecanvas').contentWindow.document.querySelector('canvas');
var textureId = 0;
if (canvas != null)
{
var canvasgl=document.querySelector("#glcanvas");
canvasgl.width=256;
canvasgl.height=256;
const gl = canvasgl.getContext("webgl");
// Generate a handle on the GPU
var textureObj = gl.createTexture(); // This is a WebGL texture object
GL.textures.push(textureObj); // "Register" the WebGL texture with Emscripten
textureId = GL.textures.length - 1; // The texture ID is it's index in the textures array
gl.bindTexture(gl.TEXTURE_2D, textureObj);
// Upload the image data to the GPU
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, canvas);
}
return textureId;
}
[DllImport("__Internal")]
private static extern int GetCanvasTexturePointer();
int texint = GetCanvasTexturePointer();
if (texint > 0)
{
IntPtr intPtr = new IntPtr(texint);
Texture2D texture = Texture2D.CreateExternalTexture(256, 256, TextureFormat.RGBA32, false, false, intPtr);
}
Thanks for the reply… so that textureId does return a number, however it doesn’t seem to contain the right texture!
I have this extra code in C# running 30 times a second, to read the texture and apply it to my picture frame.
public IEnumerator GetCanvasImages()
{
string imagebase64;
while (1 == 1)
{
if (CanvasImage != null)
{
try
{
int texint = GetCanvasTexturePointer();
if (texint > 0)
{
PlaqueText.SetText(texint.ToString());
IntPtr intPtr = new IntPtr(texint);
Texture2D texture = Texture2D.CreateExternalTexture(256, 256, TextureFormat.RGBA32, false, false, intPtr);
CanvasImage.GetComponent<MeshRenderer>().material.SetTexture("_EmissionMap", texture);
CanvasImage.GetComponent<MeshRenderer>().material.mainTexture = texture;
}
}
catch (Exception e)
{
Debug.Log(e);
}
}
yield return new WaitForSeconds(1f/30f);
}
}
This is showing an image, but not the canvas image…
Below is a video of my webgl app, the top half shows the canvas I’m trying to read the image out of. When I click on the gallery pictureframe, the canvas starts using that new html code to render.
At that point the gallery pictureframe should in theory update in sync with the canvas image.
However as you can see it doesn’t!
One interesting thing to note, is if I move the camera from side to side the image does change, but to other random images in my scene… I assume this means that the pointer is pointing to the wrong image?
I’ve also output the pointer id to the text plaque below the picture frame, which is incrementing at 30 fps, which makes sense with my current code.
If anyone has any advice as to how to fix this, please let me know, thanks!
Also of note, my previous solution was to call canvas.toDataURL() and send back the whole canvas image string from JS to C# and apply that as texture. That does work, so I know my core code is functional, it’s just very slow as I’m doing it 30 times a second!
You might just be missing setting the wrap and filter parameters of the texture. Alternatively, you can create the texture from the C# side and pass its native ptr to JS. I just tested this and it worked.
public class GetCanvasTexture : MonoBehaviour
{
[DllImport("__Internal")]
private static extern void UpdateCanvasTexture(int id);
// Start is called before the first frame update
void Start()
{
var texture = new Texture2D(256, 256);
int textureId = (int)texture.GetNativeTexturePtr();
UpdateCanvasTexture(textureId);
GetComponent<Renderer>().material.SetTexture("_MainTex", texture);
}
}
mergeInto(LibraryManager.library, {
UpdateCanvasTexture: function(id) {
// Create and fill a canvas with a gradient
var ct_canvas = document.createElement("canvas");
ct_canvas.width = 256;
ct_canvas.height = 256;
var ctx = ct_canvas.getContext("2d");
var grd = ctx.createLinearGradient(0, 0, 200, 0);
grd.addColorStop(0, "red");
grd.addColorStop(1, "white");
ctx.fillStyle = grd;
ctx.fillRect(0, 0, 256, 256);
// Get the WebGL texture object from the Emscripten texture ID.
textureObj = GL.textures[id];
// GLctx is the webgl context of the Unity canvas
GLctx.bindTexture(GLctx.TEXTURE_2D, textureObj);
GLctx.texParameteri(GLctx.TEXTURE_2D, GLctx.TEXTURE_WRAP_S, GLctx.CLAMP_TO_EDGE);
GLctx.texParameteri(GLctx.TEXTURE_2D, GLctx.TEXTURE_WRAP_T, GLctx.CLAMP_TO_EDGE);
GLctx.texParameteri(GLctx.TEXTURE_2D, GLctx.TEXTURE_MIN_FILTER, GLctx.LINEAR);
// Upload the canvas image to the GPU texture.
GLctx.texSubImage2D(GLctx.TEXTURE_2D, 0, 0, 0, GLctx.RGBA, GLctx.UNSIGNED_BYTE, ct_canvas);
}
});
I got this working using your latest code. One other thing I did need to do was flip the Y scale to -1 on my objects, as the rendering of my textures appeared to be flipped coming from webgl back into unity.
If anyone’s interested in seeing the results, my NFT gallery is available online, click the feet icon to move, choose floor 4 for an example of rendering canvas into unity
Instead of scaling Y by -1, you can use GLctx.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true); prior to the texSubImage2D call, and set it back to false afterword.
Could I use this to show a highcharts.js canvas texture on a cube?
I tried to copy the canvas to the shared texture using drawImage but it didn’t work.
function copySourceCanvasToUnity(){
sourcecanvas=document.getElementById("highchartscanvas");
sourcectx=sourcecanvas.getContext("2d");
//I made global variables for ctx and ct_canvas in the code above
ctx.drawImage(sourcecanvas, 0, 0);
}
Compared to using texSubImage2D(GLctx.TEXTURE_2D, 0, 0, 0, GLctx.RGBA, GLctx.UNSIGNED_BYTE, ct_canvas), is there a more efficient way to pass the content of the canvas to c# texture? For example, passing an imageBitmap instead of a canvas.
Additionally, GL.textures[id] in my attempts can only be used in a .jslib file, and results in an ‘undefined’ error when used in a .js file.
Can anyone help?
