Metal C++ graphics plugin - Unity returns nil for MTLCommandEncoder

Hi all,

We’re looking into a Metal version of our Unity app that uses a native C++ plugin that performs custom Metal rendering, but we’re blocked because Unity is returning a nil result for MTLCommandEncoder.

To render into Unity’s scene with Metal, it’s necessary to have an id representing the current rendering pass. This is provided by the CurrentCommandEncoder member of the IUnityGraphicsMetal interface here.

However, CurrentCommandEncoder returns nil at several points where we need to perform custom rendering. Specifically, using Camera.AddCommandBuffer() and CommandBuffer.IssuePluginEvent() to register a native callback gives a nil result for CurrentCommandEncoder when using either CameraEvent.BeforeForwardOpaque or CameraEvent.AfterForwardOpaque.

The official example here uses a coroutine with WaitForEndOfFrame() and then issues a plugin event to perform custom rendering. This gives a non-nil MTLCommandEncoder, but it is very limiting: it only allows rendering at the very end of a frame. This breaks correct rendering of transparent objects. To correctly use custom rendering supporting both opaque and transparent objects, it’s necessary to add the custom rendering commands to an earlier point in the frame, e.g. using CameraEvent.BeforeForwardOpaque and CameraEvent.BeforeForwardAlpha.

The approach we’re using here works correctly on two other platforms (D3D11 and Vulkan), but does not work for Metal because of the nil result.

Does anyone know of a solution to this? One possibility could be to create our own MTLCommandEncoder, but to do this we’d need access to the Metal texture objects for the framebuffer (color and depth buffer). I haven’t seen a way to access these from Unity.

Thanks!

Is this an MR (Mixed Reality: Bounded or Unbounded) app as opposed to a VR one, as the topic suggests? If so, you should know that we target RealityKit for MR, and RealityKit does not provide a way to hook into the Metal render pipeline in this way. Using PolySpatial, Unity does not manage the frame buffer at all; instead, it transfers the state of its scene graph to RealityKit, which performs the actual rendering. There’s no way to access the RealityKit frame buffer or depth buffer. It is possible, however, to do your own rendering to RenderTextures and use those in geometry in an MR scene.

Hi, thanks for the response. This is for a VR / fully immersive app, not MR, so Metal should be available.

1 Like

In that case, maybe @mtschoen can help you.

First of all, yeah, what you are trying to do indeed might be hit with internal metal magic - CameraEvent.BeforeForwardOpaque and CameraEvent.BeforeForwardAlpha are specifically “outside” of rendering, so we might not have encoder setup yet (or anymore). Can you bug report with smallish repro? We should definitely support this, but i would rather see this for myself to think of better way
For now i think we would need to do an extra check for “yeah, we have rendering set up, just didnt create encoder yet, so let’s create it now and return to the plugin”. But i am also worried about cases like “ahahaha, we dont know what are you talking about, the rendering is done you need to setup everything yourself” - we should support this case better too
But yeah, bug report please 8)

Thanks for the response! We can have a think through the effort needed to put together a small repro example. In the meantime, do you know if there are any other places where a valid command encoder will be present, other than after WaitForEndOfFrame()? Or is WaitForEndOfFrame() the only supported way to do Metal rendering at the moment?

i mean - it should be available almost everywhere - from the sound of it this bug is on us. For a quick check you might add some “rendering” in BeforeForwardOpaque before poking the plugin (like, for example, do a clear)

As for the sample - just “let’s do something trivial/simple” was picked. Again, it is intended that things “just work”

Forgot to reply to this earlier, sorry. I tested your idea of adding an extra step in BeforeForwardOpaque (specifically a CommandBuffer.DrawMesh() call) and that works! The Metal command encoder becomes available after adding the DrawMesh() call before calling the plugin event. Thanks for the tip!