In most modern HMD/SDKs you set the camera’s clear flags to a solid color (like black) and alpha to 0 to view the passthrough feed after enabling it in the SDK’s api. However we would like to apply overlay filters to the camera feed to produce interesting effects in a live setting.
So far I have been unsuccessful, I have tried to use a render texture and a plane to get the feed on something the renderer sees but this doesn’t work as the camera only “sees” black. We are currently using the built-in pipeline, has anyone been able to apply a filter of any kind to passthrough feed?
I’m sure it has something to do with how the unity camera is rendering objects but I don’t have enough knowledge to be able to write a shader or something that will allow us to manipulate a passthrough feed.
at least for Meta this is very much a feature rather than a bug. developers should not get, by default, access to the camera’s. with the current integration there is no way to access the feed from your app. However, I’m wondering why they do not allow it after asking the android system for camera access, just like with the microphone. Then the user is aware he’s potentially vulnerable to being recorded.
I suspect the limitation might have been due to either performance boosts and/or privacy concerns. You don’t want images or audio of a users surroundings stored or uploaded somewhere without consent, I totally get it. On the other hand a webcam texture can be used on androids and allow developers to do the same nefarious activity.
We simply want to apply effects to the camera feed to make it more interesting, but the more I look into it the more I’m thinking it’s not going to be possible. This seems to also be the case with HTC and Pico headsets. PS, love your VR mirrors & reflections asset!
Was looking for doing the exact same with the “Portals for VR” asset, making it AR compatible… being able to do interesting effects to the real world texture would be awesome… but so far, no luck.
Been wrangling with this off and on for months. I am usually good at hacking around limitations, but so far, the only post effect I can get to work in passthrough on the Quest 3 is Vignette. I get the privacy issue but have mixed feelings about the limitations. Like one commentor mentioned, either allow consent from the user to use the video or give us tools to affect the passthrough video feed in more interesting ways that don’t allow recording or nefarious access. I vote the latter.
