Camera Feed on Background ✅ Mobile Camera Stream — Unity Asset: ⭐ Device Camera

[makaka-org]

Camera Feed on Background — Unity Asset that uses Back/Rear or Front/Face Camera on the player’s mobile device to display the Live Video Input on the Background of the Scene. Safety First.

It doesn’t use any Motion Sensor of a Mobile Device (Gyroscope or Accelerometer) to rotate the Game World like in AR Camera Lite or in AR + VR: MR Camera.

---

Features of Camera Feed

Bring the enchanting Power of Camera Feed into your amazing Game or App:

• Cross-platform Camera Feed: iOS, Android.
• Device Orientations: Portrait, Landscape.
• Smartphone Cameras: Rear/Back, Front/Face/Selfie.
• Instant Launch after granting the camera permission.
• Quick Testing in Unity Editor with Web Camera.

[makaka-org]

Use Cases of Camera Feed

With adding rotation of the Game Camera with the Motion Sensor of a Mobile Device (Gyroscope or Accelerometer), Camera Feed on Background can be used as a “Pseudo AR Camera” to display 2D or 3D objects as though they were in the real world.

This technique was implemented in AR Camera Lite).

AR Camera GYRO 📱🤳🏾 Augmented Reality for Unity — Unity Asset

---

With integration of Google Cardboard XR Plugin, the Camera Feed on Background can be used as a Mixed Reality (MR) Camera. This idea was implemented in AR + VR: Mixed Reality Camera.

Unity AR + VR: Mixed Reality (MR) with Google Cardboard XR Plugin — Unity Asset — Google VR

---

AR Shooter.

AR Shooter — Unity Asset 🎯 Augmented Reality for Unity 🎯 AR Survival Shooter 🎯 FPS Game Template

[makaka-org]

Use Cases [2]

AR Basketball).

AR Basketball GO 🏀 2.0: Unity & Augmented Reality — Unity Asset

[makaka-org]

Recently, I submitted a Bug Report to Unity related to the next issue:

• WebCamTexture Lags, Stutters After Device Orientation is Changed.

Today, Unity confirmed the bug, and you can vote for the resolving:
here on Unity Issue Tracker.

So, the Asset has a stable work if you use the only orientation in your game without changing it during the game.

Unity Asset Store: Camera Feed on Background

[makaka-org]

Changelog of Camera Feed on Background
1.3:

Features:

• Front Camera Support.

1.2:

Improvements:

• [Unity 2021.3.21]

Fixes:

• Zoom Effect of Camera Feed in Vertical/Portrait Mode.

1.1:

Features:

• Horizontal/Landscape Mode support.

[BSGDevelopment]

Does your asset actually rotate the video feed of the WebCamTexture or is it simply relying on rotating a game object that displays the feed? I am looking to take in the video from the phone camera and send it to a RenderTextrure that will be streamed using Dolby.IO. The phone camera comes in at 90 degrees rotated. I can rotate the object that has the RenDerTexture so that it shows up correctly in the app, but the stream still has the rotated video. I also have a need to make sure the video is portrait instead of landscape.

[makaka-org]

@BSGDevelopment

Does your asset actually rotate the video feed of the WebCamTexture or is it simply relying on rotating a game object that displays the feed? I am looking to take in the video from the phone camera and send it to a RenderTextrure that will be streamed using Dolby.IO. The phone camera comes in at 90 degrees rotated. I can rotate the object that has the RenDerTexture so that it shows up correctly in the app, but the stream still has the rotated video. I also have a need to make sure the video is portrait instead of landscape.

I have next workaround using RawImage for displaying Camera Stream with WebCamTexture:

        Rect uvRectForVideoVerticallyMirrored = new(1f, 0f, -1f, 1f);
        Rect uvRectForVideoNotVerticallyMirrored = new(0f, 0f, 1f, 1f);
        Vector3 currentLocalEulerAngles = Vector3.zero;
----
----
----
        if (webCamTexture && webCamTexture.width >= 100f)
        {
            currentCWNeeded = targetDevice.isFrontFacing
                ? webCamTexture.videoRotationAngle
                : -webCamTexture.videoRotationAngle;

            if (webCamTexture.videoVerticallyMirrored)
            {
                currentCWNeeded += 180f;
            }

            currentLocalEulerAngles.z = currentCWNeeded;
            rawImage.rectTransform.localEulerAngles = currentLocalEulerAngles;

            if ((webCamTexture.videoVerticallyMirrored
                && !targetDevice.isFrontFacing)
                ||
                (!webCamTexture.videoVerticallyMirrored
                && targetDevice.isFrontFacing))
            {
                rawImage.uvRect = uvRectForVideoVerticallyMirrored;
            }
            else
            {
                rawImage.uvRect = uvRectForVideoNotVerticallyMirrored;
            }
        }

[makaka-org]

Camera Feed on Background 2.0 — Update

Features:

• Button: Switch Camera (Back/Front):

• Unity Event OnCameraSwitched() is also exposed in Inspector.

Improvements:

• [Unity 2022.3.18]

P.S. I have a workaround for one Known Issue (Bug in Unity Engine) for which I submitted a bug report last year. WebCamTexture lagging can be detected on some mobile devices when orientation changing or camera switching.

Solution: use the only orientation in your game without changing it and without camera switching during the game. Vote On for resolving this bug on Unity Issue Tracker: the more votes it has, the faster the issue will be solved.

Here are just a few (not all) successful examples (Unity Assets) without this bug and with using the only orientation:

• Portrait Orientation: AR Shooter
• Landscape Orientation: Mixed Reality Camera

Sincerely yours,
Andrey Sirota,
Founder of Makaka Games

[makaka-org]

Some Questions from my Clients sent to the support
Camera Feed on Background

Q 1
This asset can be used to render camera feed onto unity RawImage through WebCamTexture Unity class?

A 1
Asset uses RawImage with WebCamTexture Unity class - the only mechanics provided.

Q 2
We can get “full screen” portrait mode camera feed with decent resolution, not zoomed in (image corresponds the image that you see with native camera), proper aspect ratio, properly oriented?

A 2

• Asset provides fullscreen portrait or landscape mode of camera stream.
• Resolution is set in the constructor such way:
• new WebCamTexture(deviceName, Screen.width, Screen.height, requestedFPS)*.
  It’s enough to get a good resolution and performance, you can change it as you want.
• Zoom issue was fixed in the version 1.2. Check changelog. So the video will cover all the background without cutting the extra space of stream, but it will be zoomed to cover black areas because the aspect ratio of mobile screen and camera stream are different.
• Video will have correct orientation and aspect ratio.

Q 3
This works on iOS?

A 3
Asset works on iOS and Android. Check complete docs here.

Q 4 - Q5
We can access WebCamTexture instance that plugin uses and use it in our part of the system. In other words this plugin does not hide WebCamTexture instance behind some kind of an api? Is this plugin open source in a sense that we can adjust plugin code if needed?

A 4 - A5
All Source Code is open. You can access the WebCamTexture outside via this method:
public WebCamTexture GetWebCamTexture()

[makaka-org]

A year ago, I noticed the strange behavior during the developing apps for iOS that uses WebCamTexture class by Unity: LOGS WERE NOT DISPLAYED IN XCODE in Both Modes: Development and Non-Development.

I thought that was a temporary issue, which will be resolved in the next releases of Unity: such small bugs often happen in Unity. Moreover, the workaround is simple: you can use Console in Unity Editor to connect to the iPhone remotely and see the log.

But a year later, the problem was not solved, and I sent a bug report to Unity Team about this error.
So Please Vote ON to solve the error faster: the more votes it has, the faster the issue will be solved.

P.S. Yes, you need to always send bug reports. Yes, it takes a time, but this is extremely important to support our favorite game engine and use it with a comfort.

[makaka-org]

I just got the First User Review
about my Camera Feed on Background