I can announce that my hi-quality ChromaKey system that is targeted at live production workflow (for film and TV).
It has some advanced features that can be found mostly only on high end hardware solutions, but it is pretty performant and works in realtime.
On top, it will have also some robust and stable body motion tracking abilities for usage with cinemachine, or with any advanced scripting. I’m still working on this body motion tracking system, and on other features dedicated for broadcast.
You can watch few of my livestreams here (I’m always doing livestreams, just to show it’s real life performance and problems):
In this one I’m talking about new tracking features.
Deckard Chroma Key Indie is made to solve needs of high quality chroma keying for Virtual Production purposes, Live events, prerendered videos, live streaming, AR/VR applications for broadcast.*
Deckard Chroma Key uses new paradigms in a field of chroma keying related to virtual production (TV, moviemaking and streaming).
Chromakeying is a pretty complex work, and Deckard Chroma Key manages to meet all needs of professional chromakeying by using some features not available even on industry standard hardware solutions.
Features:
-Chromakeying that can treat transparent objects.
-Chromakeying can use custom shaders and color correctors.
-Chromakeying supports shadows, SSR, volumetric lights, DOF and raytracing.
-high performance and stable person Tracking system that can be used with Unity Cinemachine or custom scripting for various effects.
-System for avoiding (or supporting) shadows on chroma keyed surfaces.
-Support for easy Garbage matte keying**.
Garbage matte is a process where you sample green screen with it’s imperfections and achieve a perfect key even without sampling colors.
-Supports inputs from a webcam drivers, video Input or Texture (for simplified usage with third party plugins).
-Uses video relighting tech that automatically generates normal maps for usage with virtual lights in Unity.
-Deckard Chroma keyer is supported by Deckard Render for superior chromakeying for film.
-Supports HDRP and Standard pipelines.
-It features simple but powerful system for stacking denoisers, sharpeners, color correctors and other creative effects (with included holographic screen and comics effect). Authoring of those effects can be pretty simple with Amplify Shader or Shader Graph.
Indie version has all features as an Enterprise version but is meant for single developers/artists that earn less than 100.000$/Year. With Indie version you get a forum support.
If you are user that doesn’t belong into this category, you should buy a Deckard Chroma Key Enterprise version, that gives you 4 hours of live training, consultancy and support via video meetings.
If you are a part of an educational institution, please contact me.
*due to it’s nature it isn’t intended or tested for mobile phone applications or low performance systems.
** Garbage matte can be applied only on static camera
Good chromakey requres good source footage. There are few guidelines hoiw to obtain good source images out of your camera.
-Choosing a right camera
Greenscreen requires a good camera and you will never get perfect results with a simple webcam. Webcams have a lot of noise, low dynamic range, do some image pre-processing and auto adjust their features and this can and will affect overall quality of your keying. Best way to obtain good input video is to use DSLR or dedicated Video Camera for this task. There are some pretty affordable solutions available, and even a cheap second hand DSLR will do a great job. I’m personally using a Blackmagic Pocket Cinema 4k camera. It is incredibly versatile budget camera that has some important features required for good keying: Manual control, high dynamic range and low latency. Unfortunately, even if it captures at 4k, it doesn’t output 4k HDMI signal (it can output fullHD resolution - that should be enough in most cases). Tip: when Using BMPCC4K set its profile to Extended Video - this will give you an optimal dynamic range for best keying.
-Capturing with video capture card
Your camera or DSLR probably doesn’t have a webcamera interface, so you will also need a video capture device that has HDMI (or SDI) input and can act as a webcam device. I’m using mostly Blackmagic Desing Intensity Pro 4k card (or Decklink) for this task, but there are many other available solutions on a market.
-Well lit greenscreen
Greenscreen should always be lit evenly as possible. Avoid having any wrinkles on your greenscren material. UniKeyer works pretty well even with unevenly lit screens, but this works mostly for cases when you are using a static camera. When you are using moving camera, well lit screen can be really important. Deckard UniKeyer is made to deal pretty well with all extreme cases and features many tools to do some good keying even with a problematic background.
Getting Started with UniKeyer
Once imported into your project you can add it to your scene by Right Clicking on hierarchy and going to Deckard UniKeyer menu and selecting Add ChromaKey Rig. This will add a special chromakey rig that contains all neccessary scripts and objects.
Next steps will be to configure your chromakeyer.
-Keying Profile
Here you can create or assign existing Chroma Keying profile. Setting a profile is somewhat similar to setting post processing profiles in Unity. But in this case, a “profile” is in a form of a material with it’s shader. Once you create or assign an existing profile, you can then edit material properties - keying properties. But before you start doing so, you will want to set up a source of your video stream.
-Source Type
-Webcam: it will use webcam device or capture card
-Render Texture: it will use a render texture - this is usefull when you are using third party video capture plugins like AV Pro Live,AV Pro Decklink or when you are using Video player with render texture as a target.
-Video: you should use this if you are using pre-recorded video file.
-Key Type:
We have two key types, one is a standard keying type where you input a color for keying out background(for moving camera), and another is a keying with a Clean Mate Source (when your camera doesn’t move).
Capture Clean mate
What is Clean mate source? It is a simple image of a background without any objects, for example: just a unevenly lit green screen. This type of keying can produce best results in a non perfecly lit environment. You can sample this image and save it as a texture in Unity by clicking on Capture Clean Mate.
Note: it will always save image with a same name: ultimatte.png If you are using more than one profile in your unity project, be sure to rename it after capture. This behaviour is here so that you can update clean mate image across many Unity scenes in one single run. But there might be some other cases when you are using multiple cameras, where you probably want to rename those sample mate images.
Custom Mask
Sometimes you probably need to mask some zones of your screen so here you can add a texture with masking zones. Black color is a masked color.
Setting a Key Profile (keying):
When using ost important values are Key Colors (there are two of them for better flexibility of keying if a), Threshold and ThresholdLow. This is where you are setting a range of color similarity to key out. Once you have those values perfectly set, you can tune other parameters. When keying a full figure, you want to have a full control on how you want to treat your keying. UniKeyer is capable of doing soft keys (where you want to preserve shadows and preserve real shadows or semi transparent objects. But sometimes you actually want to preserve smooth transition behaviour on upper part of a body (where your actor may use glasses, hold bottles or transparent products) but you want to eliminate any shadows that are casted on the ground. So here comes a need for separate Threshold parameters, one that works for the upper part of body, and one for lower. In this way you can tune differenlty both parts of an image.
Despill Chroma and Despill luma defines how the borders will behave and how a despilling (removage of green cast from a screen behaves). Despil Chroma should mostly be set to 1 but a despil Luma should be set on a base of your virtual background. White and bright backgrounds require positive values, dark baskgrounds darker values.
Border: defines how mush smoothing has to be applied to borders of a mask. This can help to have a natural defocused borders, and it can clean up borders.
Additional passes:
This is one of the interesting features of UniKeyer. Additional passes give you an ability to add multiple layers of chromaKeying (for other colors) or to add some special filters ie. It can be used for generating some effects like holographic effects, edge refinement shaders, beauty filters and so on. NOTE: additional apsses can have impact on performance as any image effects - they should be used with care and mostly for high quality applications.
Image Tracking:
One of the features of Deckard UniKeyer is it’s ability to track figures. It uses a simple enough tracking to be able to run in realtime without big performance overhead, while giving an incredibly stable figure tracking. It can track extents of a figure, head Top and shoulders position. This tracking data can then be easily used with Cinemachine for virtual camera guidance, with robotic systems for automated camera movement, for alerting system when a figure is aproaching borders of a camera capture area, for activating behaviours (a virtual touch screen), or for example, positioning feets on a floor level and generating correct contact feet points for placing shadows. This system is in developement, and there may be many future uses for it.
Hi, I think that I should finish it this week. Right now I’m in final phase of betatesting and polishing interface. Lost probably too much time into implementing new features, and for this reason I have decided to stop working on new features before first release.
I hope to be able to submit it to assetstore this weekend.
As for pricing, I will have two versions of plugin:
-INDIE that will cost 39us$ and will have all features, but will be targeted to single team developers that earn less than 100.000/ year, with suppurt only on unity forums.
-ENTERPRISE, targeted at broadcast, filmmaking, youtubers and companies earning more than 100.000/year. It’s licence should be 599us$ and will give support/consultancy of 4 hours via video conferencing.
Oliver, this is a fantastic component and I look forward to supporting it via Asset store purchase!. Per your comments, possibly something for a future release?? Its regarding Shadows.
I don’t know if its possible or not but I’m curious if there’s a way to provide the ChromaKey component a primary shadow casting light source, whereby a duplicate of the content of the ChromaKey (after key removal), could be rotated/aligned using an angle somewhere between the direction of the camera and a designated light source?) such that the shadow cast, is more full. i.e. Keep the ‘visible’ chromakey content facing the camera, but somehow have the ‘shadow’ generated using an invisible, but rotated normal or source of the same. I used the demo scene and a video with some cubes. The directional sun sees the video as thin, but I’m thinking, perhaps if the normal or texture was angled between the camera and sun, the shadow cast would appear more full. Just an idea, and again, don’t know if it makes sense or not… Overall though, seriously, a very nice component so far with excellent control over chroma keying, and enhancing output colors.
Out of interest, does your chroma key system use an unlit shader? I was interested in why you chose to use DMX to the real world to match lighting rather than in-game lighting? Was it an aesthetic choice or a technical one? Just because presumably there will be a slight lag with going out to DMX
Hi everybody, I have a good announcement for all of you interested into virtual production.
My Deckard Chromakey is available on Assetstore. It is a high quality chromakeyer made for virtual production needs, and is tightly integrated with Deckard Render.
Deckard Chroma Key Indie is made to solve needs of high quality chroma keying for Virtual Production purposes, Live events, prerendered videos, live streaming, AR/VR applications for broadcast.*
Deckard Chroma Key uses new paradigms in a field of chroma keying related to virtual production (TV, moviemaking and streaming).
Chromakeying is a pretty complex work, and Deckard Chroma Key manages to meet all needs of professional chromakeying by using some features not available even on industry standard hardware solutions.
Features:
-Chromakeying that can treat transparent objects.
-Chromakeying can use custom shaders and color correctors.
-Chromakeying supports shadows, SSR, volumetric lights, DOF and raytracing.
-high performance and stable person Tracking system that can be used with Unity Cinemachine or custom scripting for various effects.
-System for avoiding (or supporting) shadows on chroma keyed surfaces.
-Support for easy Garbage matte keying**.
Garbage matte is a process where you sample green screen with it’s imperfections and achieve a perfect key even without sampling colors.
-Supports inputs from a webcam drivers, video Input or Texture (for simplified usage with third party plugins).
-Uses video relighting tech that automatically generates normal maps for usage with virtual lights in Unity.
-Deckard Chroma keyer is supported by Deckard Render for superior chromakeying for film.
-Supports HDRP and Standard pipelines.
-It features simple but powerful system for stacking denoisers, sharpeners, color correctors and other creative effects (with included holographic screen and comics effect). Authoring of those effects can be pretty simple with Amplify Shader or Shader Graph.
Indie version has all features as an Enterprise version but is meant for single developers/artists that earn less than 100.000$/Year. With Indie version you get a forum support.
If you are user that doesn’t belong into this category, you should buy a Deckard Chroma Key Enterprise version, that gives you 4 hours of live training, consultancy and support via video meetings.
If you are a part of an educational institution, please contact me.
*due to it’s nature it isn’t intended or tested for mobile phone applications or low performance systems.
** Garbage matte can be applied only on static camera
Thanks for sharing this! Looking great from what I’ve delved into so far. Obviously there’s a lot in there!
I’m just having a couple of issues that I think might be related to the fact my project uses the new Unity input system. I think it might be breaking a couple of key areas.
Tracking. I’m using a portrait webcam video. It tracks my sides fine, but the shoulder and bottom trackers are just at the edge of the video, so it looks like I’m hovering in mid-air. I was wondering if this could be because of the GetKeyDown call in the Update function in debugSwitcher? Just wanted to check before I go tinkering with it.
Garbage keying just results in a totally transparent image. I’m not sure if this is related? Or maybe I’m just doing something (or more than one thing) wrong?
Despite not managing to get garbage keying (or the histogram) to work [I’m told the histogramMaterial hasn’t been assigned, but I can’t see anywhere to assign it], I’ve still managed to get a very clean image with a not-very-amazing green screen, so that’s already great!! Thanks!!!
I’m using Unity 2020.2.6.f1, HDRP 10.3.1, Mac OS 11.2.3
Ok, so I’ve gone back to your live stream for Unity where you use your keyer, and I think the garbage keyer wasn’t working because I wasn’t using it right, but there definitely seems to be an issue with head, shoulder and bottom trackers. They’re just not moving at all.
You probably have to tune in better keying, or change threshold values to meet your scene. It doesn’t have anything with input system (it is only used for showing extents). BTW, do you see extents of tracking?
