[SOLVED!] Windows: Transparent window with opaque contents (LWA_COLORKEY)?

Moved from the Scripting forums as this seemed the better, proper place for such a question:

In short: How would I alter the example code below to use LWA_COLORKEY to achieve a transparent window with opaque contents, assuming that my Camera is set to clear a Solid Color (in this case, Green). Please see my second post, below, for example screenshots.

After viewing the information at http://forum.unity3d.com/threads/windows-api-calls.127719/ & Viewing desktop in-scene - Questions & Answers - Unity Discussions, I have managed to produce a semi-transparent window with semi-transparent contents. While this is certainly a step in the right direction towards what I’m trying to achieve, I was curious if any of you might suggest how to make the window itself transparent while keeping it’s contents opaque (preferably 100% visible)?

My tests with LWA_ALPHA are working well for making the window semi-transparent, but the content’s themselves are also semi-transparent as LWA_ALPHA affects the entire window. Trying LWA_COLORKEY, which should act more like chromakeying in video, has so far failed utterly and the window remains fully opaque. I’m fairly new to coding, and know little at all about MSDN and the Windows API, but I really thought that LWA_COLORKEY would be the way to go, which has made the evening’s repeated failures quite frustrating.

However it might be achieved, I’d ultimately like to make the main window 100% transparent whilst keeping the contents of the window (gameobjects, GUI elements, etc.) 100% visible. Any advice would be greatly appreciated. I’ve included richyrich’s altered code & his usage instructions below for your consideration. Thanks!!

//Note: Inspired by and uses some code found here: http://forum.unity3d.com/threads/windows-api-calls.127719/

using UnityEngine;
using System.Collections;
using System;
using System.Runtime.InteropServices; // Pro and Free!!!

//WARNING!! Running this code inside Unity when not in a build will make the entire development environment transparent
//Restarting Unity would however resolve

public class TransparentWindow : MonoBehaviour
{
   [DllImport("user32.dll", EntryPoint = "SetWindowLongA")]
   static extern int SetWindowLong(int hwnd, int nIndex, long dwNewLong);
   [DllImport("user32.dll")]
   static extern bool ShowWindowAsync(int hWnd, int nCmdShow);
   [DllImport("user32.dll", EntryPoint = "SetLayeredWindowAttributes")]
   static extern int SetLayeredWindowAttributes(int hwnd, int crKey, byte bAlpha, int dwFlags);
   [DllImport("user32.dll", EntryPoint = "GetActiveWindow")]
   private static extern int GetActiveWindow();
   [DllImport("user32.dll", EntryPoint = "GetWindowLong")]
   private static extern long GetWindowLong(int hwnd, int nIndex);
   [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
   private static extern int SetWindowPos(int hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);

   void Start()
   {
     int handle = GetActiveWindow();
     int fWidth = Screen.width;
     int fHeight = Screen.height;

     //Remove title bar
     long lCurStyle = GetWindowLong(handle, -16);  // GWL_STYLE=-16
     int a = 12582912;  //WS_CAPTION = 0x00C00000L
     int b = 1048576;  //WS_HSCROLL = 0x00100000L
     int c = 2097152;  //WS_VSCROLL = 0x00200000L
     int d = 524288;  //WS_SYSMENU = 0x00080000L
     int e = 16777216;  //WS_MAXIMIZE = 0x01000000L

     lCurStyle &= ~(a | b | c | d);
     lCurStyle &= e;
     SetWindowLong(handle, -16, lCurStyle);// GWL_STYLE=-16

     // Transparent windows with click through
     SetWindowLong(handle, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
     SetLayeredWindowAttributes(handle, 0, 51, 2);// Transparency=51=20%, LWA_ALPHA=2

     SetWindowPos(handle, 0, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
     ShowWindowAsync(handle, 3); //Forces window to show in case of unresponsive app  // SW_SHOWMAXIMIZED(3)
   }
} // end of file

richyrich’s instructions:

  • Add the above code to your project
  • Go to your camera and select it. In the Inspector, change the colors and the alpha to 0.
  • Build the (Windows) app - change the settings to windowed, then run

“You now have a full screen window (without title bar) that allows the user to select things behind it, while at the same time still displaying anything you’ve rendered on top :slight_smile: Note: this is a windows only solution.”

Update: Digging around a bit, it looks like switching from LWA_ALPHA to LWA_COLORKEY would do the trick if I supply a color for the transparency (much like chromakeying video). For that reason, I’ve set my Main Camera’s background to be lime green (RGB 0, 255, 0) and have swapped out
csharp* *SetLayeredWindowAttributes(handle, 0, 51, 2);* *
on line 47 for

csharp* *SetLayeredWindowAttributes(handle, 0x0000FF00, 0, 1); // handle = current window, 0x0000FF00 = full Green (RGB 0, 255, 0), Opacity = 0 (transparent - used with LWA_ALPHA), LWA_COLORKEY = 1* *

Unfortunately, however, the window remains completely opaque (and a hideous lime green, haha). I’ll keep fiddling with the values, but if anyone has any suggestions (or any idea why LWA_COLORKEY isn’t working for me), that would be fantastic.

Update 2: Thought I’d try UpdateLayeredWindow, but got the same results. I know this CAN be done, so I’m left to assume that I’m simply implementing it incorrectly… I’m at a loss. From everything I’ve read, this should be working just fine. On the plus side, I’ve killed a lot of time helping other people on the forums tonight, haha. Guess I’ll get some rest and attack it again later.

1 Like

Hi,

could you mock up a picture of what you’re trying to achieve? I’m not entirely sure, but it seems that you want your window to have different shape than a rectangle, so in order to achieve that, you want parts of the rectangle to be transparent?

Hey there! Thanks for responding :slight_smile: I’ll go you one better and show screenshots of what I’m actually doing (desktop icons intentionally blurred 'cause who cares what I have on my desktop, right?). I’m not looking for any irregular window shapes, just a fullscreen, transparent rectangle… Here is the closest to “working” that the idea has come, using LWA_ALPHA in the SetLayeredWindowAttributes code:


While the first image may look alright at first glance, the second illustrates how the window’s contents (the kitty) are also transparent. It’s not as noticeable, but the window itself is also semi-transparent (with a black background in these examples) and slightly darkens the desktop behind them.

What I’m aiming for is to have a completely clear window by setting my camera’s background color to a specific solid color and using LWA_COLORKEY to make just that color transparent, which would, ideally, leave my kitty 100% visible with no transparent parts, much like chromakeying video. Unfortunately, using LWA_COLORKEY and Green as my color, gives me the following results (see next image). My camera’s Solid Color is RGBA (0, 255, 0, 0) , or 0x0000FF00 as the Windows specific code would have it. I’m fairly certainly I have these values right & will kick myself viciously if it turns out to be a simple matter of incorrect color choice, haha :slight_smile: That said, I have tried several other colors, all with the same results.

From everything I’ve read, the green color should be being used as a transparency color, so it shouldn’t be rendered at all… Some threads I’ve read also suggested using UpdateLayeredWindow in place of SetLayeredWindowdAttributes, but my results were the same with either.

I have a feeling I’m simply implementing it incorrectly, but I’ve been unable to find any working examples for Unity, and the Windows examples I’ve found are, of course, full of other information that may or may not be relevant. This is my first attempt to make use of an external DLL or the Windows API, and so I’m at a loss as to just what I might be doing wrong. I have also posed my question over at CodeGurus.com, though that post hasn’t been approved yet and may not ever be. Since my case is Unity3D specific, I was really hoping that someone here might be able to point me in the right direction.

Update: For additional reference, here is the documentation on the SetLayeredWindowAttributes function on MSDN: https://msdn.microsoft.com/en-us/library/windows/desktop/ms633540(v=vs.85).aspx

I had what I’d hoped was a “Eurkea!” moment last night, but implementation is proving difficult. I’ve got a feeling that Windows itself really doesn’t notice or care what colors my Unity background is set to, and that it’s probably only interested in what Windows sees… If that is the case, I need to have Windows set the background color, not Unity. I found some example code on the MSDN forums, but I’m having trouble ferreting out just how to use it through Unity. I’ve floundered around a bit and have tried importing & making calls to CPaintDC, GetClientRect, and FillSolidRect, but I have no idea if I’m using any of the three correctly, and trying to make sense of the gobbledeygook that is the MSDN documentation makes my head spin. Most of what I’m doing now is guesswork, and I’m not at all confident that my guesses are anywhere near hitting the mark. I suppose I’ve little choice but to back-burner this project until I can learn Windows API programming inside and out, and then apply that knowledge to Unity. Since it’s taken me several years to get to the point where I’m even semi-confident in my C# coding, I doubt that it will be anytime soon. In the meantime, if anyone happens to have any ideas about where I should begin, I’m all ears :slight_smile: Thanks!

Indeed, layered window will not work - it works when you do the drawing through GDI - drawing through Direct3D ignores the setting.

Probably the easiest solution would be to write a custom image effect that would make pixels of certain color transparent, and then use DwmExtendFrameIntoClientArea function to actually make your window transparent.

Check this out:

Note though, this method will not work on Windows XP as they added this function in Windows Vista only. Even though I doubt you’re targeting such an ancient OS, but since Unity still supports it, I had to warn you :).

Fantastic! Thanks ever, ever so much :slight_smile: I’ll have a look shortly! I’m up to my elbows in fine tuning another project at the moment, but I’m excited to know that I may still be able to make this happen!

ETA: Oddly enough, I do still have an old XP box… I keep it around to run my ancient You Don’t Know Jack games (among others). Can’t say I’d bother trying to make a game for anything older than Vista though haha :stuck_out_tongue:

To be honest, I couldn’t resist and managed to implement it myself - it’s such a cool idea :). Here’s my example project:

https://1drv.ms/u/s!AnsdTLoc9uuSqA1jH49zE3BdN8Wv?e=I1Fxup

EDIT 2022-05-10: fixed link.

3 Likes

You, my friend, are a dozen kinds of way ² cool! :slight_smile: It looks terrific! I’ll be looking forward to poking through the script and shader code to glean what information I can, not to mention making use of it ASAP!!! I’ll be crediting you in the upcoming project as well, if you don’t mind, and assuming it all comes together. Thank you, thank you, thank you! \m/(>.<)\m/ If I had the means at the moment, I’d send a 12 pack your way, haha!

It works like a charm! Thank you again!! I’ve combined your implementation with a modification of the script above, using LWA_ALPHA with a transparency of 254 (just shy of solid) so that users can still interact with their desktop while the program runs. It’s absolutely beautiful! :stuck_out_tongue: I can’t wait to start teach this crazy kitty (not the mention the rest of my menagerie) to chase the mouse cursor and other fun things. I’ll probably also look into keeping the window “always on top” & maybe putting the icon in the systray, but those are nowhere near as important as the transparency was. You, sir, are a scholar and a gentleman. I salute you! :slight_smile: Have a terrific day (you’ve certainly made mine).

Glad to help :).

2 Likes

Final update :slight_smile: Here’s a quick “proof of concept” video showing the kitten in action. It’s incredibly minimalistic, but now that I have the window behaving as it should, I can get to work on blending animations properly and making it do more than beg for attention, haha. I was also able to figure out setting the window to be “always on top” this afternoon, so now I’m down to just (maybe) moving the window from the taskbar to the systray. Good times! :stuck_out_tongue:

1 Like

Awesome! I’m the OP for the threat that this one came from. Got it working with click-through and always on top.

using System;
using System.Runtime.InteropServices;
using UnityEngine;

public class TransparentWindow : MonoBehaviour
{
[SerializeField]
private Material m_Material;

private struct MARGINS
{
public int cxLeftWidth;
public int cxRightWidth;
public int cyTopHeight;
public int cyBottomHeight;
}

[DllImport(“user32.dll”)]
private static extern IntPtr GetActiveWindow();

[DllImport(“user32.dll”)]
private static extern int SetWindowLong(IntPtr hWnd, int nIndex, uint dwNewLong);

[DllImport(“user32.dll”)]
static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);

[DllImport(“user32.dll”, EntryPoint = “SetLayeredWindowAttributes”)]
static extern int SetLayeredWindowAttributes(IntPtr hwnd, int crKey, byte bAlpha, int dwFlags);

[DllImport(“user32.dll”, EntryPoint = “SetWindowPos”)]
private static extern int SetWindowPos(IntPtr hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);

[DllImport(“Dwmapi.dll”)]
private static extern uint DwmExtendFrameIntoClientArea(IntPtr hWnd, ref MARGINS margins);

const int GWL_STYLE = -16;
const uint WS_POPUP = 0x80000000;
const uint WS_VISIBLE = 0x10000000;
const int HWND_TOPMOST = -1;

void Start()
{
#if !UNITY_EDITOR // You really don’t want to enable this in the editor…

int fWidth = Screen.width;
int fHeight = Screen.height;
var margins = new MARGINS() { cxLeftWidth = -1 };
var hwnd = GetActiveWindow();

SetWindowLong(hwnd, GWL_STYLE, WS_POPUP | WS_VISIBLE);

// Transparent windows with click through
SetWindowLong(hwnd, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
SetLayeredWindowAttributes(hwnd, 0, 255, 2);// Transparency=51=20%, LWA_ALPHA=2
SetWindowPos(hwnd, HWND_TOPMOST, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
DwmExtendFrameIntoClientArea(hwnd, ref margins);

#endif
}

void OnRenderImage(RenderTexture from, RenderTexture to)
{
Graphics.Blit(from, to, m_Material);
}
}

5 Likes

Very cool! So glad you were able to get it working as well, and thanks for starting that original thread, it was a great starting point :slight_smile:

I’ve tried the code provided by @StephanM , set the color of camera to (0,0,0,0) and material in script to (255,255,255,255), and enable the windows frame at start. it works!!!
However, I’ve found that the color of the 3d objects is not quite right, comparing to the case in which the script is not used. Is it due to the material I set? What is the proper material (type and color) should I set to the script please?
Thank you and wait for your respond!

This share is not available at the moment. Could you please share it again? Thank you!

Hmm, link works from me from at least 2 PCs.

I’ve uploaded it here too, so maybe this works for you:

https://oc.unity3d.com/public.php?service=files&t=3d5386c7d69be60cbb6873853bf0d7b1

3 Likes

kyewong: The code I’ve been using is from the sample project Tautvydas posted, so it’s really your best bet.

Tautvydas: You’re listed in the credits of my latest application & I’d be happy to send you a link if you’d like to see it in action :slight_smile: Thanks again!

This link works. But the scene file is empty…I’m using 4.6.5. Is it because of the incompatible version?
Could you please pack it into an asset? Thank you@!

Probably, I think I made the project with Unity 5, can’t remember. The scene is almost empty anyway: there’s a cube with scripts attached to move and spin it, camera with script attached to make window transparent (and a material in scripts public field) and a directional light.

I’ve tested in Unity 4.6.5, it just works perfectly!
The shader is important. At first I set the material in the script to default diffuse, and found that the color is not quite right comparing to the case in which the transparent background is not set. After applying the shader, the color becomes correct.
Thank you very much guys!
BTW. I need to try Unity5.0 asap^_^

1 Like