Trying to achieve a Retro 2D Pixel effect

[wickedpopular]

Hello! I have been working on a shader to turn this

into this

To do so I know I have identified that I need the following things:

• The ability to recolor the image
• The ability to scale the screen to add a background border
• Per-pixel LCD grid lines
• Some way to create slightly darker foreground area
• filtering to blur the border between pixels, but not go over the grid or outside the pixel grid area
• shadow to define the boundry between the “foreground” and “background” layer

All while keeping the pixels perfectly square, with no warping. So far I’ve been able to achieve this:

With this Shader (some variables aren’t in use yet)

Shader "Custom/GameboyScreen"
{
    Properties
    {
        _MainTex("Gameboy Screen Render Texture", 2D) = "white" {}
        _ScaleFactor("Amount to reduce by", Range(0.0, 1.0)) = 0.25

        _ScreenResolutionH("Horizontal Screen Resolution", float) = 160
        _ScreenResolutionV("Vertical Screen Resolution", float) = 144

    }

        SubShader
        {
            Tags { "RenderType" = "Opaque" }
            LOD 100

            Pass
            {
                CGPROGRAM
                #pragma vertex vert
                #pragma fragment frag

                #include "UnityCG.cginc"

                struct appdata
                {
                    float4 vertex : POSITION;
                    float2 uv : TEXCOORD0;
                };

                struct v2f
                {
                    float2 uv : TEXCOORD0;
                    float4 vertex : SV_POSITION;
                };

                sampler2D _MainTex;
                float4 _MainTex_TexelSize;
                float4 _MainTex_ST;
                float _ScaleFactor;
                float _ScreenResolutionH;
                float _ScreenResolutionV;

                half4x4 _ColorMatrix;

                v2f vert(appdata v)
                {
                    v2f o;
                    o.vertex = UnityObjectToClipPos(v.vertex);
                    o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                    return o;
                }

                fixed4 frag(v2f i) : SV_Target
                {

                    //Screen Scaling
                    float2 scaleCenter = float2(0.5f, 0.5f);
                    i.uv = (i.uv - scaleCenter) * (1 + _ScaleFactor) + scaleCenter;


                    //Colorize
                    fixed red = tex2D(_MainTex, i.uv).r;
                    float4 colorized = _ColorMatrix[red * 3];


                    // Pixel Border
                    int x = _MainTex_TexelSize.z * i.uv.x;
                    int y = _MainTex_TexelSize.w * i.uv.y;

                    int xmod = x % 8;
                    int ymod = y % 8;

                    if (xmod == 0 || ymod == 0)
                    {
                        colorized += float4(.02, .02, .02, 0);
                    }
                   

                    return colorized;
                }
                ENDCG
            }
        }
}

I’ve also attached all my assets so you can see the setup. I was able to recolor the screen pretty easily. I can resize the texture, but I get artifacts around the border, and I don’t know if the way I’m doing it can reliably create pixel-perfect output. For the pixel grid, I was able to figure out how to create the lines at 8x8 tiles, but I’m not sure how to create the sub-pixel borders with my current setup.

The way I have it set up currently takes a render texture at the native resolution of the Gameboy, then applies that texture to a plane with the shader effects added. I think I’d have to do something to upscale the texture, but I’m not sure how to do that.

Does anyone have any tips for how I can proceed with this? I’m not sure where to go from here.

Thank you!

4996190–488051–GB Screen Test Project.unitypackage (24.1 KB)

[bgolus]

@wickedpopular

The ability to recolor the image

What you’re doing there is fine. As long as the input texture is using point sampling, the colors will remap properly.

@wickedpopular

The ability to scale the screen to add a background border

It seems like you’re already scaling the image, which is good. You just need to check in the shader if you’re outside of the 0.0 to 1.0 range of the UVs and switch to showing the background color.

@wickedpopular

Per-pixel LCD grid lines

You’re very close. The two main things to things about here is the TexelSize * UV is going to give you a value where each increment of 1 is a single pixel of the input texture. You want lines between each of those, so you want to multiply the x & y by some value (like 8) before doing the modulo (%) by the same value. That way you’re doing a grid between each pixel and not on every 8 pixels (which is what you’re doing now).

@wickedpopular

Some way to create slightly darker foreground area

You should be accomplishing this with your palette swap. Alternatively do this with just a foreground and background color, replicating how the actual GB worked where each pixel is darkening the background color (along with adding in it’s own color).

@wickedpopular

filtering to blur the border between pixels, but not go over the grid or outside the pixel grid area

I would suggest not trying to blur the pixels. That’s not actually what you want. You want to soften their edges. The solution to that would be to use a grid that has some amount of AA or softness to it rather than using the hard modulo and if statement based version you have now.

@wickedpopular

shadow to define the boundry between the “foreground” and “background” layer

Basically you have to replicate all of the code you do to make the grid, clamp the UV area, and sample the opacity of each pixel to produce the shadow. Just offset, and not as dark.

[bgolus]

I played with this a bit for fun and got this:

I’m not going to share the code for it, but here’s a couple of points:
I set the texture to use bilinear sampling, then quantize the UVs to be point sampled in the shader for the main sampling of the _MainTex.
float2 point_uv = (floor(i.uv * _MainTex_TexelSize.zw) + 0.5) * _MainTex_TexelSize.xy;

I then use slightly offset, but unquantized UVs to get the shadow. The result is a slightly blurry version of the original texture.

I’m only using three colors. The background, the pixel color, and a shadow color. I’m using the pixel color’s alpha to control the overall opacity of the pixels (how much it will go fully to the pixel color), as well as a second value to control the lowest opacity.

I’m getting a grid using a modified version of the example here:

Inigo Quilez

And I’m limiting the drawn area by a 0.0 - 1.0 UV range box either anti-aliased by the screen derivatives, or by the pixel size, depending on if I’m doing it for the pixels or the shadow.

So basically I create a shadow mask and a pixel mask, then lerping between the background color and the background multiplied the background by the shadow color using the shadow mask, and then lerp between that and the pixel color using the pixel mask.

I’m also correcting for the aspect ratio in a kind of hacky way. Really to get pixel perfect output things need to be handled a little more carefully. I’m a little too lazy to get that working.

[wickedpopular]

It seems like you’re already scaling the image, which is good. You just need to check in the shader if you’re outside of the 0.0 to 1.0 range of the UVs and switch to showing the background color.

Okay. I added this to the bottom of my shader

                    //Check for Border
                    if ((i.uv.x < 0.0) || (i.uv.y < 0.0) || (i.uv.x > 1.0) || (i.uv.y > 1.0))
                    {
                        colorized = (_ColorMatrix[3] - float4(.2, .2, .2, 0));
                    }

Which gives me this:

You’re very close. The two main things to things about here is the TexelSize * UV is going to give you a value where each increment of 1 is a single pixel of the input texture. You want lines between each of those, so you want to multiply the x & y by some value (like 8) before doing the modulo (%) by the same value. That way you’re doing a grid between each pixel and not on every 8 pixels (which is what you’re doing now).

I changed my Pixel Border code to this:

                    // Pixel Border
                    int x = _MainTex_TexelSize.z * (i.uv.x * 8);
                    int y = _MainTex_TexelSize.w * (i.uv.y * 8);

                    int xmod = x % 8;
                    int ymod = y % 8;

                    if (xmod == 0 || ymod == 0)
                    {
                        colorized = (_ColorMatrix[3] - float4(.2, .2, .2, 0));
                    }

For this:

It’s closer than what I had, but still isn’t really the behavior I was looking for. For some reason the lines aren’t distributed to each pixel, but instead gives me an irregular plaid pattern that changes as I move around in the editor.

So basically I create a shadow mask and a pixel mask, then lerping between the background color and the background multiplied the background by the shadow color using the shadow mask, and then lerp between that and the pixel color using the pixel mask.

I don’t understand.

[bgolus]

@wickedpopular

It’s closer than what I had, but still isn’t really the behavior I was looking for. For some reason the lines aren’t distributed to each pixel, but instead gives me an irregular plaid pattern that changes as I move around in the editor.

This is called aliasing. You’re still not rendering at a pixel-perfect scale, so each “pixel” (texel) of the original image is <8 pixels wide. Since you’re drawing a line every 1/8th of a texel, some of those lines aren’t getting seen as they’re thinner than the pixels you’re rendering on screen. That’s why I recommended an anti-aliased grid approach. Alternatively you need to hand the pixel-perfect aspect of all of this first, and make sure the grid is being displayed at the same texel-to-pixel scale you’re showing the base image at.

The easiest way to do this would be to convert the starting UVs into screen space. Normally I’d say use a Blit() to render, but since you’re using an orthographic camera with a perfectly aligned quad, we should be able to get away with not doing that.

Currently you’re doing this:

float2 scaleCenter = float2(0.5f, 0.5f);
i.uv = (i.uv - scaleCenter) * (1 + _ScaleFactor) + scaleCenter;

That’s centering the UV range so 0,0 is at the center, scaling them your arbitrary scale factor, and then moving them back so that 0.5, 0.5 is at the center. We can do a little more work here to convert things into screen pixels instead, and calculate a scale factor that keeps things pixel perfect as well as corrects for the aspect ratio difference.

float2 scaleCenter = float2(0.5f, 0.5f);

// center and scale UVs to match screen resolution.
// only use .y as Unity's camera frustums are vertically locked and your quad is square
// so we apply the vertical resolution to both the x and y of the UVs
float2 centeredPixelUV = (i.uv - scaleCenter) * _ScreenParams.y;

// scale the pixel UVs to match the texel size of the texture.
// using * xy instead of / zw because multiplication is faster than division.
// divide by "_PixelSizeScale", which is how many texels each pixel should cover.
i.uv = centeredPixelUV * _MainTex_TexelSize.xy / round(_PixelSizeScale) + scaleCenter;

Then later when you’re doing your grid, use the same round(_PixelSizeScale) to calculate how much to multiply the uv by, and for doing the modulo.

int xmod = x % (int)round(_PixelSizeScale);

You could even calculate the largest possible scale by comparing the screen resolution and the _MainTex resolution.

_PixelSizeScale = floor(_ScreenParams.y * min(_MainTex_TexelSize.x, _MainTex_TexelSize.y));

[wickedpopular]

Okay, I was finally able to work on this again.

As you suggested here

@bgolus

I’m getting a grid using a modified version of the example here:
http://www.iquilezles.org/www/articles/filterableprocedurals/filterableprocedurals.htm

@bgolus

That’s why I recommended an anti-aliased grid approach.

Although it was in a different language, I tried to implement the Antialiased Grid from that link. So far I have changed my pixel border section to this:

    // Pixel Border
                    float2 w = max( abs(ddx(i.uv)), abs(ddy(i.uv)));

                    float2 a = i.uv + 0.5 * w;
                    float2 b = i.uv - 0.5 * w;

                    float2 pixelI = (floor(a) + min(frac(a) * _GridScaler, 1.0) -
                                    floor(b) - min(frac(b) * _GridScaler, 1.0)) / (_GridScaler * w);

                    half4 gridline =  (1.0 - pixelI.x) * (1.0 - pixelI.y);

which gives me this:

“_GridScaler” is the replacement for the “const float N = 10.0;” from that link. It does give me an antialiased grid, but only at the texture borders, not at pixel borders. I’m not sure how to modify this further to subdivide the texture. From what I understand dpdx/ddx and dpdy/ddy are the necessary parts that find the pixel’s position on the screen, but I can’t figure out where I should be dividing the texture up to find the mid-pixel borders. Trying to apply the Modulo settings from my previous implementation just results in graphical errors like this.

[bgolus]

N controls the line width. The input UV needs to be scaled by the resolution, so the i.uv you have should be i.uv * _MainTex_TexelSize.zw. The ddx and ddy are screen space partial derivatives. In non-graphics programmer speak, it gets how much a value changes between that pixel and the next along the horizontal (ddx) and vertical (ddy). It’s how the anti-aliasing works.

[wickedpopular]

Hooray! Changing my border code as so:

                    //Screen Scaling
                    float2 scaleCenter = float2(0.5f, 0.5f);
                    float2 screenscaler = (i.uv - scaleCenter) * (1 + _ScaleFactor) + scaleCenter;

                    // Pixel Border
                    float2 texel_uv = screenscaler * _MainTex_TexelSize.zw;

                    float2 w = max(abs(ddx(texel_uv)), abs(ddy(texel_uv)));

                    float2 a = texel_uv + .5 * w;
                    float2 b = texel_uv - .5 * w;

                    float2 pixelI = (floor(a) + min(frac(a) * _GridScaler, 1.0) -
                                    floor(b) - min(frac(b) * _GridScaler, 1.0)) / (_GridScaler * w);

                    half4 gridline =  (1.0 - pixelI.x) * (1.0 - pixelI.y);

results in this:

Which finally gives a grid at each pixel! The grid can be scaled up and down without getting off the pixel grid as well, just as intended!

I’m still nos sure exactly what’s happening here though. As best I can tell, I’m getting the position on the screen of a pixel, then getting a point above and below it (a & b,) then determining somehow if that pixel is inside the bounds of the gridlines. Does that sound correct?

I’ve not quite figured out how to set those lines to the background color, however. Trying to multiply the half4 by the target color applys it to the space between the gridlines as well, darkening the whole thing. I tried inverting the gridlines by subtracting them from 1 before multiplying them by the target color, but that just darkened the spaced between the gridlines while lightening the rest up.

I’ve also started on this:

Part of the reason I was scaling the MainTexture was because I wanted the ability to achieve this effect. I’ve gotten very close, but as you can see there are slight black bars above and below the texture. By testing it with the pixel grid across the whole texture, I was able to determine that the border and the screen get off the same pixel grid. I’m not exactly sure why this happens, but I think it means I need to create a blank texture the scale of my desired output (256*224 in this case,) and then build up the layers that way instead of scaling.

So I would make a 256x224 tex2d, add the border to it, then add the 160x144 render texture into the middle of it. Is there a way to blend textures at their real scale like this without stretching them all the way across the model though?

I’ve added my whole current package, but here’s my current Frag method:

fixed4 frag(v2f i) : SV_Target
                {

                    //Screen Scaling
                    float2 scaleCenter = float2(0.5f, 0.5f);
                    float2 screenscaler = (i.uv - scaleCenter) * (1 + _ScaleFactor) + scaleCenter;

                    half4 gridcolor = (_ColorMatrix[3] - float4(.2, .2, .2, 0));

                    // Pixel Border
                    float2 texel_uv = screenscaler * _MainTex_TexelSize.zw;

                    float2 w = max(abs(ddx(texel_uv)), abs(ddy(texel_uv)));

                    float2 a = texel_uv + .5 * w;
                    float2 b = texel_uv - .5 * w;

                    float2 pixelI = (floor(a) + min(frac(a) * _GridScaler, 1.0) -
                                    floor(b) - min(frac(b) * _GridScaler, 1.0)) / (_GridScaler * w);

                    half4 gridline =  (1.0 - pixelI.x) * (1.0 - pixelI.y);


                    //Colorize
                    fixed red = tex2D(_MainTex, screenscaler).r;
                    float4 colorized = _ColorMatrix[red * 3];

                    //combine Gridlines and Colorized base texture
                    colorized = colorized * gridline;

                   
                    //Check for Border
                    if ((screenscaler.x < 0.0) || (screenscaler.y < 0.0) || (screenscaler.x > 1.0) || (screenscaler.y > 1.0))
                    {
                        colorized = (6,1,1,0);
                    }

                    fixed4 returnTexture = colorized;

                    //Apply border
                    fixed4 borderTexture = tex2D(_BorderTex, i.uv);
                    returnTexture = lerp(colorized, borderTexture, borderTexture.a);


                    return returnTexture;

                }

5076080–499136–GB Screen Test Project (Version 2).unitypackage (42.7 KB)