So i was looking into getting this nice edge effect, that you see in the Breath of the Wild. After reading about it (about sobel and other methods), it seemed that fresnel is the cheapest but also a non-consistent method of doing it, because its based on the angle to the camera.
After thinking about it tho, wouldn’t it be possible to make it more consistent? I’m no code master but I just wanted to share my thoughts here.
In most fresnel implementations, you can have a value influence the spread of the fresnel (at which angle to start showing fresnel, using a hard falloff or a smooth falloff). So what would happen if we, instead of having a static number in there, would feed in the angle towards the surface normalized (im not sure about the exact math opereations), a value between 0 and 1. Wouldn’t this basically change the fresnel spread dynamically, so that you can have a outline, which is kind of consistent??
Would be awesome, if some of you could help me out here!! It probably isnt even that comlicated to do, but like i said before, my shader knowledge is pretty limited…
Btw I also think, Nintendo did something similar with fresnel (a look at this bear model shows, that the outline effect indeed is fresnel. They just stabilized it somehow in most situations)
You could normalize it. This is a range between 0 and 1 and then scale it back up using another float, to create what is basically a gradient outline of constant thickness.
Don’t forget to modify that scale float by resolution or whatever personal preference.
Edit: read that you considered it. Why not try it?
Sadly i didn’t found a Solution yet, but I’m still searching occasionally. And thanks @neoshaman for the answer! But what you describe is just the effect, that the fresnel only is seen in lit parts of the model, right? I’m looking for this rather weirdly stable fresnel though. Im just checking ingame again right now and it is infact fresnel, thats for sure (small inconsistencies with the width and so on) But its way more stable overall, then what normal fresnel could supply.
Also whats not happening so often are these big fresnel areas, when looking at it from a really low angle.
One thing for sure tho, i could be linked to the lighting model, like you said @neoshaman . Because if the camera gets too close to Link, they turn him transparent to fade him out, which seems to change the shader. Interestingly, the lighting overall changes a bit and the fresnel outlines dissappear, until you get again far away from him and the solid shader is used again.
Ok, so I’m just analyzing the outline, writing notes and stuff, but I’m pretty sure now, that it uses the depth buffer in some way. I talked about the transparent shader earlier and am now seeing, that in some cases only the outlines disappear, once Link turns transparent. If it would be only Fresnel, without anything else to it, why would they remove this effect then? So it has to be about the depth buffer.
As far as i can tell, it’s still Fresnel in the end, not the vertex displacement like minionsArt is doing it. The outlines are still on the model and not separate. And that’s also what confuses me i guess- Other solutions like edge detection from a sobel filter would produce much more consistent Outlines, but also couldnt be masked by the lit areas. So it has to be Fresnel somehow. Thanks for the suggestion though ouo b
Yeah, I agree with this. It’s clearly a fresnel effect.
My guess is the effect is carefully tuned on Link (and other ‘hero’ characters) to look as clean as it does, but it’s not perfect and the outline gets extra thin or extra fat at some angles like you would expect from a fresnel effect, even on Link. A big part of getting these kinds of outlines to look good is having a nice, smooth, high polygon model to work from. This is part of why it looks so good on Link, and other major characters as they have the highest poly counts. There may be an additional bit of hinting baked into either their meshes or a texture that stores the relative curvature of that part of the mesh, much like what pre-integrated skin shaders use.
One interesting bit about Zelda:BotW, is it appears to be using a deferred renderer, and the cell shading on characters and painterly / sketchy effects on the world are likely applied as a post process effect of sorts. This means that something like a depth based Sobel edge detection filter could have been applied to determine the outlines and get perfect single-pixel width outlines, but they did not choose to go that route for either performance or stylistic reasons. There’s also no reason you couldn’t calculate the sobel and mask it by shadows. A sobel filter doesn’t need to be applied to the entire screen at all times.
BOTW cell shading is def a postprocess effect as highlighted by bugs. It’s nicely done, and probably use alot of hack to get the final result. Nintendo is known to be creative with visual effect like mipmap hack to get amazing ocean shading in sunshine and multiple quad duplicating the scene render and applying a max() on pixel intensity to get teh painterly effect on skyward sword.
About the postprocess effect, I think i saw something similar in a glitch video, where you can stand under a bridge, and the celshading gets turned off! So it probably was done using it (Never wrote a postprocessshader tho)
I also looked more into it and actually all models, that have the celshading applied, use this sort of fresnel lines. Probably for performance reasons. Actually most objects, that don’t use the celshading seem to use a edge detection filter. Its weird and really subtle, but its has to be one or they use evil magicxD
So one year later now, i guess I found out what was going on and @neoshaman was right all along. I was lead by two reasons too believe that it was some fresnel magic. One being the inconsistencies of the edge effect and the fact that it seemed only to be located on Link.
As my last post stated, i noticed and edge effect way more subtle on other objects too, I still can’t say for sure, where they mask it away, but the effect is there all over the scene. Probably a lower intensity in shadows and maybe some light direction mask mixed in there. They use probably the mask for the toonshading also for this effect and let the outlines on the characters be much more noticable.
Regarding the inconsistencies, I reread some old tutorials by chance and found this: Unity Outline Shader Tutorial - Roystan As you can see in the middle of the page, such artifacts can indeed happen with postprocess effects too.
Based on these facts, I would say it’s pretty clear for me now, what they used here! Thanks for the help along the way~
Yes, you could achieve something similar with stencils, but in BotWs case, I don’t think thats what they done here. Stencils only can do Outlines by shifting the whole maks into one direction and in the BotW image above, you can see that this isn’t the case. It has way more similarities with a outline postfx effect.
I should have provided the source of those conclusion:
when people discovered the glitch that tells you it’s post process
also I could have save you time with this article
I’m sorry to have let you down so long, yout twitter is fire by the way, my first stop everytime on the platform (though the sdfgi of godot 4.0 is coming close now)
That article is just someone attempting to recreate the effect. It doesn’t prove anything one way or another since it’s just someone’s guess. It also completely skips over the most impressive part of the BotW implementation, which is the surprisingly consistent line width on Link. Not perfect mind you, hence why I do agree it’s a Fresnel based effect, combined with it being applied as part of either a post process or a deferred lighting system.
The line width part is really the main topic of this thread. Whether or not it’s a post process is almost irrellevant.
@neoshaman No worries! It’s awesome alone that you two help anyways!
@bgolus Regarding the fresnel or edge detection discussion, the edge detection script I’m using at the moment can produce under specific settings with the depth threshold similar inconsistencies like fresnel. I just tried a direct comparison and tho not exactly the same, I think it’s close enough.
I also looked further into the edge effect on other objects and it is indeed mostly everywhere. Also on big structures like mountains or the skeletons in the desert.
That’s why I don’t think anymore, that Nintendo used any kind of textures to help support the effect. They just mask by lighting and add it on top of the existing color, except for cel shaded objects, where they make the lines opaque to really highlight the style.
