Debugging shaders in Visual Studio

I noticed this page in the docs:

This would be the first time ever that you can step through a Unity shader but it went largely under the radar. No announcement, no blog, no nothing.

Does anyone have experience with this?

I got it to work, but the Unity manual is both incomplete and the formatting could use some work also.

Follow this guide and you should be able to get it to work:


Update software:
-Download and install the DirectX SDK (June 2010).
-Install the latest Windows Update.
-Install Visual Studio 2015 (or higher) with the latest update. This is important because the updates often fix graphics debugging related bugs.
-Install Visual C++ for VS2015. Note that in Visual Studio 2015, Visual C++ is not installed by default. When installing, be sure to choose Custom installation and then choose the C++ components you require. Or, if Visual Studio is already installed, choose File | New | Project | C++ and you will be prompted to install the necessary components, or run the installer again. More information:

Create a new project in Unity.

Go to Hierarchy->Create->3D Object->Cube.

In the Project window, right click and select Create->Shader->Standard Surface Shader. Leave the name as NewSurfaceShader.

Open the NewSurfaceShader.shader file and add this line after “CGPROGRAM”:

#pragma enable_d3d11_debug_symbols

In the Project window, right click and select Create->Material.

Select the new material and go to Inspector->Shader->Select Custom/NewSurfaceShader.

Drag the new material onto the cube.

Go to File->Build Settings->Select PC Standalone, then click Switch Platform (if not already).

Go to Edit->Project Settings->Quality->Inspector->Disable anti aliasing for all quality levels.

Save the project.

Go to File->Build Settings->Tick the box “Development Build” and “Script Debugging”.

On the Build Settings window, select Player Settings->Inspector->Resolution and Presentation->Display Resolution Dialog->Disabled.

Select Player Settings->Inspector->Other Settings. Untick “Auto Graphics API for Windows” and check that Direct3D11 is at the top of the list. Also check if the Unity Editor title bar states “”.

Select Build, name the file test.exe, and click Save. Note the path to the executable.

Create a new project in Visual Studio 2015.
File->New->Project->Installed->Templates->Other Languages->Visual C+±>Empty Project.
Note: if Visual C++ is not available, follow the instructions at the “Update Software” section above.

Select a name and location for the project and click Ok.

Go to View->Solution Explorer. Select the project (the project, not the solution).

Go to Project->Properties->Configuration Properties->Debugging.
Note: if Debugging is not available in the Configuration Properties, you selected the solution instead of the project in the Solution Explorer, or you created a C# instead of a C++ project.

In the Command field, replace “$(TargetPath)” with the path to the Windows executable you build earlier.

In the Command Arguments field, add “-force-d3d11”.

Click OK.

Select Build->Build Solution.

Go to Debug->Graphics->Start Diagnostics

Install the Graphics Tools if a popup shows up.

The standalone build should fire up and this message should appear in the top left: “Frames captured: 0. Use Print Screen key to capture a frame”.

Press the Print Screen key.

Alt-Tab out of the standalone build and go to Visual Studio.
Note: to close the Unity test app, select ALT-F4.

You should see a frame capture in Visual Studio. Double click the captured frame and a new Visual Studio window should open.

On the new Visual Studio window, Place the crosshair on the pixel of interest and left click. You can zoom in and out of the frame using the mouse scroll wheel.

Go to the Graphics Event List window, select the first (top) occurrence of “obj:x DrawIndexed”.
Note: If you get a “No source available” warning later on, try one of the entries in the Camera.Render tree instead.

Go to Graphics Pixel History window->obj:x DrawIndexed->Triangle->Vertex Shader->click the green arrow next to it.

Now you can single step through your shader.

Trouble shooting

*If it displays the message “Source not available”, then either:
-You didn’t add this line to the shader: #pragma enable_d3d11_debug_symbols
-You didn’t install the latest Visual Studio update.
-You selected the wrong item in the Graphics Event List window. If the DrawIndexed item does not work, try one of the items in the Camera.Render tree.

*Many variables display a NaN value due to a bug in VS. Variables only show the correct value if the line of code has just been executed. If you want to view the value of variables elsewhere in the shader, you have to step over them and note them down manually.

*If the game is not displayed correctly after pressing Debug->Graphics->Start Diagnostics, then force use the integrated graphics instead of the high performance video card.

*If Visual C++ is not available, follow the instructions at the “Update Software” section above.

*If VS crashes, try running it as Administrator and make sure the latest VS update is installed.

Further reading

Edit:
-Updated to VS2015
-Updated to Unity 5.2

19 Likes

Are the steps reasonably close in VS2015? I cannot seem to find an empty project in the visual C++ section.

Update:

It appears I failed to install the C++ components when I setup 2015. :slight_smile:

Yes, but VC++ is not installed by default in VS2015. See here for more information:

I updated the tutorial now. Thanks for bringing this up.

You are welcome. After installing the VC++, it worked exactly as described.

cool tips,thanks

Hi @Elecman I tried to follow these steps but when I hit “Start Diagnostics”, even though my target for the project properties is set to run the unity editor executable, visual studio errors with

‘Unable to start program users/documents/testshader/debug/testshader.exe’

I.e. it’s still trying to start the VC++ project that was created rather than trying to run the executable, have I missed something?

Did you change the target path?

Search for this line:

Yeah I set the target correctly but the error still occurs. I’ve now gotten around this by opening the executable as a project and running from within visual studio which works ok.

My new issue is that when I go to hit the green triangle debug button from within the graphics analyzer for the object with the shader set on it. Visual studio throws up a prompt asking to find the hlsl file (temp.hlsl.377_1.hlsl) which implies that it didn’t manage to find the hlsl file.

I can’t seem to find this file anywhere in my project folder for where it might be built.

Yeah, the missing hlsl file thing is a bit of an issue. It can have many causes and sometimes even a restart will fix it. I have never gotten to the bottom of that.

You can check these, but if that doesn’t work, I don’t have any more tricks to try.

To be honest, the whole process is quite buggy, but it’s the best thing we’ve got.

Thank you for sharing! That was a great tutorial, and it worked for me.
As you mentioned, debugging shaders with VS2015 is not a great experience.
Could you please consider sharing tips on how to setup Nsight Visual Studio Edition 5.0 to debug dx11 shaders on VS2015?

Sorry, I have no clue about that one.

Anyone know how to debug a compute shader?

RenderDoc will automatically grab the temp hlsl file and all the cginc files.
2602609--182350--renderdoc_debug.png
You can have a try on it.
I think It’ s easier to use with almost equivalent features as Visual Studio Graphic tool.

Thanks for pointing that out. I didn’t even know that.

VSGA supports debugging compute shaders too. I imagine RenderDoc as well.
https://msdn.microsoft.com/en-us/library/dn217886.aspx

1 Like

Instead of creating an empty C++ project, you can just open the unity app as a project. Go to File | Open Project in Visual Studio and pick the compiled unity test app. Running ALT+F5 in VS will then let you capture that unity project.

1 Like

Thanks for the tip!

Awesome. I definitely did search and didn’t find that article You’re a great help.

Has anyone got “access denied” error? It appears when I try to start graphics debugging in VS 2015
3099021--233959--access denied.png

EDIT : In case anyone face the issue, I’ve solved the issue.

You have to mention the executable file name with extension as well with the Command directory in Properties > Debugging > Command.