How to Use Xcode to Find Out the Shader Code Bug?

I recently discovered a small bug related to the surface shader in Unity. You can see the rendering artifact shown below.

Sometimes people often attribute similar rendering artifacts to using HDR with bloom effect, but in fact the surface shader is wrong, at least as discussed in this article.

So I write this article to record the process of debugging this issue. At the same time, this article will also introduce how to use xcode tools to debug shader code to find existing rendering bugs.

The Debugging process

If you want to investigate rendering issues on iOS, just build and run your project from Xcode, then click the camera button on Xcode’s debugging toolbar to enable the Metal Frame Debugger.

Then select the render encoder that you are interested in, such as the encoder related to the downsample of bloom.

As you can see, at the beginning of the downsample process of bloom, there is a pixel that does not look normal. At this time, I think we find the target. But during the bloom process, the texture has been downsampled. To find the original pixel, let’s investigate the render encoder before bloom to see if there are any more valuable discoveries.

At first glance, nothing seems abnormal, then let’s modify the attachment’s setting to filter out our target. (By the way, right click on the attachment, you can choose whether to display visible overlays such as wireframe or flip vertical etc)

It’s here! Like the most shining star in the night sky. 😄

After finding the target pixel, we can use the shader profiler tool to debug the pixel shader code for this pixel. Seeing that its value is very abnormal, I am almost certain that there is a bug in this “Legacy Shaders/Bumped Specular” shader.

Select the target pixel, then click the Debug button. Let’s see what the issue is!

    u_xlat16_3.x = dot(input.TEXCOORD1.xyz, float3(u_xlat16_2.xyz));
    u_xlat16_3.y = dot(input.TEXCOORD2.xyz, float3(u_xlat16_2.xyz));
    u_xlat16_3.z = dot(input.TEXCOORD3.xyz, float3(u_xlat16_2.xyz));
    u_xlat16_0.x = dot(u_xlat16_3.xyz, u_xlat16_3.xyz);
    u_xlat16_0.x = rsqrt(u_xlat16_0.x);
    u_xlat0.xyz = float3(u_xlat16_0.xxx) * float3(u_xlat16_3.xyz);

As you can see, the result of dot(u_xlat16_3.xyz, u_xlat16_3.xyz).x is 0, then the rsqrt operator operates on the value. Yes, the reciprocal square root of 0 is ideally infinity and the square root of negative values ideally returns NaN (Not a Number). I think we find the shader code bug.

Simply put, the shader code does an unsafe normalize here.

normalize for a float3 vector could be implemented like this.

float3 normalize(float3 v)
{
  return rsqrt(dot(v,v))*v;
}

The ShaderLab Code

As many Unity developers know, the shader we write in Unity is called ShaderLab, and then Unity will compile the ShaderLab into shader code for the graphics library/platform.

And the Legacy Shaders / Bumped Specular is a built-in shader. So we can download the shaderlab code from the Unity download page and use an IDE open the shader file.

As shown above, the code of this shader is very very simple, and there is no normalize operation for the vector. Why?

Because it is a surface shader. And surface shaders in Unity is a code generation approach that makes it much easier to write lit shaders than using low level vertex/pixel shader programs.

If you want to see the real shaderlab code that generated from a surface shader, you can click the show generated code button on the shader inspector window.

Then we get the real vertex/pixel shaderlab code that will be compiled into the native shader code later. Furthermore, we find the unsafe normalization operation generated by Unity.

Workaround

Of course, the purpose of this article, as I said at the beginning of the article, is mainly to record the process of debugging shader code bugs and introduce the Xcode shader profiler tool. But how should this issue or similar issues be solved?

I personally think it is just not to use the surface shaders. Because most of their code is generated by Unity and you can’t control. You can use traditional standard shaders or consider new shaders for scriptable render pipelines.

Useful Links

Developing and Debugging Metal Shaders

I recently find microsoft learn helpful. Check it out!

Microsoft Learn