Flax Facts #4 – Shaders Parser

Shading Surfaces using Shaders

Hello!

It’s Friday again! As always we have some news for you and of course interesting post.
This time I will talk a little about our shaders, especially parsing shaders.
But first the news.
In this week we added more support for Compute pipeline in DirectX 12 and exposed more Engine API on C# side.
We’re also moving forward with C# Editor. Currently we’re moving our GUI from C++ to C# side (see branch master on API repo).

Flax Shaders

All shaders in Flax Engine are written in HLSL.
This is pretty straight forward because currently we support only DirectX 11 and 12 as graphics back-ends.
Additionally porting engine to Xbox One and PS4 will be easy (very similar graphics pipeline and shaders). Remember that Window+XboxOne+PS4 are the most common target for AAA titles and have very big marker coverage which is great.
But what about Linux, Mac and Mobile?
The answer is:

• Linux – mostly interested in ‘Flax-Server version’ for multiplayer games hosting without graphics running (console version)
• Mac – add Metal 2 backend (see news from WWDC 17) + use auto-converting tool for shaders
• Android/iOS – support OpenGL ES 3+ and auto-convert shaders to GLSL

Why we don’t have shaders in GLSL? Because GLSL sucks and our shader-base is quite huge (9 621 lines of HLSL code). It would be probably the easier to do it with materials which we auto-generate them from Material Editor Graph.

Shader Meta

This is how looks sample material shader in Flax:

// Vertex Shader
META_VS(false)
META_VS_IN_ELEMENT(POSITION, 0, R32G32B32_FLOAT, 0, ALIGN, PER_VERTEX, 0)
META_VS_IN_ELEMENT(TEXCOORD, 0, R16G16_FLOAT, 0, ALIGN, PER_VERTEX, 0)
Model_VS2PS VS_Model(ModelInput input)
{
	Model_VS2PS output;
	output.Position = mul(float4(input.Position.xyz, 1), WVP);
	output.Texcoord = input.Texcoord;
	return output;
}

// Pixel Shader
META_PS(false)
META_PERMUTATION_1(USE_LIGHTMAP=0)
META_PERMUTATION_1(USE_LIGHTMAP=1)
GBufferOutput PS_GBuffer(MaterialInput input)
{
#if USE_LIGHTMAP
	..sample lightmap..
#endif
	...
}

What is the best about it? We use custom set of macros eg. META_VS_IN_ELEMENT, META_PERMUTATION_1 which are used to add more functionalities and extensibility to the shaders.

Before actual compilation we parse source code and look for those special macros.
This gives us more information about the shader like for eg.: used functions, input layouts, permutations, hidden state, etc.

How to permutate?

The previous example is rather simple, now let’s talks about something more complicated.
One of the biggest shader challenges for game engines are shader permutations.
To understand the problem take a look at the following example:

// Pixel shader for spot light rendering
META_PS(false)
META_PERMUTATION_2(NO_SPECULAR=0, USE_IES_PROFILE=0)
META_PERMUTATION_2(NO_SPECULAR=1, USE_IES_PROFILE=0)
META_PERMUTATION_2(NO_SPECULAR=0, USE_IES_PROFILE=1)
META_PERMUTATION_2(NO_SPECULAR=1, USE_IES_PROFILE=1)
void PS_Spot(Model_VS2PS input, out float4 output : SV_Target0)
{
	...

	// Sample GBuffer
	GBuffer gBuffer = SampleGBuffer(uv);

	// Calculate lighting
	output = GetLighting(Light, gBuffer, shadow, true, true);
	
	// Apply IES texture
#if USE_IES_PROFILE
	output *= ComputeLightProfileMultiplier(IESTexture, gBuffer.WorldPos, Light.LightPos, -Light.LightDir);
#endif
}

This shader is used to calculate per pixel lighting for spot lights (tooltip: IES Profile is a technique to simulate real-life light bulb physical properties like non-uniform light propagation, look at the above screenshot).
The problem is that we have to create a set of different variations of this shader.

1) With specular, without IES Profile.
2) Without specular, without IES Profile.
3) With specular, with IES Profile.
4) Without specular, with IES Profile.

However we don’t want to write 4 shaders but single one and just permutate it.
That’s why we use macros META_PERMUTATION_2. Then we can simply compile shader with different sets of macros (eg. NO_SPECULAR=1, USE_IES_PROFILE=0) and generate different shaders from the same source code. The key to success is that we use data-oriented design and define all possible shader permutation statically in a shader. Later at runtime we just select desire permutation by index and use it.

Lesson learnt

Our shaders are doing pretty nice but there is always something to improve.
If you have any questions or suggestions feel free to write.
Also next time I will probably write more about our node-based Materials Editor and how we generate materials shader source code. Sounds interesting?

Bye! 🙂