Deferred Rendering in Modern OpenGL – Part 2

During the first part of this little series, we looked at the top-level overview of Deferred Rendering, it’s strengths and weaknesses. Sorry for the massive delay by the way, i’ve been caught up with work. Anyways this part will be more like a devlog where i’ll just be talking about the implementation process and thoughts on areas of improvement.

So to start off i created a new Framebuffer object with multiple render targets, with internal formats as follows :

  1. Position  : GL_RGB16F
  2. Normals : GL_RGB16F
  3. Albedo + Specular : GL_RGBA16F
  4. Depth : GL_DEPTH_COMPONENT

Note that the depth attachment is NOT a Renderbuffer Object but a Framebuffer Attachment like the others. This is because Renderbuffer Objects can only be written to and not sampled later on. This is a limitation for us, especially since a Depth texture is useful for certain Post-Process effects such as SSAO.

In the Framework i’ve created, each Model is stored in a container named ModelInstance which stores the Model, it’s transform as well as a Shader. For Deferred Rendering the Shader is NOT the Lighting Shader, but the G-Buffer Shader, and it stays the same for all Models unless a certain Model requires a specific Shader.

#version 440 core

layout (location = 0) in vec3 position;
layout (location = 1) in vec2 texcoord;
layout (location = 2) in vec3 normal;

uniform mat4 Model;
uniform mat4 Projection;
uniform mat4 View;

out vec2 TexCoord;
out vec3 FragPos;
out vec3 Normal;

void main()

{
       vec4 worldPos = Model * vec4(position, 1.0f);
       FragPos = worldPos.xyz;
       gl_Position = Projection * View * worldPos;
       TexCoord = texcoord;
       Normal = normalize(mat3(transpose(inverse(Model))) * normal);
}

Figure 1 : G-Buffer Vertex Shader

Vertex Shader isn’t anything special. Simply takes in the Models’ MVP matrices (individually for now), calculates World Space Position of current vertex, transforms Normal into World Space and passes through the TexCoord to the Fragment Shader.

#version 440 core

layout (location = 0) out vec3 g_Position;
layout (location = 1) out vec3 g_Normal;
layout (location = 2) out vec4 g_AlbedoSpecular;

in vec3 FragPos;
in vec2 TexCoord;
in vec3 Normal;
in float FragDepth;

uniform sampler2D Texture_Diffuse1;
uniform sampler2D Texture_Specular1;

void main()
{
       g_Position.rgb = FragPos;
       g_Normal = normalize(Normal);
       g_AlbedoSpecular.rgb = texture(Texture_Diffuse1, TexCoord).rgb;
       g_AlbedoSpecular.a = texture(Texture_Specular1, TexCoord).r;
 }

Figure 2 : G-Buffer Fragment Shader

If you recall the little GLSL snippet from the first part, this is the Shader it belongs to. All this does is merely output the World Space Position, Normal, Albedo and Specular, which is everything required to perform lighting, into their respective Render Targets.

To actually render to the G-Buffer all that is required is to simply iterate through the list of Models and Drawing them using the G-Buffer Shader like so;

BindGBuffer();
foreach Model
{
       GBufferShader.Bind();
       BindTextures();
       Model.Draw();
}

Figure 3 : Pseudo Code of Draw Loop

Afterwards it’s only a matter of binding a Lighting Shader of your choice, binding the necessary textures from the G-Buffer pass and Drawing everything onto a fullscreen quad with Default Framebuffer or another FBO if you wish to do some Post-Process.

void main()
{
       vec3 FragPos = texture(g_Position, TexCoords).rgb;
       float FragDepth = LinearizeDepth(texture(g_Depth, TexCoords).r);
       Normal = texture(g_Normal, TexCoords).rgb;
       vec3 Albedo = texture(g_AlbedoSpecular, TexCoords).rgb;
       float Specular = texture(g_AlbedoSpecular, TexCoords).a;

       // Perform Lighting as usual
}

Figure 4 : Simplified main method of  Lighting Fragment Shader

And there you have it! Deferred Rendering! Not so hard now is it? Now let’s take a look at some screenshots of each texture of the G-Buffer, along with one extra!

Terminus 2016-03-05 22-14-06-92

Image 1 : Final Composition

Terminus 2016-03-05 22-13-54-82

Image 2 : Position

Terminus 2016-03-05 22-13-56-15

Image 3 : Normals

Terminus 2016-03-05 22-13-57-15

Image 4 : Albedo

Terminus 2016-03-05 22-14-02-98

Image 5 : Depth

Terminus 2016-03-05 22-14-04-47

Image 6 : SSAO (Surprise!)

SSAO is something i recently implemented and it will definitely get it’s own post in the future as it is a technique that really makes use of the G-Buffer. I had to leave the Specular Texture out as this particular scene used full Specular on all the Models so i ended up with a plain white image.

Further Improvements

This implementation is by no means perfect, or even good. Remember i am still a beginner at this and this post was only meant to give you an understanding of how you could go about implementing your own Deferred Renderer. Anyways as for improvements, one thing i am considering is to reconstruct World Space Positions using the Depth Buffer, because this will allow us to remove the Position Buffer from the G-Buffer thus making it smaller. Matt Pettineo from Ready-At-Dawn has some great tutorials about it on his blog.

After writing the last post, i went ahead and implemented some basic PBR into the engine using a Microfacet BRDF and Image-Based Lighting which will get it’s post in the future. And for PBR to work as intended, each Models materials require a Roughness and Metalness value (A workflow that uses these two values is called a Metalness Workflow, with it’s alternative being Specular Workflow). Now if i am to use these values in the Lightning Shader they have to be stored in the G-Buffer as well, and since these two values are merely floats,one of them can be stored in place of Specular inside the Albedo Texture’s w component, and another can be stored inside of the Normal Textures w component. Another addition to the G-Buffer would be a ‘Velocity Buffer’ which will is used for Motion Blur.

Okay, that is a LOT of improvements! They will surely take a while but i will definitely get around creating posts for each of them along with some of the other things i promised. Until then, Happy Coding Folks!

Advertisements

Deferred Rendering in Modern OpenGL – Part 1

So i managed to implement Deferred Rendering into my little OpenGL Engine. It was mostly straightforward apart from some hiccups at the end due to my carelessness. I thought i’d write a little two part series to help anyone who’d like to implement this into their own engines too. In this first part i’ll go over the reasons switch to a Deferred Renderer and give you a top-down view of the algorithm. Anyways let’s get to it!

The typical method of rendering any beginner will be taught is to simply render an Object in a rendering queue, perform lighting calculations and move on to the next one until the entire queue is rendered, at which point the buffers get swapped and the next rendering loop begins. This method is commonly known as “Forward Rendering”. While this may be relatively straightforward and simple, it’s certainly not the most efficient. Why? Let’s take a look this some of it’s issues,

  • Pixel Overdraw : If your list of objects is not ordered by ascending depth order (which it most likely isn’t) the graphics card will not know whether one object is going to occlude another, so it’ll end up performing lighting calculations on pixels which are going to end up being overwritten, wasting our precious fillrate. This is not a big deal for a simple shader, but once things start getting complicated it will make an impact.
  • Low Performance with a large number of lights : Let’s say we have a scene with a 100 lights. Now if we forward render this scene we’d end up having to calculate the light contribution of all 100 lights per pixel! That is a LOT of work! and most of it will simply be wasted since the lights outside out view probably don’t even reach the objects in question.

Deferred Rendering is an alternative rendering method which can help you overcome these two issues. How does it archive this? Simply by Deferring the lighting calculations to a later stage. This technique consists of two rendering passes : the ‘G-Buffer Pass’ and the ‘Shading Pass’.

Step 1 : G-Buffer Pass

The G-Buffer is simply a Framebuffer with a set of textures used to store the Position, Normal, Specular value, Albedo color (your diffuse textures) and most likely depth. all of this data can be packed in various different methods and i will go about them in the next part as i have yet to try out some of them myself! The image below shows the contents of the G-Buffer for this demo scene.

Test

We can store all this data using individual passes if you wish, but there is a better method called Multiple Render Targets (MRT) that allows us to store all of this data into their respective textures during one draw call. That save us a LOT of draw calls (and CPU overhead, more on that in a future post).

Step 2 : Shading Pass

Now that we have stored all the data in textures, we can begin the actual lighting process. It’s extremely straightforward now! We simply bind the resulting textures from the first step, sample them through our Fragment / Pixel shader to get the per-pixel data required for lighting calculations such as Normals, Position, Specular values, then perform lighting as usual. Here is a GLSL snippet showing how you would go about retrieving the values from the textures.

vec3 FragPos = texture(g_Position, TexCoords).rgb;
float FragDepth = LinearizeDepth(texture(g_Depth, TexCoords).r);

vec3 Normal = texture(g_Normal, TexCoords).rgb;

vec3 Albedo = texture(g_AlbedoSpecular, TexCoords).rgb;
float Specular = texture(g_AlbedoSpecular, TexCoords).a;

Figure 1 : GLSL Snippet for Filling G-Buffer

Note how the Specular and Albedo values are stored in the same texture? the Albedo is stored in the x, y, and z components of a vec4 and the Specular value is stored in the w component. Something i picked up from Joey De Vries tutorial on Deferred Rendering!

You can choose to render this last Shading Pass onto the default framebuffer or render into another framebuffer in order to perform post-process effects such as HDR, Bloom, Depth-of-Field and Screen Space Ambient Occlusion, all of which are made extremely easy because of the G-Buffer (for example : for Depth of field all you need to do is sample the depth texture to figure out the depth of a fragment then apply a blur filter such as Gaussian Blur to fragments that are beyond a given range).

Drawbacks

Wait you’re telling me it isn’t perfect? Well sadly, nothing is. Deferred Rendering does indeed have it’s share of problems. Let’s go over them :

  • High Memory Consumption : If you are using 5 Textures as i have used here, you will take up a considerable amount of video memory to store these. Rendering at 1920 x 1080 results in roughly 6MB per texture when using 8-Bits per channel  RGB textures. But for HDR rendering and for storing position values you require textures with more precision, such 16-bits, in which case the size is well over 12MB. In a memory-restricted environment such as mobile, taking the deferred route is not worth it.
  • No Alpha Blending : In it’s usual state Deferred Rendering does not allow you to perform any kind of Transparency operations. There are methods to tackle this problem and i will not go over the details here.
  • Lack of hardware Anti-Aliasing : Because our lighting data is stored in textures, applying Anti-Aliasing techniques such as MSAA to them will interpolate the data and give us incorrect lighting. Like Alpha Blending there are plenty of ways to get around this problem, one of which is to use blur-based post-processing Anti-Aliasing solutions such as FXAA, MLAA and it’s better sibling SMAA.

As you can see Deferred Rendering is not always applicable, but in this day and age, Deferred Rending is a standard feature among Game Engines such as Unreal Engine 4 and CryEngine 3.

Conclusion

Deferred Rendering is certainly a great technique to have in your arsenal, especially for projects where fillrate is critical or if you need to implement many post-process effects. Implementing it might be a little tricky, but once you have grasped the concept, it’ll be a piece of cake! I’ll go over implementation details and some solutions to the drawbacks in the next part, along with some of the cool Post-Process effects you can archive with the G-Buffer. Until then, Happy Coding!

Further Reading

[1] GPU Gems 2 – Chapter 9. Deferred Shading in S.T.A.L.K.E.R.

[2] GPU Gems 3 – Chapter 19. Deferred Shading in Tabula Rasa