[index]

OpenGL 4 Shaders

Anton Gerdelan. Last edited: 2 October 2016

Shaders tell OpenGL how to draw, but we don't have anything to draw yet - we will cover that in the vertex buffers article. If you would rather skip the introduction, you can jump straight to the Example Shaders and get them loaded into GL with the Minimal C Code.

Overview

Shaders are mini-programmes that define a style of rendering. They are compiled to run on the specialised GPU (graphics processing unit). The GPU has lots of processors specialised for floating-point operations. Each rendering stage can be split into many separate calculations - with one calculation done on each GPU processor; transform each vertex, colour each tiny square separately, etc. This means that we can compute a lot of the rendering in parallel - which makes it much faster than doing it with a CPU-based software renderer where we only have 1-8 processors (in the graphics world, "hardware" implies the graphics adapter). Example:

Common drawing operations such as transforming each vertex and colouring each fragment can be done independently. We write a shader (mini programme) to compute these operations, and the GPU's highly parallel architecture will attempt to compute them all concurrently.

Shaders are a way of re-programming the graphics pipeline. If we wanted to use a different colouring method for the cube in the image, or have an animated, spinning cube, we could tell OpenGL to switch to using a different shader programme. The rendering process has several distinct stages of transforming a 3d object in a final 2d image. We call this staged process the graphics pipeline. All of the stages of the graphics pipeline that happen on the GPU are called the [programmable] hardware pipeline. Older OpenGL APIs had pre-canned functions like glLight() for driving the rendering model. We call this the fixed-function pipeline ("fixed" because it's not re-programmable). These functions no longer exist, and we have to write the lighting equations ourselves in shaders. In OpenGL 4 we can write a shader to control many different stages of the graphics pipeline:

A complete shader programme comprises a set of separate shader (mini-programmes) - one to control each stage. Each mini-programme - called a shader by OpenGL - is compiled, and the whole set are linked together to form the executable shader programme - called a program by OpenGL. Yes, that's the worst naming convention ever. If you look at the Quick Reference Card (or further down the page) you can see that the API differentiates functions into glShader and glProgram (note US spelling of "programme").

Each individual shader has a different job. At minimum, we usually have 1 vertex shader and 1 fragment shader per shader programme, but OpenGL 4 allows us to use some optional shaders too.

Shader Parallelism

Shader programmes run on the GPU, and are highly parallelised. Each vertex shader only transforms 1 vertex. If we have a mesh of 2000 vertices, then 2000 vertex shaders will be launched when we draw it. Because we can compute each one separately, we can also run them all in parallel. Depending on the number of processors on the GPU, you might be able to compute all of your mesh's vertex shaders simultaneously.

Because there is a lot of variation in user GPU hardware, we can only make very general assumptions about the ideal number of vertices or facets each mesh should have for best performance. Because we only draw one mesh at a time, keeping the number of separate meshes drawn per-scene to a low-ish level is more beneficial (reducing the batch count per rendered frame) - the idea is to keep as many of the processors in use at once as possible.

Difference Between Fragments and Pixels

A pixel is a "picture element". In OpenGL lingo, pixels are the elements that make up the final 2d image that it draws inside a window on your display. A fragment is a pixel-sized area of a surface. A fragment shader determines the colour of each one. Sometimes surfaces overlap - we then have more than 1 fragment for 1 pixel. All of the fragments are drawn, even the hidden ones.

Each fragment is written into the framebuffer image that will be displayed as the final pixels. If depth testing is enabled it will paint the front-most fragments on top of the further-away fragments. In this case, when a farther-away fragment is drawn after a closer fragment, then the GPU is clever enough to skip drawing it, but it's actually quite tricky to organise the scene to take advantage of this, so we'll often end up executing huge numbers of redundant fragment shaders.

Shader Language

OpenGL 4 shaders are written in OpenGL Shader Language version 4.00.9. The GLSL language from OpenGL versions 3 to 4 is almost identical, so we can port between versions without changing the code. OpenGL version 3.2 added a new type of shader: geometry shaders, and version 4.0 added tessellation control and tessellation evaluation shaders. These, of course, can not be rolled back to earlier versions. The first line in a GLSL shader should start with the simplified version tag:

#version 400

The different version tags are:

GLSL Operators

GLSL contains the operators in C and C++, with the exception of pointers. Bit-wise operators were added in version 1.30.

If you leave out the version tag, OpenGL fall back to an earlier default - it's always better to specify the version.

GLSL Data Types

The most commonly used data types in GLSL are in the table below. For a complete list see any of the official reference documents.

File Naming Convention

Each shader is written in plain text and stored as a character array (C string). It is usually convenient to read each shader from a separate plain text file. I use a file naming convention like this;

Some text editors (notepad++, gedit, ...) will do syntax highlighting for GLSL if you end with a ".glsl" extension. The GLSL reference compiler; Glslang can check your shaders for bugs if they end in ".vert" and ".frag".

Example Shaders

GLSL is designed to resemble the C programming language. Each shader resembles a small C programme. Let us examine a very minimal shader programme that has only a vertex shader and a fragment shader. Each of these shaders can be stored in a C string, or in a plain text file.

In this case we just want to be able to accept a buffer of points and place them directly onto the screen. The hardware will draw triangles, lines, or points using these, depending on the draw mode that we set. Every pixel-sized piece (fragment) of triangle, line, or point goes to a fragment shader. Just for the sake of example, we want to be able to control the colour of each fragment by updating a uniform variable in our C programme.

Vertex Shader

The vertex shader is responsible for transforming vertex positions into clip space. It can also be used to send data from the vertex buffer to fragment shaders. This vertex shader does nothing, except take in vertex positions that are already in clip space, and output them as final clip-space positions. We can write this into a plain text file called: test_vs.glsl.

#version 420

in vec3 vertex_position;

void main() {
  gl_Position = vec4(vertex_position, 1.0);
}

GLSL has some built-in data types that we can see here:

• vec3 is a 3d vector that can be used to store positions, directions, or colours.
• vec4 is the same but has a fourth component which, in this variable, is used to determine perspective. We will examine this in the virtual camera article, but for now we can leave it at 1.0, which means "don't calculate any perspective".

We can also see the in key-word for input to the programme from the previous stage. In this case the vertex_position_local is one of the vertex points from the object that we are drawing. GLSL also has an out key-word for sending a variable to the next stage.

The entry point to every shader is a void main() function.

The gl_Position variable is a built-in GLSL variable used to set the final clip-space position of each vertex.

The input to a vertex buffer (the in variables) are called per-vertex attributes, and come from blocks of memory on the graphics hardware memory called vertex buffers. We usually copy our vertex positions into vertex buffers before running our main loop. We will look at vertex buffers in the next tutorial. This vertex shader will run one instance for every vertex in the vertex buffer.

Fragment Shader

Once all of the vertex shaders have computed the position of every vertex in clip space, then the fragment shader is run once for every pixel-sized space (fragment) between vertices. The fragment shader is responsible for setting the colour of each fragment. Write a new plain-text file: test_fs.glsl.

#version 420

uniform vec4 inputColour;
out vec4 fragColour;

void main() {
  fragColour = inputColour;
}

The uniform key-word says that we are sending in a variable to the shader programme from the CPU. This variable is global to all shaders within the programme, so we could also access it in the vertex shader if we wanted to.

The hardware pipeline knows that the first vec4 it gets as output from the fragment shader should be the colour of the fragment. The colours are rgba, or red, green, blue, alpha. The values of each component are floats between 0.0 and 1.0, (not between 0 and 255). The alpha channel output can be used for a variety of effects, which you define by setting a blend mode in OpenGL. It is commonly used to indicate opacity (for transparent effects), but by default it does nothing.

Minimal C Code

Note that there is a distinction between a shader, which is a mini-programme for just one stage in the hardware pipeline, and a shader programme which is a GPU programme that comprises several shaders that have been linked together.

To get shaders up and running quickly you can bang this into a minimal GL programme:

1. load a vertex shader file and fragment shader file and store each in a separate C string
2. call glCreateShader twice; for 1 vertex and 1 fragment shader index
3. call glShaderSource to copy code from a string for each of the above
4. call glCompileShader for both shader indices
5. call glCreateProgram to create an index to a new program
6. call glAttachShader twice, to attach both shader indices to the program
7. call glLinkProgram
8. call glGetUniformLocation to get the unique location of the variable called "inputColour"
9. call glUseProgram to switch to your shader before calling...
10. glUniform4f(location, r,g,b,a) to assign an initial colour to your fragment shader (e.g. glUniform4f(colour_loc, 1.0f, 0.0f, 0.0f, 1.0f) for red)

The only variables that you need to keep track of are the index created by glCreateProgram, and any uniform locations. Now we are ready to draw - we will look at drawing geometry with glDrawArrays in the next tutorial. To set or change uniform variables you can use the various glUniform functions, but they only affect the shader programme that has been switched to with glUseProgram.

OpenGL Shader Functions

For a complete list of OpenGL shader functions see the Quick Reference Card. The most useful functions are tabulated below. We will implement all of these:

Adding Error-Checking Functionality

The first thing to do is extend the minimal code with some error-checking.

Check for Compilation Errors

Right after calling glCompileShader:

glCompileShader(shader_index);
// check for compile errors
int params = -1;
glGetShaderiv(shader_index, GL_COMPILE_STATUS, &params);
if (GL_TRUE != params) {
  fprintf(stderr, "ERROR: GL shader index %i did not compile\n", shader_index);
  _print_shader_info_log(shader_index);
  return false; // or exit or something
}

I call a user-defined function here to print even more information from the shader. See next section.

Print the Shader Info Log

void _print_shader_info_log(GLuint shader_index) {
  int max_length = 2048;
  int actual_length = 0;
  char shader_log[2048];
  glGetShaderInfoLog(shader_index, max_length, &actual_length, shader_log);
  printf("shader info log for GL index %u:\n%s\n", shader_index, shader_log);
}

This is the most useful shader debugging function; it will tell you which line in which shader is causing the error.

Check for Linking Errors

Right after calling glLinkProgram:

// check if link was successful
int params = -1;
glGetProgramiv(programme, GL_LINK_STATUS, &params);
if (GL_TRUE != params) {
  fprintf(stderr,
    "ERROR: could not link shader programme GL index %u\n",
    programme);
  _print_programme_info_log(programme);
  return false;
}

I call a user-defined function here to print even more information from the shader. See next section.

Print the Program Info Log

void _print_programme_info_log(GLuint programme) {
  int max_length = 2048;
  int actual_length = 0;
  char program_log[2048];
  glGetProgramInfoLog(programme, max_length, &actual_length, program_log);
  printf("program info log for GL index %u:\n%s", programme, program_log);
}

Where programme is printing the index of my programme.

Print All Information

One of the more common errors is mixing up the "location" of uniform variables. Another is where an attribute or uniform variable is not "active"; not actually used in the code of the shader, and has been optimised out by the shader compiler. We can check this by printing it, and we can print all sorts of other information as well. Here, programme is the index of my shader programme.

void print_all(GLuint programme) {
  printf("--------------------\nshader programme %i info:\n", programme);
  int params = -1;
  glGetProgramiv(programme, GL_LINK_STATUS, &params);
  printf("GL_LINK_STATUS = %i\n", params);
  
  glGetProgramiv(programme, GL_ATTACHED_SHADERS, &params);
  printf("GL_ATTACHED_SHADERS = %i\n", params);
  
  glGetProgramiv(programme, GL_ACTIVE_ATTRIBUTES, &params);
  printf("GL_ACTIVE_ATTRIBUTES = %i\n", params);
  for (int i = 0; i < params; i++) {
    char name[64];
    int max_length = 64;
    int actual_length = 0;
    int size = 0;
    GLenum type;
    glGetActiveAttrib (
      programme,
      i,
      max_length,
      &actual_length,
      &size,
      &type,
      name
    );
    if (size > 1) {
      for(int j = 0; j < size; j++) {
        char long_name[64];
        sprintf(long_name, "%s[%i]", name, j);
        int location = glGetAttribLocation(programme, long_name);
        printf("  %i) type:%s name:%s location:%i\n",
          i, GL_type_to_string(type), long_name, location);
      }
    } else {
      int location = glGetAttribLocation(programme, name);
      printf("  %i) type:%s name:%s location:%i\n",
        i, GL_type_to_string(type), name, location);
    }
  }
  
  glGetProgramiv(programme, GL_ACTIVE_UNIFORMS, &params);
  printf("GL_ACTIVE_UNIFORMS = %i\n", params);
  for(int i = 0; i < params; i++) {
    char name[64];
    int max_length = 64;
    int actual_length = 0;
    int size = 0;
    GLenum type;
    glGetActiveUniform(
      programme,
      i,
      max_length,
      &actual_length,
      &size,
      &type,
      name
    );
    if(size > 1) {
      for(int j = 0; j < size; j++) {
        char long_name[64];
        sprintf(long_name, "%s[%i]", name, j);
        int location = glGetUniformLocation(programme, long_name);
        printf("  %i) type:%s name:%s location:%i\n",
          i, GL_type_to_string(type), long_name, location);
      }
    } else {
      int location = glGetUniformLocation(programme, name);
      printf("  %i) type:%s name:%s location:%i\n",
        i, GL_type_to_string(type), name, location);
    }
  }
  
  _print_programme_info_log(programme);
}

The interesting thing here are the printing of the attribute and uniform "locations". Sometimes uniforms or attributes are themselves arrays of variables - when this happens size is > 1, and I loop through and print each index' location separately. I also have a home-made function for printing the GL data type as a string (normally it is an enum which doesn't look very meaningful when printed as an integer) - see next section.

GLenum Data Type to C String

This function converts a GL enumerated data type to a C string (for readable printing).

const char* GL_type_to_string(GLenum type) {
  switch(type) {
    case GL_BOOL: return "bool";
    case GL_INT: return "int";
    case GL_FLOAT: return "float";
    case GL_FLOAT_VEC2: return "vec2";
    case GL_FLOAT_VEC3: return "vec3";
    case GL_FLOAT_VEC4: return "vec4";
    case GL_FLOAT_MAT2: return "mat2";
    case GL_FLOAT_MAT3: return "mat3";
    case GL_FLOAT_MAT4: return "mat4";
    case GL_SAMPLER_2D: return "sampler2D";
    case GL_SAMPLER_3D: return "sampler3D";
    case GL_SAMPLER_CUBE: return "samplerCube";
    case GL_SAMPLER_2D_SHADOW: return "sampler2DShadow";
    default: break;
  }
  return "other";
}

I only included the most-commonly used data types - I don't use the others so I just called them "other". Look up glGetActiveUniform to get the rest of the list.

Validate Programme

You can also "validate" a shader programme before using it. Only do this during development, because it is quite computationally expensive. When a programme is not valid, the details will be written to the program info log. programme is my shader programme index.

bool is_valid(GLuint programme) {
  glValidateProgram(programme);
  int params = -1;
  glGetProgramiv(programme, GL_VALIDATE_STATUS, &params);
  printf("program %i GL_VALIDATE_STATUS = %i\n", programme, params);
  if (GL_TRUE != params) {
    _print_programme_info_log(programme);
    return false;
  }
  return true;
}

General Conventions

• All uniform variables are intitialised to 0 when a programme links, so you only need to initialise them if the initial value should be something else. Example: you might want to set matrices to the identity matrix, rather than a zeroed matrix.
• Calling glUniform is quite expensive during run-time. Structure your programme so that glUniform is only called when the value needs to change. This might be the case every time that you draw a new object (e.g. its position might be different), but some uniforms may not change often (e.g. projection matrix).
• Calling glGetUniformLocation during run-time can be expensive. It is best to do this during intialisation of the component that updates the uniform e.g. the virtual camera for the projection and view matrices, or a renderable object in the scene for the model matrix. Store the uniform locations once, then call glUniform as needed, rather than updating everything every frame.
• When calling glGetUniformLocation, it returns -1 if the uniform variable wasn't found to be active. You can check for this. Usually it means that either you've made a typo in the name, or the variable isn't actually used anywhere in the shader, and has been "optimised out" by the compiler/linker.
• Modifying attributes (vertex buffers) during run-time is extremely expensive. Avoid.
• Get your shaders to do as much work as is possible; because of their parallel nature they are much faster than looping on the CPU for most tasks.
• Drawing lots of separate, small objects at once does not make efficient use of the GPU, as most parallel shader slots will be empty, and separate objects must be drawn in series. Where possible, merge many, smaller objects into fewer, larger objects.

Possible Extensions

• Create a log file system, rather than a print-out system, so that you can scan through large volumes of information from each loaded shader.
• Larger projects will use many different shader programmes. It would make sense to have a Shader Manager interface or class to load shaders, and to make sure that shaders are re-used, rather than loaded multiple times.
• In the future we will sometimes use geometry and tessellation shaders, so we can consider upgrading our shader functions to handle more than just vertex and fragment shaders. We can just set some boolean flags to true or false to indicate which types of shader have been loaded before attaching and linking.
• If you have a shader manager, and have written a function along the lines of setUniform(shader_index, value) from the manager, it could then check to make sure that shader is in use first (a common cause of error).