Test driven development (TDD) undoubtedly helps a great deal in preventing development grinding to a halt once a project’s size surpasses a few lines of code.
The reason for first writing a failing test is to ensure that the test is actually failing and testing the next code change.
A minimal change to the code is performed to pass the new test while also still passing all previously written tests.
If necessary the code is refactored/simplified. The reason to do this after passing the test is so that one does not have to worry about passing the test and writing clean code at the same time.
Testing rendering output
One can test OpenGL programs by rendering test images and comparing them with a saved image (a test fixture).
In order to automate this, one can perform offscreen rendering and do a pixel-wise image comparison with the saved image.
Using the Clojure programming language and the Lightweight Java Game Library (LWJGL) one can perform offscreen rendering with a Pbuffer object using the following macro (of course this approach is not limited to Clojure and LWJGL):
The image is recorded initially by using the checker record-image instead of is-image and verifying the result manually.
One can use this approach (and maybe only this approach) to test code for handling vertex array objects, textures, and for loading shaders.
Testing shader code
Above approach has the drawback that it can only test complete rendering programs.
Also the output is limited to 24-bit RGB images.
The tests are therefore more like integration tests and they are not suitable for unit testing shader functions.
However it is possible to use a Pbuffer just as a rendering context and perform rendering to a floating-point texture.
One can use a texture with a single pixel as a framebuffer.
A single pixel of a uniformly colored quad is drawn.
The floating point channels of the texture’s RGB pixel then can be compared with the expected value.
Furthermore it is possible to compose the fragment shader by linking the shader function under test with a main function.
I.e. it is possible to link the shader function under test with a main function implemented just for probing the shader.
The shader-test function defines a test function using the probing shader and the shader under test.
The new test function then can be used using the Midje tabular environment.
In the following example the GLSL function phase is tested.
Note that parameters in the probing shaders are set using the weavejester/comb templating library.
Note that using mget the red channel of the pixel is extracted.
Sometimes it might be more desirable to check all channels of the RGB pixel.
Here is the actual implementation of the tested function:
The empty function (fn [program]) is specified as a setup function.
In general the setup function is used to initialise uniforms used in the shader under test.
Here is an example of tests using uniform values:
Here a setup function initialising 5 uniform values is specified.
Mocking shader functions
If each shader function is implemented as a separate string (loaded from a separate file), one can easily link with mock functions when testing shaders.
Here is an example of a probing shader which also contains mocks to allow the shader to be unit tested in isolation:
Let me know if you have any comments or suggestions.
Rhawk187 pointed out that exact image comparisons are also problematic because updates to graphics drivers can cause subtle changes. This can be adressed by allowing a small average difference between the expected and actual image.
The code of make-vertex-array-object and render-quads is added here for reference.