Category Archives: Software Engineering

Regression Testing a Renderer

These are two types of regression test that might be good to do when working on a renderer and you want to be sure that the new code won’t break something else. When doing refactoring, these tests are very important too.

The easy way to do testing is to make some test scenes and manually run them all and look for errors after each change. But this will be cumbersome so more automation is needed.

Regression testing (functionality)

Create a couple of tests with scenes that contains all that should be tested. Run them and manually control so they are working. When running, the test should save a screen shot of the perfect rendered scene for later comparison.

Then a new feature has been added, the test scenes should be automatically run again and new screen shots should be rendered and compared against the original ones. Some sources suggests that the comparison can be a simple bit to bit comparison and that any difference should be noted and reported as a failure.

Regression testing (performance)

Save the fps for all performance tests and whenever a feature has been added. It should automatically run all tests again and report if any performance has changed. It’s most important if performance has decreased, but increased performance might be good to know too if your working on optimizations.

Information about how Unity does testing of their graphics code
http://aras-p.info/blog/2007/07/31/testing-graphics-code/

Some info about how to compare images in these kinds of tests:
http://www.tilander.org/aurora/2008/03/comparing-images.html

A link to information about all kinds of automated testing in games:
http://www.gamasutra.com/features/20050329/roken_pfv.htm

These developers uses a “monkey” to test their code, which seems to work good for them:
http://powerof2games.com/node/25

Some info about what regression testing is and why it is needed:
http://en.wikipedia.org/wiki/Regression_testing