Projects & Blog | TESTING

A Quick Guide On Software Testing


In a software development project, errors can appear in any of the stages of the life cycle, the later they appear the more effort is needed to fix those errors. So testing can avoid errors in later stages and thus reduce costs. But wait, there are much for more benefits!

Why do I have to test my code? I Know that it works

Thats a question unexperienced developers may ask. In fact I have often heard that question. We already covered the fact that finding errors in early stages helps to reduce costs. But lets see what other benefits there are, the most important ones are:

Different types of tests

Unit Tests: Unit tests are low-level tests that focus on testing a specific part of the Software. They are usually written very fast with low effort. Test failures should provide enough information to find the source of an occuring error.

Integration Tests: The compatibility and functionality of the interfaces between the different "units" that make up a system are tested.

Validation Tests: These are the tests performed on fully integrated software to evaluate the fulfillment specified requirements.

System Tests: the validated software is integrated with the rest of the system and tested.

Unit tests are cheap to write and easy to automate. Additionaly, the benefit of unit tests is very high. Thats why the number of unit tests should and also generally exceeds the number of other test types.

Tests on the bottom of the pyramide are easy to automate, the ones further to the top are more difficult to automate.


Properties of good tests are: They are easily automated, they run fast and they are deterministic.

When should I use tests?

Short answer: Always!

Long answer: Almost always, I would say that you should use automated tests (at least unit tests) for any software that exceeds a size of 5 files or 500 lines of code. Yes that means that almost any software should be tested. Once you learned how to setup tests and automated build systems, the cost of doing so becomes very very low. However, the befenits grow the bigger and the longer your software lives.

Common software testing tools & How do tests look like?

Now, lets see how we can acutally write and run tests in some of the most commonly used programming languages:


Java has a very powerfull testing environment integrated in the standard tools called JUnit. It is very commonly used all around the Java ecosphere.

A typical unit test can look like the following. This unit test checks some basic geometric math functionality between triangles and lines.

    * Test method for
    * {@link Triangle#lineIntersections(ILine)}
public void testLineIntersections() {
    Vector2 a = Vector2.fromXY(0.0, 0.0);
    Vector2 b = Vector2.fromXY(1000.0, 0.0);
    Vector2 c = Vector2.fromXY(0.0, 1000.0);
    Triangle triangle = Triangle.fromCorners(a, b, c);

    ILine line1 = edu.tigers.sumatra.math.line.Line.fromPoints(Vector2.fromXY(1, 1), Vector2.fromXY(2, 2));

    ILine line4 = edu.tigers.sumatra.math.line.Line.fromPoints(Vector2.fromXY(0, 500), Vector2.fromXY(500, 500));

    ILine line2 = edu.tigers.sumatra.math.line.Line.fromPoints(Vector2.fromXY(-1, 0), Vector2.fromXY(-1, 0));

    ILine line3 = edu.tigers.sumatra.math.line.Line.fromPoints(Vector2.fromXY(0, -1), Vector2.fromXY(1, -1));


In python the unittest module is very common for writing unit tests.

import unittest

class TestSum(unittest.TestCase):

    def test_sum(self):
        self.assertEqual(sum([1, 2, 3]), 6, "Should be 6")

    def test_sum_tuple(self):
        self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")

if __name__ == '__main__':



Sadly the C/C++ environment does not offer any standard tools for testing purposes. However, there are still some widely used third party tools. One of them is googletest,


#include "gtest/gtest.h"
#include "../sim/PowerEstimation.h"
#include "../model/Vector3.h"
#include "../model/Vector2.h"
#include "../model/Receiver.h"
#include "../model/Heliostat.h"

class PowerEstimationTest : public ::testing::Test



    ~PowerEstimationTest() override

    void SetUp() override

    void TearDown() override


TEST_F(PowerEstimationTest, TestHeliostatEfficiencyCalculation)
    PowerEstimation estimation{};

    Receiver receiver{Vector3(0,0,60)};

    Heliostat heliostatBadAngleAndFar{Vector3(0,500,5), Vector2(0,0)};
    Heliostat heliostatMid{Vector3(0,100,5), Vector2(0,0)};
    Heliostat heliostatGoodAngle{Vector3(0,10,5), Vector2(0,0)};

    Vector3 sunVector(0,0,1);

    double efficiencyLow = estimation.estimateHeliostatEfficiency(sunVector, heliostatBadAngleAndFar, receiver);
    double efficiencyMid = estimation.estimateHeliostatEfficiency(sunVector, heliostatMid, receiver);
    double efficiencyHigh = estimation.estimateHeliostatEfficiency(sunVector, heliostatGoodAngle, receiver);

    ASSERT_LE(efficiencyMid, efficiencyHigh);
    ASSERT_LE(efficiencyLow, efficiencyMid);


Continuous Integration (CI)

Continuois Integration not only consists of automated testing, but also building, integrating, testing and reporting.

There is a very wide range of different tools and frameworks for CI. In this blog post I will talk more about two of them. Jenkins and the Gitlab envrionment.


The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project. Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

Read more about Jenkins on their webiste:


Besides Gitlab being a very powerful git hosting and issue tracking platform, it also comes with a built in CI environemnt. Cool! (Its my absolute favorite tool!)

The following source code shows a rather complex gitlab-ci yml file (the .yml file define how the build server should build and test the project) that builds and tests a core software written in C++ (with a cmake build system), a GUI software written in java (with a maven build system). The software is built within a docker container. Furthermore, an asciidoc documentation is being built. Thus, this a very powerful setup!

image: fluxsim

  - build
  - test
  - documentation

  stage: build
    - linux
    - fluxsim
    - cd core
    - mkdir -p build && cd build
    - cmake ..
    - make
      - ./core/build/core

  stage: build
    - linux
    - fluxsim
    - cd gui
    - mvn package
      - ./gui/target/*.jar

  stage: documentation
    - linux
    - fluxsim
    - asciidoctor-pdf -r asciidoctor-diagram ./documentation/documentation.adoc
    - mv ./documentation/documentation.pdf ./
      - ./documentation.pdf

  stage: test
    - linux
    - fluxsim
    - cd core
    - mkdir -p build && cd build
    - cmake .. -DUNIT_TEST=ON
    - make
      junit: $CI_PROJECT_DIR/core/build/test/unit_test_report.xml

Gitlab uses so called "gitlab-runners" to execute the build pipeline.

Test Driven Development

What is it? Straight from wikipedia:

Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved so that the tests pass.

Pratically this means that the test cases are written and implemented before the actualy functionality. This will lead to a "failing" test at first. After the implementation of the test case the developer starts to implement the actual feature. Until the test fails no more. TDD can lead to more modularized, flexible, and extensible code.