TDD with CMocka

Competitive programming and test memory for faster development process

⌛ 10 min read. 💡 30 min coding

Today I want to solve the problem described here combining CMake, CTests and CMocka frameworks. The function collatz to be coded should return:

collatz(4); //  return "4->2->1"
collatz(3); //  return "3->10->5->16->8->4->2->1"

Download and install CMocka

On Mac you can simply type:

brew install cmocka

Otherwise, you may clone the main CMocka repository on Gitlab.com and run cmake and make install as you’d normally do.

Project setup and first compile

Open you IDE and create new project for C executable. I use CLion and this creates a CMakeLists.txt file automatically which looks like this:

cmake_minimum_required(VERSION 3.17)
project(myproject C)

set(CMAKE_C_STANDARD 99)

add_executable(myproject main.c)

To you CMocka, we need to include its header and link to the library. Your setting may vary, but if you never worked with CMake packages, these lines are the list you can add to your CMakeLists.txt:

find_path(CMOCKA_INCLUDE_DIR
        NAMES
        cmocka.h
        PATHS
        ${CMOCKA_ROOT_DIR}/include
        )

find_library(CMOCKA_LIBRARY
        NAMES
        cmocka cmocka_shared
        PATHS
        ${CMOCKA_ROOT_DIR}/include
        )

if (CMOCKA_LIBRARY)
    set(CMOCKA_LIBRARIES
            ${CMOCKA_LIBRARIES}
            ${CMOCKA_LIBRARY}
            )
endif (CMOCKA_LIBRARY)

target_include_directories(${CMAKE_PROJECT_NAME} PUBLIC ${CMOCKA_INCLUDE_DIR})
target_link_libraries(${CMAKE_PROJECT_NAME} ${CMOCKA_LIBRARIES})

These lines will try to find CMocka and, if successful, link it to your executable. This is all very standard setup and didn’t take long to prepare — nor to compile!

Test-driven development

Background

Looking at initial the assignment, I can define the following tests:

collatz(1); // Returns “2,1”
collatz(-1); // Returns “0”
collatz(4); // Returns “4,2,1”
collatz(3); // Return “3,10,5,16,8,4,2,1”

The main function will contain an array of unit tests to execute tests, in turn these will call the function collatz. Unlike other unit testing frameworks, my understanding is that you can group tests into a function. I will have three of these functions: (i) simple_test for the first two cases, (ii) even_test and (iii) odd_test for the other cases which have a list as output, and we need to take care of memory management.

Add very simple tests

I’ll start writing this list of tests first:

int main() {
    const struct CMUnitTest tests[] = {
            cmocka_unit_test(simple_test),
    };
    return cmocka_run_group_tests_name("success_test" ,tests, NULL, NULL);
}

The last line returns 0 if all tests in the group called passing are successful, or the number of failed tests. All tests are static voids, and I’ll start from simple_test to program the function collatz:

static void simple_test(void **state){

    int sequence[1];

    collatz(1, sequence);
    assert_int_equal(1,sequence[0]);

    collatz(-1, sequence);
    assert_int_equal(0,sequence[0]);
}

All these tests need is a couple of conditional statements to open the collatz function:

static void collatz(const int n, int *sequence){

    if (n <= 0){
        sequence[0] = 0; 
    }
    if (n == 1){
        sequence[0] = 1;
    } 
}

The output shows a tests run and pass successfully:

[==========] success_test: Running 1 test(s).
[ RUN      ] simple_test
[       OK ] simple_test
[==========] success_test: 1 test(s) run.
[  PASSED  ] 1 test(s).

Adding a test with memory allocation

In this project because the length of the output array is not known a priori and that means I can use dynamic memory 🙃. All memory management should be part of the testing suite, but not of the core function collatz and here’s how we can keep things tight and clan with CMocka.

I will add the two units tests to the array of tests:

const struct CMUnitTest tests[] = {
        cmocka_unit_test(simple_test),
        cmocka_unit_test_setup_teardown(even_test, setup, cleanup),
        cmocka_unit_test_setup_teardown(odd_test, setup, cleanup),
};

The cmocka_unit_test_setup_teardown will call setup to allocate the memory processed in the even_test and odd_test which is then released by cleanup. First thing first, let’s look at the setup function:

static int setup(void **state) {
    int *sequence = calloc(5, sizeof(int));
    if (sequence == NULL ){
        return -1;
    }
    *state= sequence;
    return 0;
}

All this function does is allocating a vector of 5 integers based on the assumption that the length of the vector is enough (I’ll address the exception later). The cleanup function will simply free this memory.

The testing function is:

static void even_test(void **state){
    int *sequence = *state;
    collatz(4, sequence);
    int i = 0;

    assert_int_equal(4,sequence[i++]);
    assert_int_equal(2,sequence[i++]);
    assert_int_equal(1,sequence[i++]);
}

The index i is pure demonstration of laziness, while the result sequence 4, 2 and 1 is the result provided in the problem statement. Perhaps a little lengthy and not as elegant as a for loop but simple enough 😐. In order for these tests to pass, I need to add real functionality to collatz:

    sequence[i] = n;
    
    while ( sequence[i] != 1){
        i++;
        if(n % 2 == 0){
            sequence[i] = sequence[i-1]/2;
        }
        else {
            sequence[i] = 3 * sequence[i-1] + 1;
        }
    }

This loop would do, however the else statement is not actually covered because only called for odd values of n. Let’s now move on and test this scenario.

Advanced tests on `realloc` with Valgrind

The function for testing odd numbers is very simple to code, but forn=3the result isn’t good. First because I had a bug in the ifabove — easy enough to fix by replacing n%2 with sequence[i-1] % 2, and second because the resulting sequence is longer than 5 🤯. This means that the program is hungry for memory and the vector of 5 needs to be reallocated. What’s even worse is that if I run the code as it is all tests PASS!

Overrunning is not detected in my tests and the compiler Apple clang version 12.0.0 which I’m using does not complain either. Long story short, the bug remains in the code despite even if all tests pass, but CLion can help to fix it!

There are few icons that should attract your attention:

These are:

Run
Debug
Test coverage (my code is covered at 98%)
Valgrind memcheck
Profile

After some setting, you can run Valgrind and spot the bug. Add few lines for realloc in collatz and the job is done! 😌

Competitive programming and test memory for faster development process