This tutorial will show you how to generate the coverage for sonarqube using g++, cmake, gcovr and gitlab.

Resources

The example cmake project can be found at the following url:

Test Framework

Google Test is the easiest way to generate tests, at least as far as I am concerned. You will need to make some changes to your cmake project but again that’s pretty easy and we’ll start there.

CMake Testing

Create a cmake/tests.cmake and modify as follows

cmake/tests.cmake

if( BUILD_TESTS )

include(CTest)

enable_testing()

set ( TEST_APP ${APP}_test )

include( cmake/test_src.cmake )

add_executable(
    ${TEST_APP}
    ${TEST_SOURCES}
)

include( cmake/test_req.cmake )

include ( cmake/test_link.cmake )

add_test(NAME primary_tests COMMAND ${PROJECT_BINARY_DIR}/${TEST_APP})

endif(BUILD_TESTS)

The one thing to note in the above is APP variable typically points to cmake’s PROJECT_NAME, you can of course name the test app whatever you like. We also link to three other files, cmake/test_src.cmake, cmake/test_req.cmake and cmake/test_link.cmake

The cmake/test_src.cmake just holds the .c, .cpp and .h files that make up the tests. The test_req.cmake finds the test framework and we’ll look at it in the following:

cmake/test_req.cmake

include(FetchContent)

FetchContent_Declare(
googletest
DOWNLOAD_EXTRACT_TIMESTAMP True
URL https://github.com/google/googletest/archive/03597a01ee50ed33e9dfd640b249b4be3799d395.zip
)

# For Windows: Prevent overriding the parent project's compiler/linker settings
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)

FetchContent_MakeAvailable(googletest)

The above just tells cmake to download the google test framework and make it available. The only gotcha is DOWNLOAD_EXTRACT_TIMESTAMP, there appears to be a bug in the FetchContent_Declare and anything after URL gets parsed as part of the url, so make sure to put any options before that.

GTest

You’ll want to create a test for each branch you want to cover, sometimes it’s not practical to test every possible branch but an 80% coverage is diserable and is the default for Sonarqube.

We’ll start with an example test case:


#include <application.hpp>
#include <gtest/gtest.h>
#include <string>
#include <vector>

using namespace gcp;

TEST(Basic_main_no_options, Basics) {
  std::vector<std::string> opts;
  opts = {"basic_no_option_test"};
  std::vector<char *> args;
  for (auto iter = opts.begin(); iter != opts.end(); ++iter) {
    args.push_back((*iter).data());
  }
  auto v = Application::ApplicationMain(args.size(), args.data());

  ASSERT_EQ(v, 0);
}

In this C++ example we have an Application class which is the default entry point called once in the main(). It parses the argc, argv with boost/program_options. In order for this to be a test we include <gtest/gtest.h> and use it’s macros. The test is defined in the macro TEST(Basic_main_no_options, Basics) which just defines a group and title, group being “Basics” in this example.

The actual testing is done by the ASSERT_EQ macro, a list of all the macros available can be found here.

Gcovr

CMake Options

cmake/options.cmake

option( GCOVR "Use gcovr" OFF )

if (GCOVR)
    set(CMAKE_CXX_FLAGS " ${CMAKE_CXX_FLAGS} --coverage")
    set(CMAKE_CXX_FLAGS " ${CMAKE_CXX_FLAGS} -fno-exceptions -fno-inline")
endif(GCOVR)

If you are using C you just need the first flag "–coverage" and append it to CMAKE_C_FLAGS however for C++ you need to disable "-fno-exceptions" and "-fno-inline", if you don’t do this you will probably see a lot of unhandled branches and you coverage will most likely take a sizeable hit.

Gcovr Command

In order to generate your reports you will need to follow a few steps.

  • Generate your cmake project with -DGCOVR=ON and -DBUILD_TESTS=ON
  • Build your project
  • Run the tests, ctest .
  • Call the gcovr executable

The gcovr command has a few options that allow filtering the results, however these need to be combined with the build options, especially for the commercial version of sonarqube as typically you’ll be using the buildwrapper which will pick up spurious branches if you don’t compensate.

Some typical options you’ll want to call are:

gcovr --xml-pretty --exclude-unreachable-branches --exclude-throw-branches --print-summary -o $root/coverage.xml --sonarqube=$root/gcovr/coverage.xml --root $root -e "\.\./src/main.cpp"

The –root=$root needs some expanation, it need to point to the root folder of your source code, gcovr defaults to running in the current folder down, so if you are in a subfolder which is the typical way to build a cmake project you will need to tell gcovr how to index your code hierarchy. The -e “../src/main.cpp” is a regex filter to exclude a match relative to the current working directory. It needs to be escaped and unfortunately doesn’t seem to work with canonical paths. Here I exclude ../src/main.cpp so it’s branches will be minified.

Testing Gcovr

You will probably want to preview your coverage before commiting to git, to do that you just need to add an option to gcovr, –html coverage.html

gcovr --html coverage.html --xml-pretty --exclude-unreachable-branches --exclude-throw-branches --print-summary -o $root/coverage.xml --sonarqube=$root/gcovr/coverage.xml --root $root -e "\.\./src/main.cpp"
firefox coverage.html

Sonarqube

If you are using the commercial version of Sonarqube then the setup is nearly the same as a regular build, you just need to add flags to the compiler and to gcovr itself. The opensource is a lot more complicated but also gives you a lot more tools and refinement.

Example Sonarqube Commercial

sonarqube-main:
  tags:
    - linux
    - docker
    - x64
  stage: sonar
  image: $MYIMAGE
  cache:
    policy: pull-push
    key: "${CI_COMMIT_SHORT_SHA}"
    paths:
      - sonar-scanner/
      - "${BUILD_WRAPPER_OUT_DIR}"
  before_script:
    # gcovr
    - apt -y install gcovr curl unzip
    # Download sonar-scanner
    - curl -sSLo sonar-scanner.zip "https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-${SONAR_SCANNER_VERSION}-linux.zip"
    - unzip -o sonar-scanner.zip 
    - mv sonar-scanner-${SONAR_SCANNER_VERSION}-linux sonar-scanner
    # Download build-wrapper
    - curl -sSLo build-wrapper.zip "${SONAR_URL}/static/cpp/build-wrapper-linux-x86.zip" 
    - unzip -o build-wrapper.zip
    - mv build-wrapper-linux-x86 build-wrapper
  script:
    - export MINOR=`cat VERSION | awk '{ split($1,m,"."); print m[1]"."m[2]}'`
    - echo "Version - $MINOR"
    - echo "\nsonar.projectVersion=$MINOR" >> sonar-project.properties
    - root=$PWD
    - cmake -DCMAKE_BUILD_TYPE=Release -DGCOVR=ON -DBUILD_TESTS=ON .
    - make -j ${THREADS}
    - ctest --output-junit $root/gtest-report.xml .
    - mkdir -p $root/gcovr
    - gcovr --xml-pretty --exclude-unreachable-branches --exclude-throw-branches --print-summary -o $root/coverage.xml --sonarqube=$root/gcovr/coverage.xml --root $root  -f "src/" -f "include/" -e "src/main.cpp"
    - build-wrapper/build-wrapper-linux-x86-64 --out-dir "${BUILD_WRAPPER_OUT_DIR}" make -j $THREADS
    - sonar-scanner/bin/sonar-scanner -Dsonar.host.url="${SONAR_URL}" -Dsonar.cfamily.build-wrapper-output="${BUILD_WRAPPER_OUT_DIR}"
  artifacts:
    name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA}
    expire_in: 2 days
    reports:
        coverage_report:
            coverage_format: cobertura
            path: coverage.xml
        junit: gtest-report.xml
  variables:
    BUILD_WRAPPER_OUT_DIR: bw
    SONAR_SCANNER_VERSION: 4.8.0.2856
    THREADS: 4
    MYIMAGE: example.com/build-servers/gcc-cmake:latest
    GIT_STRATEGY: clone
    GIT_DEPTH: 0

Example Sonarqube OpenSource

sonar-linux-x64-local:
    stage: sonar
    image: ubuntu:22.04
    tags:
        - linux
        - docker
        - x64
    #when: manual
    allow_failure: true
    before_script:
        - export TZ=Europe/Dublin
        - ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
        - apt update && apt -y install gcc g++ make cmake ssh git gcovr
        - apt -y install libboost1.74-all-dev libgcrypt20-dev
        - apt -y install unzip curl wget
        - apt -y install googletest libgtest-dev
        - apt -y install vera++ cppcheck valgrind
        - apt -y install python3-pip clang clang-tools
        - mkdir -p .sonar/bw-output
        - curl -sSLo sonar-scanner.zip https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.7.0.2747-linux.zip
        - unzip -o sonar-scanner.zip -d .sonar
    script:
        - mkdir valgrind && cd valgrind
        - cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON ..
        - make
        - valgrind --xml=yes --xml-file=valgrind.xml $PWD/GenericCmakeProject_test || e=$?
        - cd ..
        - mkdir clang && cd clang
        - cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
        - analyze-build-14 -o reports -plist --analyze-headers
        - cd ..
        - root=$PWD
        - cp -f sonar.ignifi sonar-project.properties  
        - export MINOR=`cat VERSION | awk '{ split($1,m,"."); print m[1]"."m[2]}'`
        - echo "\nsonar.projectVersion=$MINOR" >> sonar-project.properties   
        - mkdir buildgcov && cd buildgcov
        - cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ENCDEC=ON -DBUILD_TESTS=ON -DGCOVR=ON ..
        - make
        - make test
        - ctest --output-junit $root/gtest-report.xml .
        - mkdir -p $root/gcovr
        - gcovr --xml-pretty --exclude-unreachable-branches --exclude-throw-branches --print-summary -o $root/coverage.xml --sonarqube=$root/gcovr/coverage.xml --root $root -e "\.\./src/main.cpp"
        - cd ..
        - rm -Rf buildgcov
        - cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_CXX_CPPCHECK=cppcheck .
        - make cppcheck-analysis 2> /dev/null || e=$?
        - cat cppcheck.xml || e=$?
        - .sonar/sonar-scanner-4.7.0.2747-linux/bin/sonar-scanner -Dsonar.login=$SONAR_TOKEN_LOCAL -Dsonar.host.url=$SONAR_HOST_URL_LOCAL -X -Dsonar.verbose=true
    coverage: /^\s*lines:\s*\d+.\d+\%/
    cache:
        paths:
            - .sonar
    artifacts:
        name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA}
        expire_in: 2 days
        reports:
            coverage_report:
                coverage_format: cobertura
                path: coverage.xml
            junit: gtest-report.xml
    variables:
        THREADS: 4
        GIT_DEPTH: 0
        GIT_STRATEGY: clone