Automating Static and Dynamic Code Analysis with PlatformIO

As we are working on our agile-embedded toolchain, additional functionality came to our mind, which should be part of every modern development stack. Automated unit and integration testing is surely part of this, as we introdced in our last blog posts, and PlatformIO is a great tool to automate much of it. This posts topic will be about static code analysis and unit test's code coverage, and how this can be integrated and execute automatically upon each build.

Advanced scripting FTW!

Today we discuss a prior mentioned feature of PlatformIO – Advanced Scripting. Last time we introduced some details about its build internals around PlatformIO and SCons. In the end, it's a python-programmable build system. Now, advanced scripting is a super cool feature to execute Python scripts to nearly any time during compilation of your PlatformIO project. Since the basics are already very good explained at PlatformIO’s own blog, I’ll go into specific use-cases instead of the basic setup.

Today we'd like to take care of three use cases, or requirements from our build system.

We're interested in three use cases:

  • First, we want to have static code analysis to find potential bugs which the compiler does not detect. We've chosen cppcheck.
  • Second, we need memory leackage detection. Most of our embedded code is still static, but maybe there are some erroneour mallocs around, so we'd be able to detect them. We're using Valgrind
  • And third, after running unit tests, we want to be able to see code coverage. gcov is our choice here.

All tools are open source and available on all major platforms. If you want to replicate this sample, make sure that you have installed the above mentioned tools which are available through the systems default package manager or via downloads. Or maybe you prefer to compile from source :)

Test code

To begin with, we need some code which forces to trigger some of the tools notifications to even test if our setup works properly. First idea was to extend the unit testing examples from our last blog posts, it's on GitHub, thingforward/unit-testing-with-platformio. Please switch to branch advanced scripting when looking at it using the web ui, or clone the repository using:

$  git clone -b advanced_scripting
$ cd unit-testing-with-platformio
$ git branch -v
* advanced_scripting 15c31d5 initial advanced scripting test

This is how main.cpp looks like: ThingForward > Automating Code Analysis, Debugging and Coverage with PlatformIO > Screenshot from 2017-11-07 10-59-07.png

Integrating the tools

As you see by the #ifdef statements, the project is runnable on a ESP8266 platform but also on in a native environment (ubuntu in our case). To make this possible we have a slightly modified default platformio.ini file:

Image 1

By default, PlatformIO builds all for all mentioned targets specified in this file. (esp12e & native). Native environment has some specific build flags to support code coverage properly. Both have an argument called extra_scripts which points to our Python file. Additionally we support a bunch of build_flags to the native environment. This is where unit tests will run, and we'd like to have code coverage data there. The board does not receive these build flags since coverage etc. is not supported for that platform.

Our Python file looks like the following:

Image 2

The first line imports the Scons environment which we already mentioned in a prior post. To execute specific tasks and hook them during the compilation, we can add pre and post actions to the Scons environment, which you see in line 21 and 22. The first argument is the target (or better: point in time) when to execute the second argument which represents the callback.

There are some built-in targets for PlatformIO which buildprog is one of. It means that runCommonChecks should be executed before the actual program packaging. The second line uses a filepath to the binary of the source code which is going to be compiled. Once the file is created/modified, the PostAction is executed and triggers runNativeChecks. The functions itself are kept quite simple and use Pythons subprocess module to make native system calls. In our cases it calls cppcheck prior the compilation to run Code Analysis on our source code. runNativeChecks is only executed for the native build, since the embedded (esp8266) build won’t create the binary file which triggers the callback. The output is automatically sent in the standard output, which PlatformIO displays via it’s Build Panel.

Test run

Once we start the build, we get the following output, stripped to relevant parts.

valgrind tells us that "1,024 bytes in 1 blocks are definitely lost", so that is the leak we introduces in f().

Image 3 - valgrind

cppcheck reminds us of some code spots where we should have a look at:

Image 4 - cppcheck

Image 5 - cppcheck

And gcov found out that 100.0% of all LOCs are executed. Easy for a simple example, but you get the point.

Image 6 - coverage

Surely you can implement many more useful tools to enhance your work with PlatformIO and automate many test steps which otherwise would be more time consuming if handled manually. With advanced scripting we can modify the SCons work even more by setting custom compilation flags and more. Please take a look at the PlatformIO post linked above about this.