Unit Testing and Arduino

When I worked in software, I was an early adopter of agile software development (eXtreme Programming) and so have always loved unit tests and TDD. Like many people from a software background, when I first started programming Arduino, my instinct was to write frameworks, patternize and objectify my code. Then I realised that I have 32k of program space to play with and a really big program might stretch to 100 lines of code (shock horror). So I adjusted my big-software thinking and like everyone else drew the line at functional decomposition and occasional library writing.

Background

For the fun of it, I recently decided to make a calculator (pocket calculator if you happen to have huge pockets) from the schematic design up.

For this project I wanted to use my new favourite low cost microcontroller (the ATTINY816) with Arduino IDE using Spence Konde's rather fantasic Arduino core.

To my surprise, making a calculator that actually does arithmetic well is a lot harder than expected. One obvious hurdle is that float in Arduino C is only 32 bits. So once your numbers get to about 7 or 8 digits they become disconcertingly approximated. So, I thought, I'm (or used to be) a computer scientist, I'll just implement my own floating point representation with more precision than you can shake a stick at. So, I did, as a C++ library (I'll put it on Github when I've got it working and tidied up).

However, for the first time, since I had first started using Arduino, I felt the need to write unit tests, to make sure that my number class was doing arithmetic and generating an 8 character string plus decimal point position information that I could then easily map onto an 8 digit 7-segment display.

Unit Tests

If you haven't used unit testing before, then the basic idea is that you write a load of test functions (the more the better) that exercise the code being tested in some way and compare the outcome with the expected outcome. So, for example in a number class, you might want to check that when you add 2 and 2 you get 4, but also you write a test to make sure that when you add -123.45 and 0.01 you get -123.44 and not -123.46.

Having a unit test suite behind your code means that if you change your code to fix one bug, when you run the tests it will immediately tell you if you have broken something else in doing so. As you build up your suite of tests, you get more and more test coverage for the tricky situations that may uncover bugs.

My Solution for Arduino

The solution I came up with is very specific to the problem, so, I'll try and include some general principals rather than just the code.

Firstly, I created a separate test sketch, specifically for the purpose of testing my number class, without any of the other code related to keyboard scanning and display refresh.

The second thing I did was to get out an Arduino Uno (well actually I used a MonkMakesDuino) because one of the great things about the Uno, is that compiling and uploading a sketch is MUCH quicker than the likes of an ESP32 or indeed the ATTiny816 programmed using UPDI. So, the round-trip time when adding tests or fixing code is greatly reduced.

I used the Serial Monitor to view the results of the unit tests, and a test pass would simply marked by a single line giving the name of the test that passed. A test failure, would include as much information as possible to help with debugging.

Here's the start of my test sketch:

#include "CalcNum.h"

char digits[] = "........";
const int numDigits = 8;

void setup() {
  Serial.begin(9600);
  testNums();
  testAdd1();
  testAdd2();
  testSub1();
  testMult1();
  testMult2();
  testMult3();
  testDiv1();
  testDiv2(); 
}

void loop() {}

My number class is CalcNum (imaginative right!).
digits[] is a data structure used by CalcNum in its writeDigits() method that prepares a string for easy mapping onto an 8 digit 7-segment display.

All the test functions to be called are then listed out in the setup function, as we only need to run them once.

The first of these (testNums()) tests the representation of numbers themselves rather than arithmetic, so lets skip on to the test function testAdd1():

void testAdd1() {
  CalcNum x = CalcNum(12, 0);
  CalcNum y = CalcNum(3, 0);
  CalcNum z;
  CalcNum::add(x, y, &z);
  test("testAdd1", z, "      15", 7);
}

This function defines two numbers (x and y) using an exponent form (x = 12 x 10^0 = 12).
Adds them together and then calls the general purpose function test to see if the result was as expected.

As an aside, I haven't used C++ operator overloading in my number class, as this would inevitably lead to the need for dynamic memory allocation, which I avoid like the plague when working on embedded systems.

So, what are the parameters to test?

The first is a string, that is the name of the test, the second is the CalcNum to be checked. the third is the expected result string from calling writeDigits() - in this case 15 with leading spaces. The final parameter is the expected position of the decimal point on the display (zero indexed, left to right).

Here's what the function test looks like:

void test(char *testName, CalcNum z, char *expected, int expectedDP) {
  z.writeDigits(numDigits, digits);
  int dp = z.dpPos(numDigits);
  if (strcmp(digits, expected) == 0 && dp == expectedDP) {
    pass(testName, expected, expectedDP, z);
  }
  else {
    fail(testName, expected, expectedDP, z);
  }
}

As you can see, the test function compares the calculated and expected 8 digit string and decimal point positions and if the match, calls pass and if they don't calls fail.

void pass(char *testName, char *expected, int expectedDP, CalcNum z) {
  Serial.print("PASS: "); Serial.println(testName);
  //report(expected, dp, expectedDP, z);
}

The function pass just prints out a message that the test passed, along with the name of the passing test. Note the commented out call to report. Sometimes this gets commented back in to shed light on why one test passed when another didn't.

The fail function is much the same as pass, but with a different starting message.

void fail(char *testName, char *expected, int expectedDP, CalcNum z) {
  Serial.print("**** FAIL: "); Serial.println(testName);
  report(expected, expectedDP, z);
}

The report function just prints out as much useful information about the result as possible, to help me fix the bug.

void report(char *expected, int expectedDP, CalcNum z) {
  Serial.print("got digits["); Serial.print(digits); Serial.print("]dp="); Serial.print(z.dpPos(numDigits));
  Serial.print("\t expected ["); 
  Serial.print(expected);Serial.print("]dp="); Serial.println(expectedDP);
  Serial.print("float: "); Serial.println(z.toFloat(), 10);
  Serial.print("m: "); Serial.print(z.m); Serial.print(" e:"); Serial.println(z.e);
  
  Serial.println();
}

Conclusion

Once this project is written, I probably won't write any more tests until I meet a similar project for which 'it seems to work ok' is not sufficient.

However, its really easy just to put together some tests if you need to. For me, it wan't even worth looking to see if anyone had made a framework to do this and then taking the trouble to work out how to use it.

I hope this write-up will help you if you find yourself needing some Arduino unit tests!