Firmata unit testing

[firmata]

Firmata unit testing

[soundanalogous]

If anyone wants to take this on, please start a discussion here before beginning work.

[mmurdoch]

Firmata can largely be unit tested using the following mechanisms:

• Use Firmata.begin(Stream&) with an in-memory stream to test serial reads and writes
• Attach callback functions for capturing and checking processed input data

The only functions which cannot be tested this way are blinkVersion() and pin13strobe() which directly use Arduino functions (pinMode() and digitalWrite()). Some small refactoring would likely be needed to test them.

Also this method won't test any functionality in StandardFirmata.ino. I'd suggest the best way to get this under test would be to move as much of it into a separate library which can be tested as above. Doing this will likely require wrapping the Arduino functions so that they can be stubbed out for testing. This would make StandardFirmata.ino into a very thin wrapper around this library. There could be overlap with this and the implementation for issue #25 (configurable firmata).

Here's a potential starting unit test sketch using ArduinoUnit (https://github.com/mmurdoch/arduinounit - a library I maintain). I'd be happy to add to these tests if we think this is a sensible approach . All the unit tests are of the form test(<test_name>):

#include <Firmata.h>
#include <ArduinoUnit.h>

// Allows full control of a stream for testing purposes
class InMemoryStream : public Stream
{
public:
  virtual ~InMemoryStream() 
  {
  }

  size_t write(uint8_t val)
  {
    _bytesWritten += (char) val;

    return size_t(1);
  }

  void flush() 
  {
  }

  const String& bytesWritten() 
  {
    return _bytesWritten;
  }

  void nextByte(byte b)
  {
    _nextByte = b;
  }

  int available() 
  {
    return 1;
  }

  int read() 
  {
    return _nextByte;
  }

  int peek()
  {
    return _nextByte;
  }

private:
  String _bytesWritten;
  byte _nextByte;
};

TestSuite suite;

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  suite.run();
}

// Helper function for checking strings
void assertStringsEqual(Test& __test__, const char* expected, const String& actual)
{
  size_t expectedLength = strlen(expected);
  assertEquals(expectedLength, actual.length());
  for (size_t i = 0; i < strlen(expected); i++)
  {
    assertEquals(expected[i], actual[i]);
  }
}

test(beginPrintsVersion)
{
  InMemoryStream stream;

  Firmata.begin(stream);

  char expected[] = 
  {
    0xF9, // Version reporting identifier 
    2,    // Major version number
    3,    // Minor version number
    0
  };
  assertStringsEqual(__test__, expected, stream.bytesWritten());
}

// Helper function to simplify tests
void processMessage(const byte* message, size_t length)
{
  InMemoryStream stream;
  Firmata.begin(stream);

  for (size_t i = 0; i < length; i++)
  {
    stream.nextByte(message[i]);
    Firmata.processInput();
  }
}

// Record digital port and value for checking in a test
byte _digitalPort;
int _digitalPortValue;
void writeToDigitalPort(byte port, int value)
{
  _digitalPort = port;
  _digitalPortValue = value;
}

test(processWriteDigital_0)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, 0, 0 };
  processMessage(message, 3);

  assertEquals(0, _digitalPortValue);
}

test(processWriteDigital_127)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, 127, 0 };
  processMessage(message, 3);

  assertEquals(127, _digitalPortValue);
}

test(processWriteDigitalStripsTopBit)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, B11111111, 0 };
  processMessage(message, 3);

  assertEquals(B01111111, _digitalPortValue);
}

test(processWriteDigital_128)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, 0, 1 };
  processMessage(message, 3);

  assertEquals(128, _digitalPortValue);
}

test(processWriteLargestDigitalValue)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, 0x7F, 0x7F };
  processMessage(message, 3);

  // Maximum of 14 bits can be set (B0011111111111111)
  assertEquals(0x3FFF, _digitalPortValue);
}

test(defaultDigitalWritePortIsZero)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE, 0, 0 };
  processMessage(message, 3);

  assertEquals(0, _digitalPort);
}

test(specifiedDigitalWritePort)
{
  Firmata.attach(DIGITAL_MESSAGE, writeToDigitalPort);

  byte message[] = { DIGITAL_MESSAGE + 1, 0, 0 };
  processMessage(message, 3);

  assertEquals(1, _digitalPort);
}

[soundanalogous]

I think this is a good start. I've created a new branch called unit-tests and created a test directory and unit sub directory. Please open a pull request against unit-tests. Thanks!

[ntruchsess]

I guess we also need test-stubs of Servo, I2C and OneWire etc... to unit-test the more advanced features?

[soundanalogous]

Yeah we need to merge dev into configurable or at least copy the test code to the configurable branch. Then we can add a unit test for each feature class (at least in theory... some features may be more difficult to test than others if not impossible without writing extra methods just to test, but I don't want to go down that route with a microcontroller).