Mocking Your Way to the Bottom

Unit testing is finally making its way into embedded development, though most see testing as an impediment to completion of a task. Mocks provide a lightweight mechanism to simulate interactions with lower layers of the system. This empowers us to do simultaneous top-down design and development. We will take attendees through the full top-down implementation of a full project on an Arduino armed with only the C language, the GCC toolchain, and Ceedling, our bundle of free and open-source tools. Lucky attendees will receive an Arduino starter kit with all the goodies to replicate this project and take it home.

Presented at Design East (Boston, MA), by Greg Williams and Jordan Schaenzle.

Transcript

1. 10/1/12 1 Embedded Development… •  Limited Memory •  Limited Processing Power •  Language Limitations •  Short Timelines •  Growing Complexity Non-Standard •  Hardware Is HARD
2. 10/1/12 2 What Can We Do? •  Give up? •  Pray? •  Complain? •  Get a new job?!? •  Get better! •  Be responsible! There is NO “Silver Bullet”!!!
3. 10/1/12 3 Success in Testing •  Smart and Capable People •  Good Process •  Tools to Support Good People and Good Process Validation •  Need to verify we have done the right thing •  What is the right thing? •  How do we check it? •  Will it always work? •  Is it worth it? •  But it takes to much F!$%#$G time!!!
4. 10/1/12 4 Manual Validation •  Real Deal / Simulation 1. Stimulate conditions / Modify state 2. Run to breakpoint 3. Check resultant state 4. Tedious 5. Painful •  Usually do it ONCE Automation •  Buy a robot!!! •  Create test plans •  Buy fancy hardware •  Spend a lot of time figuring out what and how •  How can we automate everything?? •  $$$$$$$
5. 10/1/12 5 Unit Testing •  Focus on testing individual modules and functions •  Verify that a given scenario produces the correct result •  Ensures building blocks perform specific operations according to their design •  Drives toward proper encapsulation Design can evolve naturally… Unit Testing •  Automatable •  Provides living documentation of design •  Instant regression testing •  Facilitates refactoring •  Eliminate dead code •  Eliminates: “Don’t fix it if it ain’t broke” or “We’ll fix that next time we touch it” •  Increases sanity across the board!
6. 10/1/12 6 What is TDD? •  What do we mean by Test-Driven Development? •  Mindset of Maximizing Testability •  First executable code is TEST code •  Write JUST ENOUGH code to satisfy tests The TDD Cycle 1. Select a Feature to Implement 2. Write a Small Test 3. Execute Test and Watch It Fail 4. Implement source code to satisfy test 5. Correct until the tests pass 6. goto 1.
7. 10/1/12 7 Types of Tests •  Unit Tests •  Integration Tests •  System Tests Unit Tests •  Unit Tests •  Exercise a unit of source code •  Tests unit in isolation from surroundings
8. 10/1/12 8 Integration Tests •  Same fundamentals as unit tests, but exercises a group of modules / subsystem System Tests •  Run against full application on target hardware •  Exercises behavior that unit/integration tests cannot •  Ideally targeted at validating FEATURES •  Sanity checking of “the system” •  NOT for validating every possible variation
9. 10/1/12 9 Benefits of TDD •  Tested Software •  High Level of Code Coverage •  Full Coverage Measured by Coverage Tools •  Well-Designed Software •  Well-Documented Software •  Maintainable Software •  Sanity!!! Mechanics:  Testable  Project   Mechanics: Testable Project source test source source test test artifact fixture fixture fixture results PRODUCTION TEST SUITE UNIT TESTS source test fixture
10. 10/1/12 10 Where to start… •  Project Scope: •  Measure temperature (thermistor voltage) and report degrees Celsius once per second over serial port Time to plant a Ceedling! > gem install ceedling > ceedling new MyProjectProject 'MyProject' created! - Tool documentation is located in vendor/ceedling/docs - Execute 'rake -T' to view available test & build tasks > cd MyProject> rake test:delta ------------------------- OVERALL UNIT TEST SUMMARY ------------------------- No tests executed. >
11. 10/1/12 11 Feature Request Read Analog to Digital Converter X times per second Let’s Go! 1.  Dig through datasheet 2.  ADC_Init() •  Setup ADC in proper mode 3.  ADC_Read() •  Trigger a conversion and wait for completion Return the results 4.  Wire it into the system •  We still need to time the samples 5.  WAIT!!! 6.  Where do the results go? Hmmmm......
12. 10/1/12 12 Feature-Driven Development •  “We need to read something from and ADC converter, so let’s write a driver!!” •  NOOOOOOOOOOOOOOOOOOO •  Leads to cluttered APIs and DEAD CODE! •  Focus on what is required NOW, and implement ONLY THAT •  Use a top-down approach, discovering needs along the way Feature-Driven Development (…continued) •  Software IS features •  Customers pay for features, NOT infrastructure •  Infrastructure-First => WASTE
13. 10/1/12 13 Feature-Driven Development (…continued) •  Feature-Driven Development •  Minimally working system as soon as possible •  Build towards feature completion •  Tests are a safety net while refactoring •  Features •  Yield meaningful progress metrics •  Satisfied project managers AND developers •  Keep developers focusing on “the big picture” Top-Down Implementation •  Mocks to the rescue! •  CMock generates mocks using only header files (INTERFACES) •  The lower levels need not be implemented AT ALL!! •  Leads to easy refactoring of interfaces prior to implementation of underlying code
14. 10/1/12 14 Mocking with CMock •  Creates mocks of modules using only the header files (interfaces) •  Utilizes Ruby to make the magic happen •  Creates stand-ins for real methods •  Creates helper methods for setting expectations •  Verifies interactions with other modules and libraries CMock Example int      ParseStuff(char*  Cmd);   void    HandleNeatFeatures(NEAT_FEATURE_T  NeatFeature);   int    ParseStuff(char*  Cmd);   void  ParseStuff_ExpectAndReturn(char*  Cmd,  int  toReturn);   void  ParseStuff_IgnoreAndReturn(int  toReturn);   void  ParseStuff_StubAndCallback(CMOCK_ParseStuff_CALLBACK  Callback);       void  HandleNeatFeatures(NEAT_FEATURE_T*  NeatFeature);   void  HandleNeatFeatures_Expect(NEAT_FEATURE_T*  NeatFeature);   void  HandleNeatFeatures_ExpectWithArrays(NEAT_FEATURE_T*  NeatFeature,                      int  NeatFeature_Depth);   void  HandleNeatFeatures_Ignore(void);   void  HandleNeatFeatures_StubAndCallback(      CMOCK_HandleNeatFeatures_CALLBACK  Callback);  
15. 10/1/12 15 Example Test Case void  test_MyFunc_should_ParseStuffAndCallTheHandlerForNeatFeatures(void)   {          NEAT_FEATURES_T  ExpectedFeatures  =  {  1,  "NeatStuff"  };              ParseStuff_ExpectAndReturn("NeatStuff",  1);          HandleNeatFeatures_Expect(ExpectedFeatures);              //Run  Actual  Function  Under  Test          MyFunc("NeatStuff");   }   Example: Function Under Test void  MyFunc(char*  Command)   {      int  ID;      NEAT_FEATURES_T  Neat;          ID  =  ParseStuff(Command);      switch(ID)      {          case  0:                  HandleStupidFeatures();              break;          case  1:              Neat.id  =  1;              Neat.cmd  =  Command;              HandleNeatFeatures(Neat);              break;          default:              break;      }   }  
16. 10/1/12 16 Anatomy of a Ceedling Test void foo(void) {...} int bar(char) {...} SOURCE(S) void test_bar(void) { ASSERT( 5, bar(‘a’)); } TEST void main(void) { setUp(); test_foo(); tearDown(); } RUNNER int foo(int) {...} void foo_Expect(void) {...} MOCK(S) +! +! +! = TEST FIXTURE EXECUTABLE int ASSERT(int, int) {...} FRAMEWORK +! Ceedling Quick Reference •  rake -T •  List all tasks •  rake test:my_module •  Test the specified module •  Also can specify test, header or source •  rake test:all •  Test all modules •  rake test:delta •  Test changes (incremental)
17. 10/1/12 17 Questions? Let’s get our hands dirty!!
18. 10/1/12 18 Feature Request #1 •  Blink onboard LED at 1Hz rate •  Specs •  Use 50% duty cycle •  LED located on PortB at bit 5 •  Use PORTB5 definition •  Use Timer0 Overflow Vector •  Configure for 1ms period Feature Request #2 •  Implement a ASCII command using the USB-to-serial interface that will change the blink rate on the LED
19. 10/1/12 19 Feature Request #2 - Spec •  Command •  blink <period_ms> •  <period_ms> •  period of a blink in ms (50% duty cycle) •  supported range: 2 - 2000ms •  if no argument specified, blinking will halt •  Response •  <OK> - success •  <period out of range> - period_ms out of range •  <invalid format> - command could not be parsed Links •  Tutorial Project •  http://github.com/ThrowTheSwitch/blinky •  ThrowTheSwitch •  http://throwtheswitch.org •  Atomic Embedded •  http://atomicembedded.com •  Atomic Spin Blog •  http://spin.atomicobject.com