Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Agile Testing for Large Embedded Systems

Agile Testing for Large Embedded Systems

Approaching TDD for Large Embeded systems. Provides a list of Gotchas and Fixes for those problems. Speakers Notes included.

Rich Dammkoehler

May 01, 2015
Tweet

More Decks by Rich Dammkoehler

Other Decks in Technology

Transcript

  1. H T T P : / / S D .

    K E E P C A L M - O - M AT I C . C O . U K / I / K E E P - C A L M - I T S - J U S T- A - T H E S I S . P N G H T T P : / / M E D I A . T U M B L R . C O M / 9 9 B 6 A 6 6 1 8 F E C 0 B 2 B 2 1 E 3 5 E F D A D 6 A 5 E 5 1 / T U M B L R _ I N L I N E _ M O I B O 6 X V C R 1 Q Z 4 R G P. J P G Thesis: Effective testing for embedded systems requires many layers. Layering tests enables us to verify that our software works sooner, tightening the feedback loop and letting us move more quickly. Additionally, the consequent modularized code narrows the focus of where the issues can be found. What we are going to cover - What is testing? - Gotchas & Remedies - How do I apply it? - Supporting Activities http://sd.keepcalm-o-matic.co.uk/i/keep-calm-its-just-a-thesis.png http://media.tumblr.com/99b6a6618fec0b2b21e35efdad6a5e51/tumblr_inline_moibo6xvcr1qz4rgp.jpg
  2. H T T P : / / 2 3 A

    X Y 6 K 2 Q 8 VA FA J U . Z I P P Y K I D . N E T D N A - C D N . C O M / W P - C O N T E N T / U P L O A D S / 2 0 1 4 / 0 6 / T E S T I N G - D A RT H - VA D E R . J P G So what is testing. It takes many forms, but for our purposes we will discuss the application of Acceptance Test Driven Development, Test Driven Development, Integration Testing, Bench Testing, and Performance/Load testing. http://23axy6k2q8vafaju.zippykid.netdna-cdn.com/wp-content/uploads/2014/06/testing-darth-vader.jpg
  3. • ATDD H T T P : / / A

    P L E A S U R E T O T E A C H . F I L E S . W O R D P R E S S . C O M / 2 0 1 3 / 0 6 / S T O P C O L L A B O R AT E . J P G Acceptance Test Driven Development is a form of Test First that concentrates on collaboration between the Business and the Development team. The business is typically represented by a product owner or systems engineer who understands the objective of the work being done. The development team has testers and developers. These players can get together to design acceptance tests that will satisfy the high level criteria the Product Owner is looking to satisfy. This is called the Three Amigo’s process. The key point is Developer-Tester-Business collaboration for the creation of tests. This ensures that the developers understand and have written down the goal of the work. Typically this is done in the form of scenarios for each feature. The developed then work with the testers to automate as much of these tests as possible. http://apleasuretoteach.files.wordpress.com/2013/06/stopcollaborate.jpg
  4. • TDD TDD H T T P : / /

    W W W. S C I O N . C O M / S C I O N R A C I N G / W P - C O N T E N T / U P L O A D S / 2 0 1 3 / 0 6 / F L O R I D A - F R I D AY- 3 8 1 . J P G Test Driven Development (TDD) is a software development process that relies on a tight cycle of feedback to organically grow the code to meet the requirements of the system. TDD has a significant impact on the design of software, particularly in its modularity. Simply stated TDD has 3 high level steps, Red-Green-Refactor. The idea is that you create a test before doing anything else. If you can’t think of what the test should be, you don’t know what your doing and more thinking and questions need to be answered. But once you know what test you want, write that test and execute it. It will run Red. Now write just enough code on the other side of the test to make it pass. This is your production code. Don’t add anything unnecessary to the code, only what is required to pass the test. Now clean up. Modify the design so that it is clean, format your code, make sure the design is as good as it can be. Now repeat. A key aspect of this is to keep the tests as simple as possible. No test should make more than one assertion if possible. That means there will be lots of tests. When applying TDD to embedded systems it is best to isolate the code under test in to modules. Keep the business code isolated from the hardware specific code. Within the business code, isolate presentation code from business logic, and business logic from any kind of persistence or I/O code. The smaller the modules the better off you will be. Eventually you will integrate all these small modules back together to create the whole application. http://www.scion.com/scionracing/wp-content/uploads/2013/06/florida-friday-381.jpg
  5. • Integration H T T P : / / A

    D M I N . U T E P. E D U / P O RTA L S / 1 8 7 8 / I N T E G R AT I O N % 2 0 T E S T I N G % 2 0 P I C . J P G T E S T I N G T H E A G G R E G AT E When you get ready to combine the individual modules together you will need to define higher level tests that confirm the aggregation of working components work together as expected. Often times these tests can be written before the lower level modules even exist. The might take longer to get running green, but they act as a guide post for completing development on a feature of behavior. Otherwise the process is identical to TDD. You are combining just enough parts together to meet a specific objective. Each test should have only only one assertion that it validates. Once we have verified that the code works correctly in our development environment we need to start moving it toward the target platform. There any may pitfalls and gotchas involved here, and we will talk about those soon. But before we do, lets talk about some more specialize testing that can help us ensure we’ve created the highest quality software possible. http://admin.utep.edu/Portals/1878/integration%20testing%20pic.jpg
  6. • Bench Testing H T T P : / /

    P E S W I K I . C O M / I M A G E S / 3 / 3 0 / M P I _ T E S T _ B E N C H _ 6 0 0 . J P G B E N C H T E S T I N G When hardware isn’t readily available, or loading to target is expensive/difficult, we can use a bench test to simulate on-target. This allows us to confirm that the components work as expected together with more realistic hardware before the expense of going to target. We can also employ external monitors at this point to help us measure the behavior of the code. This is our first stop on deterring if the application will work correctly on the target hardware. Its a great place to discover issues and significantly cheaper than full deployment. http://peswiki.com/images/3/30/MPI_test_bench_600.jpg
  7. • NRL/RL Testing H T T P : / /

    M E D I A 4 . S - N B C N E W S . C O M / I / N E W S C M S / 2 0 1 4 _ 1 2 / 2 5 7 9 8 6 / 1 4 0 3 1 7 - S I M U L AT O R - J H C - 1 3 4 5 _ 0 B A C D 5 0 4 D 6 B A 0 2 9 3 2 3 E A A B 5 5 4 A 1 8 3 5 C 6 . J P G H T T P : / / W W W. A L A S K A - I N - P I C T U R E S . C O M / D ATA / M E D I A / 2 0 / M I L I TA RY- J E T- C O C K P I T _ 4 4 8 3 . J P G S I M U L AT I O N & R E A L W O R L D T E S T I N G Another form of testing we can do is with simulators. Simulators can help us to understand how our software is going to respond to near real life conditions. We can integrate a significant amount of software and hardware together without going to the real target. Verify that everything is working, and then move to the target hardware. Once we’ve verified that the system works on our simulator we can move to the real target environment. At this point we should be looking closely at the features of the system that could not be emulated in a lab setting. Much of our work should already be verified as working correctly. http://media4.s-nbcnews.com/i/newscms/2014_12/257986/140317-simulator-jhc-1345_0bacd504d6ba029323eaab554a1835c6.jpg http://www.alaska-in-pictures.com/data/media/20/military-jet-cockpit_4483.jpg
  8. • Load and Performance H T T P : /

    / T O O L S . U T E S T. C O M / W P - C O N T E N T / U P L O A D S / S I T E S / 3 / 2 0 1 4 / 0 7 / W T 4 - L O A D - T E S T- A N D - R E C O R D - W E B S I T E - P E R F O R M A N C E - O V E R - T I M E . J P G P E R F O R M A N C E T E S T I N G For some components, performance matters. Simulating performance can be very difficult. Software simulations of CAN traffic often cannot emulate real life. Finding creative ways to simulate reality can be daunting. Being sure the logic in the code is valid before attempting the solution can save a lot of time. If the logic is correct, and guarded by tests, performance modifications can be done without risk of breaking that logic. http://tools.utest.com/wp-content/uploads/sites/3/2014/07/wt4-load-test-and-record-website-performance-over-time.jpg
  9. • Gotchas and Remedies H T T P : /

    / I M G . W O N K E T T E . C O M / W P - C O N T E N T / U P L O A D S / 2 0 1 4 / 0 8 / S N A K E - O I L . J P G So lets talk about some of the gotchas that can occur when developing embedded systems and address them with a few key fixes. You will see that these remedies aid in resolving more than one of the gotchas. http://img.wonkette.com/wp-content/uploads/2014/08/snake-oil.jpg
  10. • Hardware isn’t available/scarce/expensive • Dual targeting • Virtualization •

    Bench Testing • 5 Stage Dev Cycle H T T P : / / W W W. O X FA M B L O G S . O R G / F P 2 P / W P - C O N T E N T / U P L O A D S / WAT E R - S C A R C I T Y. B M P G O T C H A : H A R D WA R E TA R G E T S C A R C I T Y Scarcity of Hardware to test on Causes - - Not ready yet - Too expensive What to do? http://www.oxfamblogs.org/fp2p/wp-content/uploads/water-scarcity.bmp
  11. • Dual Targeting H T T P : / /

    I M G 1 . W I K I A . N O C O O K I E . N E T / _ _ C B 2 0 1 3 0 3 1 1 0 5 1 3 3 8 / S TA R WA R S / I M A G E S / B / B C / T I E _ TA R G E T _ F F 7 . P N G F I X : D U A L TA R G E T I N G Dual Targeting Code designed on day 1 to run on 2 platforms Eliminates the risk and waste of creating un-verified work Minimizes the number of unknowns when you reach the target Focuses troubleshooting on the right thing, the code or the hardware Requires a good understanding of the boundary between “pure code” and hardware specific code. Leading to a modularized design Bonus benefit, migration to new hardware is simplified by the modular design, requiring changes mostly to the hardware specific code ==== Risks introduced; Compiler Features Library/Compiler bug differences Header differences (location, content) Primitive Datatype sizes Byte ordering & Data structure alignments may be different
  12. • Virtualization H T T P : / / B

    P L R E A D I N G S U G G E S T I O N S . F I L E S . W O R D P R E S S . C O M / 2 0 1 3 / 0 9 / V I RT U A L - R E A L I T Y. J P G F I X : V I RT U A L I Z AT I O N Virtual Target Using a virtual target to emulate the real hardware enables a tighter test cycle. - examples: iPhone/iPad simulator on the Mac, Android phone simulator, Traditional VMs, Vendor supplied VM appliances like WindRiver Simics Puts the simulation bench in the hands of the developer Very cheap, often free Usually significantly faster deployment cycles ===== Risks - May not have the same constraints as the real hardware - more memory - CPU not the same speed http://bplreadingsuggestions.files.wordpress.com/2013/09/virtual-reality.jpg
  13. • Compiler Differences • Software Design Modularity • Continuous Integration

    • 5 Stage Dev Cycle H T T P : / / M I C H A E L H YAT T. C O M / W P - C O N T E N T / U P L O A D S / 2 0 1 2 / 0 1 / I S T O C K _ 0 0 0 0 0 3 5 0 1 5 0 4 S M A L L . J P G G O T C H A : C O M P I L E R D I F F E R E N C E S Compiler Differences The compiler on one target does not match the compiler on the other target. http://michaelhyatt.com/wp-content/uploads/2012/01/iStock_000003501504Small.jpg
  14. • (Dual Targeting) Library Differences/Bugs • Software Design Modularity •

    TDD / Contract Testing • Continuous Integration • 5 Stage Dev Cycle H T T P : / / I M G 1 . W I K I A . N O C O O K I E . N E T / _ _ C B 2 0 1 3 0 3 1 1 0 5 1 3 3 8 / S TA R WA R S / I M A G E S / B / B C / T I E _ TA R G E T _ F F 7 . P N G F I X : D U A L TA R G E T I N G Dual Targeting http://img1.wikia.nocookie.net/__cb20130311051338/starwars/images/b/bc/TIE_target_FF7.png
  15. • Software Design Modularity H T T P : /

    / W W W. I B M S Y S T E M S M A G . C O M / G E TAT TA C H M E N T / 5 B 3 E F 7 1 4 - F F E 9 - 4 6 9 C - 8 7 C F - 5 C 9 F 9 B 1 F E A 5 7 / F I X : D E S I G N M O D U L A R I T Y Software Design Modularity Isolate “pure code” from the hardware related code - be very clear about where this boundary is - accept no excuses for violating this boundary http://www.ibmsystemsmag.com/getattachment/5b3ef714-ffe9-469c-87cf-5c9f9b1fea57/
  16. • Continuous Integration H T T P : / /

    N E B U L A . W S I M G . C O M / E 6 D 2 E 4 7 7 1 5 8 6 1 3 8 D F D 9 1 E 1 1 7 2 0 3 4 F D C 6 ? A C C E S S K E Y I D = 5 3 9 0 C B 7 3 FA A C 4 5 B 7 A 2 3 C & D I S P O S I T I O N = 0 & A L L O W O R I G I N = 1 F I X : C O N T I N U O U S I N T E G R AT I O N Continuous Integration Automate the test harness, packaging and deployment Use multiple jobs to build and test each module independently, then integrate them together Use Semantic Versioning to identify module changes and understand when re-testing is necessary http://nebula.wsimg.com/e6d2e4771586138dfd91e1172034fdc6?AccessKeyId=5390CB73FAAC45B7A23C&disposition=0&alloworigin=1
  17. • 5 stage test cycle H T T P :

    / / 2 . B P. B L O G S P O T. C O M / - E K 4 M H Z A L R Z E / U H K M 7 I 3 Y P T I / A A A A A A A A N K O / K A N T T Q O M 4 U M / S 1 6 0 0 / M I C R O _ C Y C L E . P N G E M B E D D E D T D D C Y C L E Embedded TDD Cycle Apply these five steps when developing code. First, follow the TDD practice to develop a story card or feature. Once it works in your development environment, compile it again using the target compiler. Make sure it compiles and that any software level tests are still working. Follow that by running the tests on the bench hardware and/or simulator. Then take the code all the way out to the target hardware. Once on target, run the tests again to ensure that everything still works. And finally, run the acceptance tests on the target. Assuming you pass those hurdles tee up sometime with the Field Verification Team and have them run some exploratory tests on the real hardware if possible. http://2.bp.blogspot.com/-eK4mHZAlrZE/UHKM7I3yptI/AAAAAAAANKo/kAntTQOm4UM/s1600/micro_cycle.png
  18. • Library’s have bugs H T T P : /

    / W W W. M Y C L E A N I N G P R O D U C T S . C O M / B L O G / W P - C O N T E N T / U P L O A D S / 2 0 1 3 / 0 8 / B E D - B U G S - I N - A - B O O K 2 . J P G G O T C H A : L I B R A R I E S H AV E B U G S Gotcha: Libraries have bugs. If you are dual targeting, the bugs you find on one target might not match the bugs on the other target. In some cases the bug only exists on one platform. Here are some techniques you can use to protect your code from Library bugs. http://www.mycleaningproducts.com/blog/wp-content/uploads/2013/08/bed-bugs-in-a-book2.jpg
  19. • Software Design Modularity H T T P : /

    / W W W. I B M S Y S T E M S M A G . C O M / G E TAT TA C H M E N T / 5 B 3 E F 7 1 4 - F F E 9 - 4 6 9 C - 8 7 C F - 5 C 9 F 9 B 1 F E A 5 7 / F I X : D E S I G N M O D U L A R I T Y Software Design Modularity Isolate “pure code” from library interactions http://www.ibmsystemsmag.com/getattachment/5b3ef714-ffe9-469c-87cf-5c9f9b1fea57/
  20. • Continuous Integration H T T P : / /

    N E B U L A . W S I M G . C O M / E 6 D 2 E 4 7 7 1 5 8 6 1 3 8 D F D 9 1 E 1 1 7 2 0 3 4 F D C 6 ? A C C E S S K E Y I D = 5 3 9 0 C B 7 3 FA A C 4 5 B 7 A 2 3 C & D I S P O S I T I O N = 0 & A L L O W O R I G I N = 1 F I X : C O N T I N U O U S I N T E G R AT I O N Continuous Integration Use the CI server to automate the execution of all tests.
  21. H T T P S : / / I G

    O R . I O / I M G / G I T- B R A N C H I N G / V E R S I O N . P N G F I X : S E M A N T I C V E R S I O N I N G Semantic Versioning Use Semantic Versioning to detect library changes and the need to re-execute tests https://igor.io/img/git-branching/version.png
  22. • 5 stage test cycle H T T P :

    / / 2 . B P. B L O G S P O T. C O M / - E K 4 M H Z A L R Z E / U H K M 7 I 3 Y P T I / A A A A A A A A N K O / K A N T T Q O M 4 U M / S 1 6 0 0 / M I C R O _ C Y C L E . P N G F I X : E M B E D D E D T D D C Y C L E Embedded TDD Cycle 5 steps http://2.bp.blogspot.com/-eK4mHZAlrZE/UHKM7I3yptI/AAAAAAAANKo/kAntTQOm4UM/s1600/micro_cycle.png
  23. • Memory/Performance Constraints on Target • Software Design Modularity •

    Better Emulation • Use budgets • 5 Stage Dev Cycle H T T P : / / W E B . S TA N F O R D . E D U / G R O U P / S T V P / C G I - B I N / B L O G / W P - C O N T E N T / U P L O A D S / 2 0 1 1 / 0 2 / C O N S T R A I N T S - 4 6 0 X 2 5 0 . J P G G O T C H A : P L AT F O R M C O N S T R A I N T S Memory and Performance Constraints http://web.stanford.edu/group/stvp/cgi-bin/blog/wp-content/uploads/2011/02/Constraints-460x250.jpg
  24. • Software Design Modularity H T T P : /

    / W W W. I B M S Y S T E M S M A G . C O M / G E TAT TA C H M E N T / 5 B 3 E F 7 1 4 - F F E 9 - 4 6 9 C - 8 7 C F - 5 C 9 F 9 B 1 F E A 5 7 / F I X : D E S I G N M O D U L A R I T Y Software Design Modularity Have a ‘Lab’ version of the environment where you can deploy all of the code and tests. Use Software Design Modularity to enable deployment of independent modules to the target where they can be tested with the real constraints. Use a small framework like Unity to ensure that the code can be deployed. Use multiple test runners http://www.ibmsystemsmag.com/getattachment/5b3ef714-ffe9-469c-87cf-5c9f9b1fea57/
  25. • Better Emulation H T T P : / /

    A S S E T S . V I C E . C O M / C O N T E N T- I M A G E S / C O N T E N T I M A G E / 1 5 1 8 2 4 / M AT R I X C O D E . J P G F I X : C O N S T R A I N T E M U L AT I O N Emulation Find ways to emulate the target by setting constraints on the emulator. - Make the memory match - Make the byte order match - Make the clock speed match - Make everything match if you can http://assets.vice.com/content-images/contentimage/151824/MatrixCode.jpg
  26. • Budgeting H T T P : / / W

    W W. R E N A I S S A N C E S O F T WA R E . N E T / F I L E S / I M G / F L A S H U S A G E G R A P H . J P G F I X : B U D G E T S Memory Budgets Establish budgets for memory. Track and make visible. Automate the reporting if possible. e.g. Your flash has 1M of memory, graph the usage with every iteration. Make big and visible. http://www.renaissancesoftware.net/files/img/FlashUsageGraph.jpg
  27. • Header Incompatibilities • Software Design Modularity • Continuous Integration

    • TDD H T T P S : / / W W W. M O D M Y P I . C O M / I M A G E / D ATA / T U T O R I A L S / M O D E L - B - P L U S - C O M PAT I B I L I T Y / R A S P B E R RY- P I - C O M PAT I B L E - 5 . J P G G O T C H A : H E A D E R I N C O M PAT I B I L I T I E S Header Incompatabilities https://www.modmypi.com/image/data/tutorials/model-b-plus-compatibility/raspberry-pi-compatible-5.JPG
  28. • Software Design Modularity H T T P : /

    / W W W. I B M S Y S T E M S M A G . C O M / G E TAT TA C H M E N T / 5 B 3 E F 7 1 4 - F F E 9 - 4 6 9 C - 8 7 C F - 5 C 9 F 9 B 1 F E A 5 7 / F I X : D E S I G N M O D U L A R I T Y Software Design Modularity Isolate the interaction with headers in one place Create platform independent test cases that describe how these functions must behave. http://www.ibmsystemsmag.com/getattachment/5b3ef714-ffe9-469c-87cf-5c9f9b1fea57/
  29. • Continuous Integration H T T P : / /

    N E B U L A . W S I M G . C O M / E 6 D 2 E 4 7 7 1 5 8 6 1 3 8 D F D 9 1 E 1 1 7 2 0 3 4 F D C 6 ? A C C E S S K E Y I D = 5 3 9 0 C B 7 3 FA A C 4 5 B 7 A 2 3 C & D I S P O S I T I O N = 0 & A L L O W O R I G I N = 1 F I X : C O N T I N U O U S I N T E G R AT I O N Continuous Integration Order test execution to test platform isolation early rather than later. This can get you an error or warning sooner rather than later.
  30. • Long Build Times • Extreme Automation • Software Design

    Modularity • Continuous Integration • Semantic Versioning • 5 Stage Dev Cycle H T T P : / / S K I N N Y G L I T T E R . C O M / W P - C O N T E N T / U P L O A D S / 2 0 1 3 / 0 6 / B O R E D - W O M A N - 2 . J P G G O T C H A : L O N G B U I L D S Long Builds http://skinnyglitter.com/wp-content/uploads/2013/06/bored-woman-2.jpg
  31. • Extreme Automation H T T P : / /

    U P L O A D . W I K I M E D I A . O R G / W I K I P E D I A / C O M M O N S / 8 / 8 A / A U T O M AT I O N _ O F _ F O U N D RY _ W I T H _ R O B O T. J P G F I X : H E AV Y A U T O M AT I O N Extreme Automation Automate everything you can. Reduces the barrier to testing Reduces the sense of fatigue Reduces the chance of human error http://upload.wikimedia.org/wikipedia/commons/8/8a/Automation_of_foundry_with_robot.jpg
  32. • Software Design Modularity H T T P : /

    / W W W. I B M S Y S T E M S M A G . C O M / G E TAT TA C H M E N T / 5 B 3 E F 7 1 4 - F F E 9 - 4 6 9 C - 8 7 C F - 5 C 9 F 9 B 1 F E A 5 7 / F I X : D E S I G N M O D U L A R I T Y Software Design Modularity Breaking the build up into it’s constituent parts allows you to test in isolation and then build up the number of integrations. This gets you feedback sooner rather than later on your changes. In order to do this you need to isolate the code by design, then by packaging, then by build job. Do this even if the final bundle is a completely merged component. This will enable testing individual modules on the target. http://www.ibmsystemsmag.com/getattachment/5b3ef714-ffe9-469c-87cf-5c9f9b1fea57/
  33. • Continuous Integration H T T P : / /

    N E B U L A . W S I M G . C O M / E 6 D 2 E 4 7 7 1 5 8 6 1 3 8 D F D 9 1 E 1 1 7 2 0 3 4 F D C 6 ? A C C E S S K E Y I D = 5 3 9 0 C B 7 3 FA A C 4 5 B 7 A 2 3 C & D I S P O S I T I O N = 0 & A L L O W O R I G I N = 1 F I X : C O N T I N U O U S I N T E G R AT I O N Continuous Integration Make the server do the work. With all the automation added to the system, detecting the need for execution and then doing the execution can be left to the server. That said, pay attention to the CI server. React immediately to build failures. Don’t mark things ‘Ignored’ just to get a green build. Uncle Bob “The only way to go fast, is to go well” Accumulated ‘ignored tests’ will destroy you in the end
  34. H T T P S : / / I G

    O R . I O / I M G / G I T- B R A N C H I N G / V E R S I O N . P N G F I X : S E M A N T I C V E R S I O N I N G Semantic Versioning Use semantic versioning on each module to detect/identify API changes and potential compatibility issues. This can be used to identify what tests need to be run. https://igor.io/img/git-branching/version.png
  35. • How do I apply it? • 5 Cycle Approach

    (Embedded TDD Cycle) H T T P : / / 2 . B P. B L O G S P O T. C O M / - E K 4 M H Z A L R Z E / U H K M 7 I 3 Y P T I / A A A A A A A A N K O / K A N T T Q O M 4 U M / S 1 6 0 0 / M I C R O _ C Y C L E . P N G F I X : E M B E D D E D T D D C Y C L E Embedded TDD Cycle 5 steps Regular TDD cycle + - compile for target, fix compiler incompatibilities - Run unit tests in target, fix execution problems - run first on the bench - run second on the target - Run acceptance tests on target, find and fix problems. http://2.bp.blogspot.com/-eK4mHZAlrZE/UHKM7I3yptI/AAAAAAAANKo/kAntTQOm4UM/s1600/micro_cycle.png
  36. • In support of Testing • Configuration Management • Source

    Control Management • Collective Ownership H T T P : / / S 1 . C D N . A U T O E V O L U T I O N . C O M / I M A G E S / N E W S / H O W - P I T- C R E W S - W O R K - I N - M O T O R S P O RT- 1 4 2 2 5 _ 2 . J P G S U P P O RT I N G A C T I V I T I E S Additional Activities to Support Testing http://s1.cdn.autoevolution.com/images/news/how-pit-crews-work-in-motorsport-14225_2.jpg
  37. • Configuration Management H T T P : / /

    W W W. T R O L L . M E / I M A G E S / T H E - B O B S / B U T- D O - W E - H AV E - E N O U G H - C O N F I G U R AT I O N - M A N A G E M E N T. J P G Configuration Management Identify your configuration up front and keep it maintained. Do not build from a different version of libraries and headers unless necessary, even when dual targeting. Bugs on one platform might not be present on the other. Header signatures may differ between platforms. Identify these things and protect yourself from them. The number of interconnections between libraries in a large system is very large. Keeping them all managed is very difficult. It is better to pay the cost upfront and maintain it then chase issues later down the line. When you wait you put all the prior work at risk. This can have catastrophic impacts on the project deliverable and your timeline. Knowing each dependency and relationship in the entire stack is necessary to be sure of the quality and reliability of the system. http://www.troll.me/images/the-bobs/but-do-we-have-enough-configuration-management.jpg
  38. • Source Control Management H T T P : /

    / AT O M A N T I C . G I T H U B . I O / S L I D E S - V E R S I O N C O N T R O L / I M G / M E M E - V E R S I O N C O N T R O L A L LT H I N G S . J P G Source Control Management Use a really good Source Control Management system. Make sure everyone knows how to use it and all its features. Pay for the training if you must. Source Control Management allows us to control what is built and when, it also gives us the ability to roll back. Commits should be small. Failed Builds should result in a roll back to prevent blocking the whole program. If it’s hard to roll back, it’s hard to move forward. http://atomantic.github.io/slides-versioncontrol/img/meme-versioncontrolallthings.jpg
  39. • Collective Ownership H T T P : / /

    D I Y C O N F E S S I O N S . C O M / W P - C O N T E N T / U P L O A D S / 2 0 1 3 / 0 5 / N O T- S H A R I N G . J P G C O L L E C T I V E C O D E O W N E R S H I P Collective Ownership Make sure the team understands that we are all working toward the same goal and that the product belongs to all of us. There should be no prohibition on any developer changing any code in order to make the system better as a whole. That is not a license to violate agreed upon standards for architecture and design or to break existing features and functionality. It is however an opening to jump in and do what needs to be done without waiting for someones permission. http://diyconfessions.com/wp-content/uploads/2013/05/not-sharing.jpg
  40. T H E F I N A L P R

    O D U C T http://fc05.deviantart.net/fs71/i/2014/055/5/f/cyborg_with_human_camo_by_jessedr1-d77td5g.jpg So we’ve covered some of techniques for testing embedded systems. These techniques are applicable to both large and small systems. As the project scales to ever larger size more emphasis must be applied to Configuration Management and Continuous Integration as they necessarily become more complicated. Of all of the fixes discussed, I’m sure you noticed the repeated mention of Design Modularity. Isolating the application code from the hardware is one of most important design decision that can be made. This holds true for embedded systems as well as all software design. The benefits far outweigh the costs for effectiveness of testing and quality of product. It also carries the added benefit of enabling hardware upgrades without disrupting the entire application. Finally, the Embedded TDD Cycle. Five simple steps to build software effectively and mitigate the majority of issues in embedded software development. It is not a cure all, but it goes a long way to making systems adaptable and robust. http://fc05.deviantart.net/fs71/i/2014/055/5/f/cyborg_with_human_camo_by_jessedr1-d77td5g.jpg
  41. Rich Dammkoehler, Pillar Technology [email protected] H T T P :

    / / S D . K E E P C A L M - O - M AT I C . C O . U K / I / K E E P - C A L M - A N D - A S K - M E - Q U E S T I O N S . P N G Questions Test Driven Development for Embedded C https://pragprog.com/book/jgade/test-driven-development-for-embedded-c Rich Dammkoehler Pillar Technology [email protected] Twitter: @net_zer0 http://sd.keepcalm-o-matic.co.uk/i/keep-calm-and-ask-me-questions.png