Break Your API: Adapting Unit Testing Frameworks to API Integration Testing by Bonnie Mattson

Transcript

1. Break Your API Adapting Unit Testing Frameworks to API Integration Testing Bonnie Mattson | @bonmatts | Software Engineer Context.io | @contextio
2. Integration Test?
3. Integration Test? • Blackbox
4. Integration Test? • Blackbox • Client
5. Integration Test? • Blackbox •A really, really mean client. • Client
6. Integration Test? • Blackbox •A really, really mean client. • Client
8. PROCESS
9. PROCESS
10. PROCESS Agile
11. PROCESS Agile
12. PROCESS Agile D i s t i n c t R o l e s
13. No one likes waterfalls.
14. Unit Testing frameworks for Integration Testing?!?
15. Unit Testing frameworks for Integration Testing?!? • Familiar paradigm
16. Unit Testing frameworks for Integration Testing?!? • Familiar paradigm •Ordered Randomly
17. Unit Testing frameworks for Integration Testing?!? • Familiar paradigm •Ordered Randomly • Idempotent
18. Unit Testing frameworks for Integration Testing?!? • Familiar paradigm •Ordered Randomly • Idempotent • Can’t chain tests
19. Unit Testing frameworks for Integration Testing?!? • Familiar paradigm •Ordered Randomly • Idempotent • Can’t chain tests
20. Case 1: All the JSON Documents, ever.
21. Case 1: All the JSON Documents, ever. • A Resftul API backed by mongoDB
22. Case 1: All the JSON Documents, ever. • A Resftul API backed by mongoDB • 500 example json documents
23. Case 1: All the JSON Documents, ever. • A Resftul API backed by mongoDB • 500 example json documents • Verify that basic CRUD operations can be performed on each document
24. Case 1: All the JSON Documents, ever. • A Resftul API backed by mongoDB • 500 example json documents • Verify that basic CRUD operations can be performed on each document • BTW, these documents may change as the specifications for them that are determined by some committee somewhere change
25. Solution 1: All The Callbacks
26. Solution 1: All The Callbacks POST GET POST GET PUT GET GET DELETE verify creation create modify part verify mod modify whole verify mod . . . verify delete
27. Solution 1: All The Callbacks POST GET POST GET PUT GET GET DELETE verify creation create modify part verify mod modify whole verify mod . . . verify delete
28. Source: https://github.com/vlucas/frisby
29. Source: https://github.com/vlucas/frisby
30. Source: https://github.com/vlucas/frisby
31. Solution 2: ONCE AGAIN, THIS TIME WITH JAVA
32. Solution 2: ONCE AGAIN, THIS TIME WITH JAVA Source: https:// github.com/junit- team/junit/wiki
33. Solution 2: ONCE AGAIN, THIS TIME WITH JAVA Source: https:// github.com/junit- team/junit/wiki
34. Solution 2: ONCE AGAIN, THIS TIME WITH JAVA Source: https:// github.com/junit- team/junit/wiki
35. Case 2: Context.io
36. Case 2: Context.io • Given a set of email account credentials, developers can interact with the email account mailboxes in a Restful way.
37. Case 2: Context.io • Given a set of email account credentials, developers can interact with the email account mailboxes in a Restful way. • 2 versions of the api, with ~152 endpoints, ~168 parameters across endpoints, and ~ 2.5 million requests a day
38. Case 2: Context.io • Given a set of email account credentials, developers can interact with the email account mailboxes in a Restful way. • 2 versions of the api, with ~152 endpoints, ~168 parameters across endpoints, and ~ 2.5 million requests a day • TECH DEBT
39. Case 2: Context.io • Given a set of email account credentials, developers can interact with the email account mailboxes in a Restful way. • 2 versions of the api, with ~152 endpoints, ~168 parameters across endpoints, and ~ 2.5 million requests a day • TECH DEBT • Not the canonical data source.
40. What do we need to do?
41. What do we need to do? • Control the data source, the mailbox
42. What do we need to do? • Control the data source, the mailbox • Set up state before the tests run
43. What do we need to do? • Control the data source, the mailbox • Set up state before the tests run • Prevent state leakage across tests
44. What do we need to do? • Control the data source, the mailbox • Set up state before the tests run • Prevent state leakage across tests • Reuse examples
45. Solution? Rspec.
46. Solution? Rspec. • Set up state before the test, tear it down after
47. Solution? Rspec. • Set up state before the test, tear it down after • Shared examples to keep it DRY
48. Solution? Rspec. • Set up state before the test, tear it down after • Shared examples to keep it DRY • Readable
49. Solution? Rspec. • Set up state before the test, tear it down after • Shared examples to keep it DRY • Readable
50. Where do we go from here?
51. Where do we go from here?
52. IN CONCLUSION
53. IN CONCLUSION • It is vitally important to break your API before your customers do.
54. IN CONCLUSION • It is vitally important to break your API before your customers do. • There is no “right” way to test your API. Programatic integration test suites using unit testing frameworks is one approach.
55. IN CONCLUSION • It is vitally important to break your API before your customers do. • There is no “right” way to test your API. Programatic integration test suites using unit testing frameworks is one approach. • Use whatever framework you want to.
56. IN CONCLUSION • It is vitally important to break your API before your customers do. • There is no “right” way to test your API. Programatic integration test suites using unit testing frameworks is one approach. • Use whatever framework you want to. • We use a combination of tools to test and monitor context.io, including Runscope
57. Thanks
58. • Shawn Nussbaum, Michael Bernstein, Dan Corbin, Aaron Mills, Bob Potter, Eric Pinzur, Susi Rathmann, Ryan Michael, David Lyons, Aaron Labrie, Clare Glinka, Cecy Correra, Sarabjit Singh,Allyson Weber, and Tony Blank for convincing me to submit this talk Thanks
59. Questions? Bonnie Mattson | @bonmatts | Software Engineer | Context.io | @context.io