Getting Started with eunit

How do you know if your Erlang program works? You test it! Erlang comes with a unit-testing framework called "eunit" that is easy to slip into your project. We'll talk briefly about what eunit gives you and work through a bunch of examples together, including how to use assertions, how to structure test suites, and how to deal common pitfalls and gotchas.

Presented at Chicago Erlang: http://www.meetup.com/ErlangChicago/events/172139782/

Transcript

1. Getting Started with eunit Sean Cribbs 2014-03-26 Wed Sean Cribbs Getting Started with eunit 2014-03-26 Wed 1 / 23
2. Introduction Why use eunit? Simple to get started Familiar if you’ve written unit-tests in other languages Long runway, good tool integration Sean Cribbs Getting Started with eunit 2014-03-26 Wed 2 / 23
3. Getting Started Boilerplate/Setup Have complete Erlang/OTP installed (some distros have erlang-eunit as a package) Create a directory to work in Create a module to hold tests -module(mytests). -compile([export_all, debug_info]). -include_lib("eunit/include/eunit.hrl"). Sean Cribbs Getting Started with eunit 2014-03-26 Wed 3 / 23
4. Getting Started Assert True All test functions are suffixed with _test or _test_. Assertions are macros included from eunit.hrl Create a simple test that asserts true: truth_test() -> ?assert(true). Sean Cribbs Getting Started with eunit 2014-03-26 Wed 4 / 23
5. Getting Started Run the Test Start an Erlang shell $ erl Eshell V5.10.3 (abort with ^G) Compile the test module 1> c(mytests). {ok,mytests} Run eunit on the module 2> eunit:test(mytests). %% or mytests:test(). Test passed. ok Sean Cribbs Getting Started with eunit 2014-03-26 Wed 5 / 23
6. Getting Started Failing Assertion Make the test fail! truth_test() -> ?assert(false). Run eunit 3> c(mytests), eunit:test(mytests). mytests: truth_test (module ’mytests’)...*failed* in function mytests:’-truth_test/0-fun-0-’/0 (mytests.erl, l **error:{assertion_failed,[{module,mytests}, {line,6}, {expression,"false"}, {expected,true}, {value,false}]} Sean Cribbs Getting Started with eunit 2014-03-26 Wed 6 / 23
7. Getting Started Assertions Simple truth, falsehood ?assert( TruthValue ) ?assertNot( FalseValue ) Equality, inequality ?assertEqual( Expected, Expression ) ?assertNotEqual( Unexpected, Expression ) Pattern matching ?assertMatch( Pattern, Expression ) ?assertNotMatch( Pattern, Expression ) Sean Cribbs Getting Started with eunit 2014-03-26 Wed 7 / 23
8. Getting Started More Assertions Exceptions ?assertException( Class, Term, Expression ) ?assertNotException( Class, Term, Expression ) %% Class = exit | error | throw ?assertError( Term, Expression ) ?assertExit( Term, Expression ) ?assertThrow( Term, Expression ) External commands ?cmd( ShellCommand ) ?assertCmdStatus( ExitStatus, ShellCommand ) ?assertCmdOutput( Text, ShellCommand ) Sean Cribbs Getting Started with eunit 2014-03-26 Wed 8 / 23
9. Testing a Simple Module Testing a Simple Module Learning by example Let’s test a naive implementation of a familiar data structure, a stack. In Erlang we will represent these as lists, wrapped in a module that implements New, Push, Pop and Size operations. Sean Cribbs Getting Started with eunit 2014-03-26 Wed 9 / 23
10. Testing a Simple Module Stack module -module(stack). -export([new/0, push/2, pop/1, size/1]). new() -> []. push(Thing, Stack) -> [Thing|Stack]. pop(Stack) -> {hd(Stack), tl(Stack)}. size(Stack) -> length(Stack). Sean Cribbs Getting Started with eunit 2014-03-26 Wed 10 / 23
11. Testing a Simple Module Review: Setup eunit Create stack_tests module Include eunit.hrl Compile both stack and stack_tests in the shell Sean Cribbs Getting Started with eunit 2014-03-26 Wed 11 / 23
12. Testing a Simple Module Things we could test New stacks are empty (size 0) Push followed by Pop returns original item and stack Push increases size of stack Pop decreases size of stack Pop on an empty stack raises an error Sean Cribbs Getting Started with eunit 2014-03-26 Wed 12 / 23
13. Testing a Simple Module Pop empty stack pop_empty_raises_error_test() -> ?assertError(badarg, stack:pop(stack:new())). Sean Cribbs Getting Started with eunit 2014-03-26 Wed 13 / 23
14. Testing a Simple Module Improving output 16> eunit:test(stack_tests, [verbose]). ======================== EUnit ======================== module ’stack_tests’ stack_tests: new_stacks_are_empty_test...ok stack_tests: lifo_test...ok stack_tests: push_increases_size_test...ok stack_tests: pop_decreases_size_test...ok stack_tests: pop_empty_raises_error_test...ok [done in 0.014 s] ======================================================= All 5 tests passed. See also github.com/seancribbs/eunit_formatters Sean Cribbs Getting Started with eunit 2014-03-26 Wed 14 / 23
15. Advanced Test Suites Advanced Test Suites Things you might want to do: Integrate the test suite with build tools Perform setup/teardown around tests Enforce timeouts on tests that might stall Group multiple related tests together more tightly Debug state and progress inside a test Report on code coverage of your tests Sean Cribbs Getting Started with eunit 2014-03-26 Wed 15 / 23
16. Advanced Test Suites Build tools rebar Run rebar eunit Defines TEST macro Includes modules in test/ directory when testing erlang.mk Uses common_test only, you have to call eunit manually in your ct suite. Sean Cribbs Getting Started with eunit 2014-03-26 Wed 16 / 23
17. Advanced Test Suites Setup and Teardown Sometimes you need to do setup or teardown around tests. You can group together tests that require the same environment: similar_tests_() -> % use trailing _! {setup, fun() -> start_server() end, % setup fun(Pid) -> stop_server(Pid) end, % cleanup [ %% {TestDescription, TestFunction} {"test 1", fun test1/0}, {"test 2", fun test2/0} ]}. You can also use foreach instead of setup to run setup/cleanup around every test. Sean Cribbs Getting Started with eunit 2014-03-26 Wed 17 / 23
18. Advanced Test Suites Timeouts eunit runs every test in a new process and kills it after 5 seconds by default. You can increase/decrease the timeout like so: slow_test_() -> % use trailing _! {timeout, 60, % 1 minute, in seconds [fun slow_thing/0]}. % can be a list of tests Sean Cribbs Getting Started with eunit 2014-03-26 Wed 18 / 23
19. Advanced Test Suites Grouping tests Sometimes you want to treat a bunch of tests as a group, in which case you can just return a list of them. Note an assertion with the underscore prefix creates a test function that wraps the assertion. fizzbuzz_test_() -> % use trailing _! [ ?_assertEqual(fizz, fizzbuzz(3)), ?_assertEqual(buzz, fizzbuzz(5)), ?_assertEqual(fizzbuzz, fizzbuzz(15)), ?_assertEqual(7, fizzbuzz(7)) ]. Sean Cribbs Getting Started with eunit 2014-03-26 Wed 19 / 23
20. Advanced Test Suites Debugging It can be hard to tell what’s going on in complex tests. Debug macros help. ?debugMsg("Trying fizzbuzz") mytests.erl:62:<0.126.0> Trying fizzbuzz ?debugHere mytests.erl:65:<0.126.0> <- ?debugVal(Count) mytests.erl:71:<0.126.0> Count = 10 ?debugFmt("Got ~p", [A]) mytests.erl:83:<0.126.0> Got fizz ?debugTime("fizz", netfizz()) mytests.erl:100:<0.126.0> fizzing: 0.321 s Sean Cribbs Getting Started with eunit 2014-03-26 Wed 20 / 23
21. Advanced Test Suites Code coverage rebar supports computing and outputting coverage automatically. Add this line to rebar.config: {cover_enabled, true}. After running rebar eunit, open .eunit/index.html Otherwise: cover:start(). cover:compile_beam_directory("."). eunit:test(mytests). [cover:analyze_to_file(Mod, [html]) || Mod <- cover:modules()]. cover:stop(). Sean Cribbs Getting Started with eunit 2014-03-26 Wed 21 / 23
22. Gotchas and Pitfalls Gotchas and Pitfalls eunit swallows console output by default. Write to log files or use ?debug macros. Ensure your tests are pure or cleanup properly after themselves. Ordering issues can plague test suites run on different machines. Complicated setup/teardown combos can be hard to reason about. Move to common_test if your suite becomes hard to setup. Know when to use the _ on test functions: If your function executes directly and uses regular assertions, omit the underscore. If your function uses fixtures or returns a list of tests, include the underscore. Sean Cribbs Getting Started with eunit 2014-03-26 Wed 22 / 23
23. Gotchas and Pitfalls Resources rebar: https://github.com/rebar/rebar eunit guide: http://www.erlang.org/doc/apps/eunit/users_guide.html Prettier suite output: https://github.com/seancribbs/eunit_formatters These slides: http://speakerdeck.com/seancribbs/getting-started-with-eunit Sean Cribbs Getting Started with eunit 2014-03-26 Wed 23 / 23