Recording Meaningful UI Tests

Espresso Test Recording has been introduced with Android Studio 2.2. Recording tests is as easy as using an app. But how do you assert that your tests are meaningful?

In order to do that you need to consider your use case, the app architecture and the testing strategy.

Transcript

1. Recording meaningful UI Tests Jan Kettner @realitymatters #devfestHH
2. • https://github.com/kaskasi/piScroller
3. Universal Testing Machine
4. Admet Expert 2600 http://www.admet.com/
5. https://www.youtube.com/watch?v=LhUclxBUV_E
6. max speed : 50 mm/min http://www.admet.com/
7. min speed: 0,0005 mm/min http://www.admet.com/
8. weight: 45 kg http://www.admet.com/
9. Fixture
10. what I found interesting in tensile tests, is that the tests are supposed to fail at some point. And what’s being tested under which parameters the objects don’t behave as expected.
11. https://en.wikipedia.org/wiki/List_of_industrial_disasters List of industrial disasters
12. Espresso Test Recording • presented at Google IO 2016 • still experimental feature • Available since Android Studio 2.2 Preview 3 • Users can record journeys through the app and perform assertions by interacting with the app on an actual device
13. What do you mean by meaningful Tests ?
14. What is the use case ?
15. Acceptance Criteria given I open the app and I open the quiz when I enter answer correctly then digit color green
16. Acceptance Criteria given I open the app and I open the quiz when I enter answer correctly then digit color green given I open the app and I open the quiz when I enter pi incorrectly then button text is Restart
17. Acceptance Criteria given I open the app and I open the quiz when I enter answer correctly then digit color green given I open the app and I open the quiz when I enter pi incorrectly then button text is Restart given I open the app when I scroll thousand digits of pi
18. Acceptance Criteria • given I open the app and I open the quiz • when I enter answer correctly • then digit color green • given I open the app and I open the quiz • when I enter pi incorrectly • then button text is Restart • given I open the app • when I scroll thousand digits of pi
19. What do you mean by meaningful Tests ?
20. ViewModel View Model
21. ViewModel View Model business logic and data
22. ViewModel View presentation Model business logic and data
23. ViewModel abstraction of view property state View presentation Model business logic and data
24. ViewModel abstraction of view property state View presentation Model business logic and data
25. ViewModel abstraction of view property state View presentation Model business logic and data Databinding
26. ViewModel abstraction of view property state View presentation Model business logic and data Databinding
27. “It is what I sometimes have called the separation of concerns, which, even if not perfectly possible, is yet the only available tequnique for effective ordering of one’s thoughts.” Edsger W. Dijkstra, 1974, On the role of schientific thought
28. How to test the Acceptance Criteria ?
29. who does the testing ?
30. who does the testing ? what is tested ?
31. who does the testing ? what is tested ? what is the goal of the test ?
32. https://twitter.com/ThePracticalDev/status/733908215021199360
33. What do you mean by meaningful Tests ?
34. Testing Pyramid http://martinfowler.com/bliki/TestPyramid.html
35. Let’s write some code
36. Model PiQuiz.java PiQuiz(int position) int position String answer getPosition() setPosition(…) getAnswer() setAnswer(…) PiProvider.java String π getDigitOfPi(…) getNumberOfDigitsOfPi()
37. View pi_quiz.xml <layout>
 <data>
 <variable
 name="piQuiz"
 type="PiQuizViewModel"/>
 </data>
 
 <RelativeLayout>
 <TextView/>
 <TextView/>
 <EditText/>
 </RelativeLayout>
 </layout>
38. Button <TextView
 android:clickable="@{piQuiz.buttonEnabled}"
 android:onClick="@{() -> piQuiz.onNext()}"
 android:text= "@{piQuiz.isButtonTextNext ? @string/next : @string/restart}” android:background= "@{piQuiz.buttonEnabled ? @color/blue : @color/grey}”/>
39. Button <TextView
 android:clickable="@{piQuiz.buttonEnabled}"
 android:onClick="@{() -> piQuiz.onNext()}"
 android:text= "@{piQuiz.isButtonTextNext ? @string/next : @string/restart}” android:background= "@{piQuiz.buttonEnabled ? @color/blue : @color/grey}”/>
40. InputField <EditText
 android:enabled="@{piQuiz.possibleToEnterAnswer}"
 android:text="@={piQuiz.answer}"
 android:textColor= "@{piQuiz.answerColorGreen ? @color/green : @color/red}"
 />
41. InputField <EditText
 android:enabled="@{piQuiz.possibleToEnterAnswer}"
 android:text="@={piQuiz.answer}"
 android:textColor= "@{piQuiz.answerColorGreen ? @color/green : @color/red}"
 />
42. ViewModel PiQuizViewModel.java CharSequence getAnswer() {}
 void setAnswer(CharSequence answer) {}
 
 boolean isQuestionHowPiStarts() {}
 boolean isButtonTextNext() {}
 boolean isButtonEnabled() {}
 boolean isAnswerColorGreen() {}
 boolean isPossibleToEnterAnswer() {}
 
 String getDigitPosition() {} 
 void onNext() {}
43. ViewModel PiQuizViewModel.java CharSequence getAnswer() {}
 void setAnswer(CharSequence answer) {}
 
 boolean isQuestionHowPiStarts() {}
 boolean isButtonTextNext() {}
 boolean isButtonEnabled() {}
 boolean isAnswerColorGreen() {}
 boolean isPossibleToEnterAnswer() {}
 
 String getDigitPosition() {} 
 void onNext() {}
44. UnitTest PiQuizViewModelTest.java @Test
 public void when_correct_answer_ then_answer_color_is_green() {
 PiQuiz quiz = new PiQuiz(0);
 PiQuizViewModel viewModel = new PiQuizViewModel(quiz);
 
 viewModel.setAnswer("3");
 assertTrue(viewModel.isAnswerColorGreen()); }
 

45. UnitTest PiQuizViewModelTest.java @Test
 public void when_wrong_answer_ then_button_text_is_restart() {
 PiQuiz quiz = new PiQuiz(0);
 PiQuizViewModel viewModel = new PiQuizViewModel(quiz);
 
 viewModel.setAnswer("4");
 
 assertFalse(viewModel.isButtonTextNext());
 } 
 

46. PiQuizViewModelTest.java when_answer_is_entered_then_viewmodel_notifies_view when_answer_entered_then_viewmodel_returns_answer then_button_is_not_enabled when_answer_entered_then_button_is_enabled when_correct_answer_entered_then_answer_color_is_green when_wrong_answer_entered_then_answer_color_is_not_green when_correct_answer_entered_then_button_text_is_next when_wrong_answer_entered_then_button_text_is_restart when_position_is_thousand_then_button_text_is_restart then_it_is_possible_to_enter_answer when_answer_is_entered_then_it_is_not_possible_to_enter_answer when_answer_is_entered_when_next_is_pressed_then_it_is_possible _to_enter_answer when_next_is_pressed_then_viewmodel_notifies_view_ when_correct_answer_is_entered_when_next_is_pressed _then_quiz_position_is_incremented when_wrong_answer_is_entered_when_restart_is_pressed _then_quiz_position_is_zero when_position_is_thousand_when_correct_answer_is_entered_ when_restart_is_pressed_then_quiz_position_is_zero when_position_is_zero_then_question_is_how_pi_starts then_viewmodel_returns_digit_position_as_text then_viewmodel_returns_pi_quiz
47. PiQuizViewModelTest.java when_answer_is_entered_then_viewmodel_notifies_view when_answer_entered_then_viewmodel_returns_answer then_button_is_not_enabled when_answer_entered_then_button_is_enabled when_correct_answer_then_answer_color_is_green when_wrong_answer_entered_then_answer_color_is_not_green when_correct_answer_entered_then_button_text_is_next when_wrong_answer_entered_then_button_text_is_restart when_position_is_thousand_then_button_text_is_restart then_it_is_possible_to_enter_answer when_answer_is_entered_then_it_is_not_possible_to_enter_answer when_answer_is_entered_when_next_is_pressed_then_it_is_possible _to_enter_answer when_next_is_pressed_then_viewmodel_notifies_view_ when_correct_answer_is_entered_when_next_is_pressed _then_quiz_position_is_incremented when_wrong_answer_is_entered_when_restart_is_pressed _then_quiz_position_is_zero when_position_is_thousand_when_correct_answer_is_entered_ when_restart_is_pressed_then_quiz_position_is_zero when_position_is_zero_then_question_is_how_pi_starts then_viewmodel_returns_digit_position_as_text then_viewmodel_returns_pi_quiz
48. Let’s record a Test!
49. onView(…) .perform(…) .check(…);
50. onView(…) .perform(…) .check(…); ViewMatcher
51. onView(…) .perform(…) .check(…); ViewAction
52. onView(…) .perform(…) .check(…); ViewAssertion
53. onView(…) .perform(…) .check(…); ViewMatcher ViewAction ViewAssertion
54. onView(withId(R.id.my_view)) .perform(…) .check(…); ViewMatcher ViewAction ViewAssertion
55. onView(withId(R.id.my_view)) .perform(click()) .check(…); ViewMatcher ViewAction ViewAssertion
56. onView(withId(R.id.my_view)) .perform(click()) .check(matches(withText(“Hello”))); ViewMatcher ViewAction ViewAssertion
57. Fixtures & Assertions
58. I don’t like to read Espresso Code ViewInteraction actionMenuItemView = onView(
 allOf(withId(R.id.menu_quiz), withContentDescription("Quiz"), isDisplayed()));
 actionMenuItemView.perform(click());
 
 ViewInteraction appCompatEditText = onView(
 allOf(withClassName(is("android.support.v7.widget.A ppCompatEditText")), isDisplayed()));
 appCompatEditText.perform(click());
 
 ViewInteraction appCompatEditText2 = onView(
 allOf(withClassName(is("android.support.v7.widget.A ppCompatEditText")), isDisplayed()));
 appCompatEditText2.perform(replaceText("3"), closeSoftKeyboard());
 
 ViewInteraction appCompatTextView = onView(
 allOf(withId(R.id.quiz_next), withText("Next"), isDisplayed()));

59. public class OpenQuiz implements Fixture {
 
 @Override
 public void perform() {
 ViewInteraction actionMenuItemView = onView(
 allOf( withId(R.id.menu_quiz), withContentDescription(“Quiz"), isDisplayed()));
 actionMenuItemView.perform(click());
 }
 } Open Quiz
60. public class OpenQuiz implements Fixture {
 
 @Override
 public void perform() {
 ViewInteraction actionMenuItemView = onView(
 allOf( withId(R.id.menu_quiz), withContentDescription(“Quiz"), isDisplayed()));
 actionMenuItemView.perform(click());
 }
 } Open Quiz
61. Enter Answer public class EnterAnswer implements Fixture {
 
 private String answer;
 
 public EnterAnswer(String answer) {
 this.answer = answer;
 }
 
 @Override
 public void perform() {
 ViewInteraction appCompatEditText = onView(
 allOf( withClassName( is("android.support.v7.widget.AppCompatEditText")), isDisplayed()));
 appCompatEditText.perform(replaceText(“3”), closeSoftKeyboard());
 }
 }
62. Enter Answer public class EnterAnswer implements Fixture {
 
 private String answer;
 
 public EnterAnswer(String answer) {
 this.answer = answer;
 }
 
 @Override
 public void perform() {
 ViewInteraction appCompatEditText = onView(
 allOf( withClassName( is("android.support.v7.widget.AppCompatEditText")), isDisplayed()));
 appCompatEditText.perform(replaceText(answer), closeSoftKeyboard());
 }
 }
63. Check Digit Color public class CheckDigitColor implements Assertion {
 
 @ColorRes
 private int color;
 
 public CheckDigitColor(@ColorRes int color) {
 this.color = color;
 }
 
 @Override
 public void performAssertion() {
 ViewInteraction appCompatEditText = onView(
 allOf(withClassName( is(“android.support.v7.widget.AppCompatEditText")), isDisplayed()));
 appCompatEditText.check( matches(withTextColor(R.color.green)));
 }
 }
64. Check Digit Color public class CheckDigitColor implements Assertion {
 
 @ColorRes
 private int color;
 
 public CheckDigitColor(@ColorRes int color) {
 this.color = color;
 }
 
 @Override
 public void performAssertion() {
 ViewInteraction appCompatEditText = onView(
 allOf(withClassName( is(“android.support.v7.widget.AppCompatEditText")), isDisplayed()));
 appCompatEditText.check( matches(withTextColor(color)));
 }
 }
65. @Test
 public void Given_in_quiz _when_correct_answer_entered _then_answer_has_green_textcolor() { 
 }
66. @Test
 public void Given_in_quiz _when_correct_answer_entered _then_answer_has_green_textcolor() { 
 new OpenQuiz().perform();
 }
67. @Test
 public void Given_in_quiz _when_correct_answer_entered _then_answer_has_green_textcolor() { 
 new OpenQuiz().perform();
 
 new EnterAnswer("3").perform();
 }
68. @Test
 public void Given_in_quiz _when_correct_answer_entered _then_answer_has_green_textcolor() { 
 new OpenQuiz().perform();
 
 new EnterAnswer("3").perform();
 
 new CheckDigitColor(R.color.green) .performAssertion();
 }
69. @Test
 public void Given_in_quiz _when_wrong_answer_is_entered _then_button_text_is_restart() { 
 }
70. @Test
 public void Given_in_quiz _when_wrong_answer_is_entered _then_button_text_is_restart() { 
 new OpenQuiz().perform();
 }
71. @Test
 public void Given_in_quiz _when_wrong_answer_is_entered _then_button_text_is_restart() { 
 new OpenQuiz().perform();
 
 new EnterAnswer("4").perform();
 }
72. @Test
 public void Given_in_quiz _when_wrong_answer_is_entered _then_button_text_is_restart() { 
 new OpenQuiz().perform();
 
 new EnterAnswer("4").perform();
 
 new CheckButtonText(“Restart”) .performAssertion();
 }
73. PiScrollerTest.java Given_in_scroller_when_scroll_to_1000_digits Given_in_scroller_when_scrolled_to_beginning_then_digits_with_even_position _have_beige_background Given_in_scroller_when_scrolled_to_beginning_then_digits_with_uneven_position _have_white_background PiQuizTest.java Given_open_quiz_when_correct_answer_entered_then_answer_has_green_textcolor Given_open_quiz_when_wrong_answer_entered_then_answer_has_red_textcolor Given_open_quiz_then_button_text_is_next Given_open_quiz_when_wrong_answer_is_entered_then_button_text_is_restart Given_open_quiz_when_answer_was_entered_thousand_times_then_button_text _is_restart Given_open_quiz_when_answer_entered_then_next_button_color_is_grey Given_open_quiz_when_answer_entered_then_next_button_is_blue Given_open_quiz_then_question_is_how_does_pi_start Given_open_quiz_when_answer_entered_and_when_next_pressed_then_question _is_the_nth_digit_of_pi Given_open_quiz_then_next_button_is_disabled Given_open_quiz_when_answer_entered_then_next_button_is_enabled
74. PiScrollerTest.java Given_in_scroller_when_scroll_to_1000_digits PiQuizTest.java Given_open_quiz_when_correct_answer_entered_then_answer_has_green_textcolor Given_open_quiz_when_wrong_answer_is_entered_then_button_text_is_restart
75. Gherkin, anyone ?
76. Feature: Quiz
 The app gives direct visual confirmation if quiz was answered correctly
 
 Scenario Outline: Input answer
 Given I have a PiScroller
 When I open the Quiz
 When I input answer <answer>
 Then I should <see> text color is green
 
 Examples:
 | answer | see |
 | 3 | true |
 | 4 | false |
77. @Given("^I have a PiScrollerActivity$")
 public void I_have_a_PiScrollerActivity() {
 assertNotNull(getActivity());
 }
 
 @When("^I open the Quiz$")
 public void I_open_the_quiz() { new OpenQuiz().perform(); }
 
 @When("^I input answer (\\S+)$")
 public void I_input_answer(final String answer) {
 new EnterAnswer(answer).perform();
 }
 
 @Then("^I should (true|false) text color is green$")
 public void I_should_see_textcolor_is_green(final boolean isTextColorGreen) {
 new CheckDigitColor(R.color.green, isTextColorGreen).performAssertion();
 }
78. Are these Tests meaningful?
79. given I talk about testing when I am done then I say “Thank you” Jan Kettner @realitymatters #devfestHH