Specification of a Visual Programming Language by Example

Specification of a Visual Programming Language by Example

Master's Thesis at Graz University of Technology

F9bf9c0311137e6c4878d74cd54a8e41?s=128

Maximilian Fellner

December 19, 2013
Tweet

Transcript

  1. Institute for Software Technology – Maximilian Fellner 2013 1 Specification

    of a Visual Programming Language by Example Behavior-Driven Development with Cucumber for Catrobat Master’s Thesis by Maximilian Fellner
  2. Institute for Software Technology – Maximilian Fellner 2013 Question: What

    are the advantages of standards and specifications? 2
  3. Institute for Software Technology – Maximilian Fellner 2013 Standardization ‣

    Competitive advantages ‣ Strategic alliances ‣ Reduction of costs ‣ Safety & quality ‣ Public interest 3 Economic Benefits of Standardization, Deutsches Institut für Normung, Beuth Verlag
  4. Institute for Software Technology – Maximilian Fellner 2013 Specification ‣

    Understand the requirements ‣ Describe a product abstractly ‣ Formulate binding contracts ‣ Quality assurance 4 Prerequisite for standardization
  5. Institute for Software Technology – Maximilian Fellner 2013 Existing approaches

    to language specification ‣ Natural language ‣ Formal semantics ‣ Reference implementations ‣ Test suites ‣ Syntax specified by context-free grammars and regular expressions 5
  6. Institute for Software Technology – Maximilian Fellner 2013 Existing approaches

    to language specification ‣ Specification documents,
 e.g., ECMA-262 (JavaScript) ‣ Readily understood by most ‣ Free choice of content and format ‣ Cannot be automatically verified 6 Natural language
  7. Institute for Software Technology – Maximilian Fellner 2013 Existing approaches

    to language specification ‣ Can be difficult to read and write ‣ Rarely used for consumer software ‣ Most precise, unambiguous, provable ‣ Tool support, e.g, VDM-SL 7 Formal semantics
  8. Institute for Software Technology – Maximilian Fellner 2013 ‣ Are

    self-sufficient ‣ Written in a programming language ‣ Possible bugs & unexpected behavior ‣ Dependency on implementation details 8 Reference implementations Existing approaches to language specification
  9. Institute for Software Technology – Maximilian Fellner 2013 ‣ Specification

    consists of tests written in the specified language ‣ Can define positive and negative test cases ‣ Should be executable by any implementation ‣ Nondeterministic behavior difficult to specify 9 Test suites Existing approaches to language specification
  10. Institute for Software Technology – Maximilian Fellner 2013 ‣ Languages

    with greater abstraction ‣ No tool support for specification ‣ Visual tests not feasible 10 Visual programming languages Existing approaches to language specification
  11. Institute for Software Technology – Maximilian Fellner 2013 Can we

    combine the advantages of natural language and those of test suites? 11 Question:
  12. Institute for Software Technology – Maximilian Fellner 2013 Specification by

    Example 12 From business goals to executable specification ‣ Introduced by Gojko Adzic ‣ Agile methodology ‣ Acceptance Test-Driven Development ‣ Behavior-Driven Development
  13. Institute for Software Technology – Maximilian Fellner 2013 13 Right

    Product Built Right Business Failure Success Useless Software Maintenance Difficulties test-driven development programming languages frameworks & tools specification by example project management Success of a software product, adapted from Adzic [1] business intent
  14. Institute for Software Technology – Maximilian Fellner 2013 Question: How

    can tests better specify the business intent? 14
  15. Institute for Software Technology – Maximilian Fellner 2013 15 Customer

    Tests (or Acceptance Tests) Business Intent Usability Testing Is it pleasurable? Component Tests Architect Intent Exploratory Testing Is it self-consistent? Unit Tests Developer Intent Property Testing Is it responsive, secure, scalable? Per Functionality Cross-Functional Business Facing Technology Facing The testing matrix, adapted from Meszaros [2]
  16. Institute for Software Technology – Maximilian Fellner 2013 Behavior-Driven Development

    ‣ Derived from test-driven development ‣ Strong community interest & many tools ‣ Ubiquitous language 16 Specification by example concretized
  17. Institute for Software Technology – Maximilian Fellner 2013 What is

    a ubiquitous language? A common language for a shared understanding of the domain. 17 E. Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software [3]
  18. Institute for Software Technology – Maximilian Fellner 2013 18 Title

    (one line describing the story) ! Narrative: As a [role] I want [feature] So that [benefit] ! Acceptance Criteria: (presented as Scenarios) Scenario: Title Given [context] And [some more context]… When [event] Then [outcome] And [another outcome]… D. North. What’s in a story? [4]
  19. Institute for Software Technology – Maximilian Fellner 2013 Cucumber ‣

    Open-source project ‣ Ubiquitous language Gherkin ‣ Available for Ruby and JVM languages 19 Behavior-driven development tool
  20. Institute for Software Technology – Maximilian Fellner 2013 Describing a

    Calculator with Gherkin Feature: Addition Scenario: Adding two numbers Given I enter 1 into the calculator And I enter 2 into the calculator When I press add Then I should see 3 on the display 20
  21. Institute for Software Technology – Maximilian Fellner 2013 Given /^I

    enter (d+) into the calculator$/ do |n| @stack.push n.to_i end 21 Then /^I should see (d+) on the display$/ do |n| @result.should == @result end When /^I press add^$/ do while not @stack.emtpy? do @result += @stack.pop end end
  22. Institute for Software Technology – Maximilian Fellner 2013 22 Project

    Features Scenarios Steps Step Definitions Support Code Automation Library System Business Facing Technology Facing The Cucumber testing stack, adapted from Hellesoy and Wynne [5]
  23. Institute for Software Technology – Maximilian Fellner 2013 Cucumber ‣

    Gherkin is simple but effective ‣ Steps are platform independent ‣ Definition methods are reusable ‣ Redundant code is reduced 23 Advantages of steps & step definitions
  24. Institute for Software Technology – Maximilian Fellner 2013 24 However,

    there is a problem… They do not support Ruby or the JVM! ® ®
  25. Institute for Software Technology – Maximilian Fellner 2013 Existing approaches

    to Cucumber on mobile ‣ Open-source software by Xamarin Inc. ‣ Extends Ruby variant of Cucumber ‣ Supports Google Android & Apple iOS 25 E.g., Calabash
  26. Institute for Software Technology – Maximilian Fellner 2013 26 developer’s

    computer or build server Ruby client library & step definitions Cucumber features device or emulator Calabash HTTP server & native test runner Mobile Application Calabash system architecture
  27. Institute for Software Technology – Maximilian Fellner 2013 Existing approaches

    to Cucumber on mobile ‣ Ruby installation required ‣ Only pre-defined operations available ‣ Step-definitions are written in Ruby ‣ No integration with native libraries 27 Disadvantages
  28. Institute for Software Technology – Maximilian Fellner 2013 Cucumber JVM

    on Android ‣ Use Java implementation of Cucumber ‣ Integrate with Android test framework ‣ Run Cucumber directly on the device 28 A different approach
  29. Institute for Software Technology – Maximilian Fellner 2013 29 Overview

    of the Android test framework application package (.apk) InstrumentationTestRunner test package (.apk) test tools monkey runner test case classes Instrumentation JUnit mock objects process
  30. Institute for Software Technology – Maximilian Fellner 2013 30 Class

    diagram of the Cucumber-Android module Instrumentation CucumberInstrumenation Runtime RuntimeOptions AndroidResourceLoader ClassLoader JavaBackend 1..* DexClassFinder AndroidObjectFactory ResourceLoader ClassFinder ObjectFactory 1 1 1 1 1 1 1
  31. Institute for Software Technology – Maximilian Fellner 2013 Cucumber JVM

    on Android ‣ Still no solution for Windows Phone and iOS ‣ Best option is to improve Cucumber ‣ Gherkin parser is based on Ragel ‣ Cucumber runtime can be ported to C 31 Remaining issue
  32. Institute for Software Technology – Maximilian Fellner 2013 The visual

    language Catrobat ‣ Android, Windows Phone, iOS, Web ‣ Phones, tablets & desktop browsers ‣ Inspired by Scratch ‣ Sophisticated bricks (blocks) 32 Cross-platform, mobile, visual
  33. Institute for Software Technology – Maximilian Fellner 2013 The visual

    language Catrobat 33 Pocket Code (Android) and Scratch 2.0
  34. Institute for Software Technology – Maximilian Fellner 2013 From requirements

    to finished product ‣ Team meetings and discussions ‣ Write requirements as story cards ‣ Implement stories using Kanban & TDD 34 Before
  35. Institute for Software Technology – Maximilian Fellner 2013 35 weekly

    sub- team meeting bi-weekly team meeting planning game create new stories discuss implementation get feedback write unit tests
  36. Institute for Software Technology – Maximilian Fellner 2013 From requirements

    to finished product ‣ Stories are imprecise ‣ Story cards are archived or thrown away ‣ No automatic validation ‣ Android implementation serves as the model for other teams 36 Problems
  37. Institute for Software Technology – Maximilian Fellner 2013 From requirements

    to finished product ‣ Translate stories into features and scenarios ‣ Share feature files with everybody ‣ Each team implements their own step- definitions independently 37 With Cucumber
  38. Institute for Software Technology – Maximilian Fellner 2013 38 weekly

    sub- team meeting bi-weekly team meeting planning game create new stories discuss implementation get feedback write unit tests write cucumber features
  39. Institute for Software Technology – Maximilian Fellner 2013 From requirements

    to finished product ‣ Features and scenarios are precise ‣ Feature files are stored in SCM system (Git) ‣ Automated validation of specifications ‣ Implementation-agnostic model for all teams ‣ Better synchronization between teams 39 Advantages
  40. Institute for Software Technology – Maximilian Fellner 2013 Specifying semantics

    of a visual language ‣ Android implementation in Java ‣ Furthest developed variant ‣ Existing set of unit tests 40 Catrobat & Pocket Code
  41. Institute for Software Technology – Maximilian Fellner 2013 Creating a

    new Catrobat program Background: Given I have a program And this program as an object ‘Object’ 41
  42. Institute for Software Technology – Maximilian Fellner 2013 @Given(“^I have

    a Program$”) public void I_have_a_program() throws IOException { ProjectManager pm = ProjectManager.getInstance(); pm.initializeNewEmptyProject(“Cucumber”, getContext()); } 42 Step definitions should be self-contained
  43. Institute for Software Technology – Maximilian Fellner 2013 @And(“^this program

    has an Object ‘(\\w+)’$”) public void program_has_object(String name) { ProjectManager pm = ProjectManager.getInstance(); Sprite sprite = new Sprite(name); pm.getCurrentProject().addNewObject(sprite); Cucumber.put(Cucumber.KEY_CURRENT_OBJECT, sprite); } 43 Step definitions can require a mechanism to share state
  44. Institute for Software Technology – Maximilian Fellner 2013 Behavior-Driven Development

    asks: How will the system behave when a user interacts with it? 44
  45. Institute for Software Technology – Maximilian Fellner 2013 How will

    a Catrobat program behave when it is executed? 45 The language specification should answer:
  46. Institute for Software Technology – Maximilian Fellner 2013 Specifying a

    loop (Repeat brick) Feature: Repeat brick A Repeat Brick should repeat another set of bricks a given number of times. ! Background: ... ! Scenario: Increment variable inside loop Given ‘Object’ has a Start script And this script has a set ‘i’ to 0 brick And this script has a Repeat 8 times brick And this script has a change ‘i’ by 1 brick And this script has a Repeat end brick When I start the program And I wait until the program has stopped Then the variable ‘i’ should be equal 8 46
  47. Institute for Software Technology – Maximilian Fellner 2013 Specifying a

    loop (Repeat brick) Feature: Repeat brick A Repeat Brick should repeat another set of bricks a given number of times. ! Background: ... ! Scenario: Increment variable inside loop Given ‘Object’ has a Start script And this script has a set ‘i’ to 0 brick And this script has a Repeat 8 times brick And this script has a change ‘i’ by 1 brick And this script has a Repeat end brick When I start the program And I wait until the program has stopped Then the variable ‘i’ should be equal 8 47 Add script to Object for 8 times do i = i + 1; end
  48. Institute for Software Technology – Maximilian Fellner 2013 @Then(“^the variable

    ‘(\\w+)’ should be equal (\\d+.?\\d*)$”) public void var_should_equal_float(String name, float expected) { Sprite object = (Sprite) Cucumber.get(Cucumber.KEY_CURRENT_OBJECT); Project project = ProjectManager.getInstance().getCurrentProject(); UserVariable var = project.getUserVariables() .getUserVariable(name, object); assertNotNull(var); float actual = var.getValue().floatValue(); assertThat(actual, equalTo(expected)); } 48 Step definition for equality assertion
  49. Institute for Software Technology – Maximilian Fellner 2013 49 Automatically

    generated report
  50. Institute for Software Technology – Maximilian Fellner 2013 50 Using

    the loop (Repeat brick) Altering position and transparency
  51. Institute for Software Technology – Maximilian Fellner 2013 51 Using

    the loop (Repeat brick) Question of performance
  52. Institute for Software Technology – Maximilian Fellner 2013 Specifying performance

    of a loop Scenario: No more than 100 iterations in 2 seconds Given ‘Object’ has a Start script And this script has a set ‘i’ to 0 brick And this script has a set ‘k’ to 0 brick And this script has a Wait 2 seconds brick And this script has a set ‘k’ to ‘i’ brick ! Given ‘Object’ has a Start script And this script has a Repeat 400 times brick And this script has a change ‘i’ by 1 brick And this script has a Repeat end brick ! When I start the program And I wait until the program has stopped Then the variable ‘k’ should be less than or equal 100 52
  53. Institute for Software Technology – Maximilian Fellner 2013 Defining an

    upper bound for iterations per second Scenario: No more than 100 iterations in 2 seconds Given ‘Object’ has a Start script And this script has a set ‘i’ to 0 brick And this script has a set ‘k’ to 0 brick And this script has a Wait 2 seconds brick And this script has a set ‘k’ to ‘i’ brick ! Given ‘Object’ has a Start script And this script has a Repeat 400 times brick And this script has a change ‘i’ by 1 brick And this script has a Repeat end brick ! When I start the program And I wait until the program has stopped Then the variable ‘k’ should be less than or equal 100 53 first Start script second Start script store value of i after 2 s. increment i by 1 1 loop should take at least 20 ms
  54. Institute for Software Technology – Maximilian Fellner 2013 Specifying concurrency

    control (Broadcast Wait) Scenario: Broadcast Wait brick sends message in program with two When scripts Given ‘Object’ has a Start script And this script has a BroadcastWait ‘hello’ brick And this script has a Print brick with ‘-S1-’ Given ‘Object’ has a When ‘hello’ script And this script has a Wait 100 milliseconds brick And this script has a Print brick with ‘-W1-’ Given ‘Object’ has a When ‘hello’ script
 And this script has a Wait 200 milliseconds brick And this script has a Print brick with ‘-W2-’ When I start the program And I wait until the program has stopped Then I should see the printed output ‘-W1--W2--S1-’ 54
  55. Institute for Software Technology – Maximilian Fellner 2013 Specifying concurrency

    control (Broadcast Wait) Scenario: Broadcast Wait brick sends message in program with two When scripts Given ‘Object’ has a Start script And this script has a BroadcastWait ‘hello’ brick And this script has a Print brick with ‘-S1-’ Given ‘Object‘ has a When ‘hello’ script And this script has a Wait 100 milliseconds brick And this script has a Print brick with ‘-W1-’ Given ‘Object’ has a When ‘hello’ script
 And this script has a Wait 200 milliseconds brick And this script has a Print brick with ‘-W2-’ When I start the program And I wait until the program has stopped Then I should see the printed output ‘-W1--W2--S1-’ 55 Start script initiates two When scripts first When script runs ≥ 100 ms second When script runs ≥ 200 ms output shows execution order of scripts
  56. Institute for Software Technology – Maximilian Fellner 2013 ‣ Functional

    ‣ Expected output of a program ‣ Object-oriented ‣ Properties of objects (attributes & methods) ‣ Hardware-oriented ‣ Comparable performance ‣ Measured values from sensors 56 Specifying semantics of a visual language
  57. Institute for Software Technology – Maximilian Fellner 2013 Conclusion ‣

    Working implementation on Android ‣ Cross-platform with Cucumber is feasible 57 Cucumber for the Catrobat language
  58. Institute for Software Technology – Maximilian Fellner 2013 ‣ Cucumber

    extends present test-driven approach ‣ New features can be developed inside existing project workflow ‣ Features and steps can be shared between teams working on different platforms 58 Advantages Conclusion
  59. Institute for Software Technology – Maximilian Fellner 2013 ‣ Features

    and scenarios need to be well- formulated ‣ When does a certain property correspond to a feature and when to a scenario? ‣ Step definitions need to be implemented per-platform and depend on existing code 59 Issues with Cucumber features Conclusion
  60. Institute for Software Technology – Maximilian Fellner 2013 ‣ Adopt

    Cucumber throughout all Catrobat sub-teams ‣ Port Cucumber to C or C++ ‣ Enhance development process with continuous integration & cucumber.pro 60 Future work Conclusion
  61. Institute for Software Technology – Maximilian Fellner 2013 61 Bibliography

    1. G. Adzic. Specification by Example: How Successful Teams Deliver the Right Software. Manning Pubs Co Series. Manning, 2011. isbn: 9781617290084. 2. G. Meszaros. XUnit Test Patterns: Refactoring Test Code. A Martin Fowler signature book. Addison Wesley Professional, 2007. isbn: 9780131495050. 3. E. Evans. Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison Wesley Professional, 2004. isbn: 9780321125217. 4. D. North. What’s in a story? Online; accessed 2013-10-05. url: http: //dannorth.net/whats-in-a-story. 5. A. Hellesoy and M. Wynne. The Cucumber Book: Behaviour-Driven Development for Testers and Developers. Pragmatic Programmers. Pragmatic Book- shelf, 2012. isbn: 9781934356807.