Conference for Kotliners - Kotlin: the Next Frontier in Metaprogramming

Conference for Kotliners - Kotlin: the Next Frontier in Metaprogramming

In an epic quest using Kotlin and native desktop development to generate UI testing for the TornadoFX framework, we turn to metaprogramming in Kotlin to explore AST language parsing and the philosophy of design-driven UI testing in the pursuit for smarter test-generation.

209bed5a8a467a07a397b25c00a26e9e?s=128

mvndy_hd

June 07, 2019
Tweet

Transcript

  1. 1.

    Kotlin: The Next Frontier in Metaprogramming Using metaprogramming to formalize

    the theory of human error in applications in generative UI testing Amanda Hinchman-Dominguez @hinchman_amanda
  2. 2.

    UIs increasingly critical to the success of applications • An

    application’s GUI must create a rich, intuitive and pleasant user experience • Validate + guarantee quality assurance of event-driven applications through the active process of exploring an environment
  3. 3.
  4. 4.
  5. 7.

    Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  6. 8.

    Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  7. 9.

    E2E Integration Unit Black box testing • Tests for the

    behavior/ functionality of software White box testing • Tests how the system functions UI Testing Quality Assurance
  8. 11.

    A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  9. 12.

    A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  10. 13.

    • Kastree - a simple wrapper around the Kotlin compiler

    • Program Structure Interface (PSI): the layer in IntelliJ parses syntactic and semantic code models • An Abstract Syntax Tree (AST) parser surpasses challenges found in statically-typed metaprogramming: • Access levels • Type casting in recursion Phase 1: Detecting Inputs w/ AST Parsing
  11. 15.

    override val root = hbox { form { fieldset("Edit person")

    { field("Owner") { textfield(model.ownerName) { ownerNameField = this } } field("Cat") { textfield(model.catName) { catNameField = this } } field("Time") { textfield(model.time) { timeField = this } } button("Save") { enableWhen(model.dirty) action { save() } } } } } Saving UI Nodes field fieldset form textfield field textfield field field textfield field fieldset form hbox textfield hbox field textfield textfield button button
  12. 16.
  13. 17.

    A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  14. 18.

    A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  15. 19.

    • Kotlin scripting to write Kotlin test files for node

    interactions • TestFX robots to stimulate user interactions, check node states with matchers • Dynamically insert randomized ids for detected nodes at runtime Phase 2: Generating Tests with TestFX
  16. 20.

    • Kotlin scripting to write Kotlin test files for node

    interactions • TestFX robots to stimulate user interactions, check node states with matchers • Dynamically insert randomized ids for detected nodes at runtime Phase 2: Generating Tests with TestFX
  17. 21.

    A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  18. 22.
  19. 23.
  20. 24.

    How do we know if a node looks as it

    should? • Mapping a compositional abstraction for our components can help define properties surrounding actions of our views • When a control event is triggered, apply binary operations against states and check against the contract holistic result of an application’s representative compositional map
  21. 25.

    Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  22. 26.

    Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  23. 27.

    Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  24. 28.

    button form hbox override val root = hbox { form

    { fieldset("Edit person") { field("Owner") { textfield(model.ownerName) { ownerNameField = this } } field("Cat") { textfield(model.catName) { catNameField = this } } field("Time") { textfield(model.time) { timeField = this } } button("Save") { enableWhen(model.dirty) action { save() } } } } } fieldset form fieldset field field field field button textfield field textfield textfield textfield field textfield textfield field textfield Post-parsing analysis
  25. 30.

    UINode( nodeType = “Button”, uiLevel = 3, nodeChildren = JsonObject(),

    valueAssignment = “”, functionComposition = Digraph<UIFunctionComposition>(), associatedFunctions = ArrayList<String>() ) button Post-parsing analysis
  26. 31.

    UINode( nodeType = “Button”, uiLevel = 3, nodeChildren = JsonObject(),

    valueAssignment = “”, functionComposition = Digraph<UIFunctionComposition>(), associatedFunctions = ArrayList<String>() ) button Post-parsing analysis
  27. 35.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  28. 36.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  29. 37.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  30. 38.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  31. 39.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  32. 40.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  33. 41.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  34. 42.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  35. 43.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  36. 44.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  37. 45.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  38. 46.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  39. 47.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  40. 48.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  41. 49.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  42. 50.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  43. 51.

    avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) Associated Functions
  44. 52.

    button("Save") { enableWhen(model.dirty) action { save() update(model) } } Functional

    composition - what functions are spurned from interacting with a particular node?
  45. 54.

    Function Composition button("Save") { enableWhen(model.dirty) action { save() update(model) }

    } private fun save() { model.commit() val catSchedule = model.item close() } private fun update(model: CatScheduleModel) { println("Contact updated!") controller.changeCatAvi(model.item) } save save action fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } changeCatAvi changeCatAvi update update
  46. 55.

    Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  47. 56.

    Creating Finite State Machines to predict node behavior Cat Schedule

    Model Save Button Dirty Not Dirty Disabled Enabled
  48. 57.

    Creating Finite State Machines to predict node behavior Cat Schedule

    Model Save Button Dirty Not Dirty Disabled Enabled
  49. 58.

    Cat Schedule Model Save Button Dirty Not Dirty Disabled Enabled

    enum class Toggle { Enable, Disable; fun isButtonEnabled(): Boolean { return when (this) { Enable -> true Disable -> false } } fun enableOnDirty(model: ModelState): Toggle { return when (model) { ModelState.Dirty -> Enable ModelState.NotDirty -> Disable } } } enum class ModelState { Dirty, NotDirty; fun initModelStateTransition(): ModelState { return when (this) { Dirty -> NotDirty NotDirty -> Dirty } } }
  50. 59.

    sealed class EditorFragment { var modelState = ModelState.NotDirty fun commitModel()

    {} fun onChangeText() { modelState.initModelStateTransition() } open class OwnerTextField: EditorFragment() {} open class CatTextField: EditorFragment() {} open class TimeTextField: EditorFragment() {} class SaveButton(var toggle: Toggle = Toggle.Disable): EditorFragment() { fun toggleButtonOnDirtyModel(model: ModelState) { toggle.enableOnDirty(model) } fun saveModel() { if (toggle.isButtonEnabled()) commitModel() } } }
  51. 60.

    sealed class EditorFragment { var modelState = ModelState.NotDirty fun commitModel()

    {} fun onChangeText() { modelState.initModelStateTransition() } open class OwnerTextField: EditorFragment() {} open class CatTextField: EditorFragment() {} open class TimeTextField: EditorFragment() {} class SaveButton(var toggle: Toggle = Toggle.Disable): EditorFragment() { fun toggleButtonOnDirtyModel(model: ModelState) { toggle.enableOnDirty(model) } fun saveModel() { if (toggle.isButtonEnabled()) commitModel() } } }
  52. 61.

    Specifying UIs Formally through Metaprogramming Benefits: • Implicit: Clearly expressing

    ourselves leads to a better understanding of the system • Explicit: Using formal specs to check properties - does our UI have any dead ends? Can we get locked out of a chunk of the UI?
  53. 62.

    Promising discoveries in metaprogramming analysis research • Metaprogramming locator mechanism

    is more accurate/less intensive detecting UI components in applications than current AI locator mechanisms • Current AI image classification locator mechanisms suffer from partial view render recognition, detecting custom object interactions
  54. 63.

    Promising discoveries in metaprogramming analysis research • Using Kotlin metaprogramming

    to further analyze the relationship of UI hierarchies, functional composition, and generated test results • Potential for generating unit testing in the future • Traditional machine learning only offers immediate extrinsic reward analysis; metaprogramming generated relation analysis may provide more in-depth analysis to supplement future deep learning
  55. 64.

    Future Plans for TornadoFX-Suite • DSLs for more flexible test

    building + abstracted compositional mapping • Make AST parsing stronger + pull out into an independent library • Expand TornadoFX-Suite to analyze other UI frameworks • Collect voluntary data from future users to begin deep learning
  56. 65.

    • TornadoFX-Suite: https://github.com/ahinchman1/TornadoFX-Suite • Automata UI research: https://github.com/ahinchman1/Finite-State-Machine-Crash-Course • Data

    science research: https://github.com/ahinchman1/Data-Science-Crash-Course • Neighborhood Cat Scheduler: https://github.com/Kotlin-Thursdays/Neighborhood-Cat-Scheduler • TornadoFX-DnD-TilesFX: https://github.com/ahinchman1/TornadoFX-DnD-TilesFX Amanda Hinchman-Dominguez @hinchman_amanda Relevant Links