Metamorphic Domain-Specific Languages

Metamorphic Domain-Specific Languages

External or internal domain-specific languages (DSLs) or (fluent)
APIs? Whoever you are – a developer or a user of a DSL –
you usually have to choose side; you should not! What about
metamorphic DSLs that change their shape according to your
needs? Our 4-years journey of providing the "right" support
(in the domain of feature modeling), led us to develop an external
DSL, different shapes of an internal API, and maintain
all these languages. A key insight is that there is no one-size-fits-
all solution or no clear superiority of a solution compared
to another. On the contrary, we found that it does make sense
to continue the maintenance of an external and internal DSL.
Based on our experience and on an analysis of the DSL engineering
field, the vision that we foresee for the future of
software languages is their ability to be self-adaptable to the
most appropriate shape (including the corresponding integrated
development environment) according to a particular
usage or task. We call metamorphic DSL such a language,
able to change from one shape to another shape.

The talk has been presented at SPLASH conference in Portland (USA), Onward! Essays track.

Bb59170948068df05f09ab11a9f3f35e?s=128

FAMILIAR project

October 22, 2014
Tweet

Transcript

  1. Metamorphic Domain-Specific Languages   Mathieu Acher, Benoit Combemale, Philippe Collet

  2. Acknowledgements Thomas Degueule, Guillaume Bécan, Olivier Barais, Julien Richard-Foy, Jean-Marc

    Jézéquel members and Jonathan Aldrich
  3. Graphviz Make Matlab PGN Finite State Machine Domain-Specific Languages (DSLs)

  4. « Another lesson we should have learned from the recent

    past is that the development of 'richer' or 'more powerful' programming languages was a mistake in the sense that these baroque monstrosities, these conglomerations of idiosyncrasies, are really unmanageable, both mechanically and mentally. I see a great future for very systematic and very modest programming languages » ACM Turing Lecture, « The Humble Programmer » Edsger W. Dijkstra 4 aka Domain- Specific Languages 1972
  5. 5 2011

  6. 6 2011 « Domain-specific languages are far more prevalent than

    anticipated »
  7. Syntax + Services DSL = Specialized notation: Textual or Graphical

    Specific Vocabulary Idiomatic constructs Specialized tools/IDE: Editor with auto-completion, syntax highlighting, etc. Compiler Interpreter Debugger Profiler Syntax/Type Checker …
  8. Domain-Specific Languages (DSLs) Why DSLs exist? Because there is no

    one-size-fits-all solution!
  9. Domain-Specific Languages (DSLs) General Purpose Languages (e.g., Java, Scala, Haskell,

    Ruby) No one-size-fits-all solution! We need DSLs, whatever they’re
  10. Domain-Specific Language (DSL) Even for a given domain and class

    of problem, there is no one-size-fits-all solution! (Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL)
  11. Even for a given domain and class of problem, there

    is no one-size-fits-all solution! (Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL) Polymorphic DSLs: different shapes
  12. (Java) (Plain text SQL) Notation and Services differ. Pros. Cons.

    in both sides.
  13. (Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL) Metamorphic DSLs:

    moving from one shape to another
  14. shape #3 shape #2 shape #1 Scala (internal DSL) Java

    (internal DSL) Plain SQL (external DSL)
  15. shape #3 shape #2 shape #1 Scala (internal DSL) Java

    (internal DSL) FAMILIAR (external DSL) shape #0 Java (SPLAR API)
  16. Metamorphic DSL Vision: software languages should be self- adaptable to

    the most appropriate shape (including the corresponding IDE) and according to a particular usage or task. shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) Plain SQL (external DSL)
  17. •  Analysis of the DSL erea – Socio-technical aspects of DSLs

    suggest the idea of supporting different shapes •  4-years experience of developping different shapes of a DSL – FAMILIAR (for performing operations over feature models) •  Scenarios with Metamorphic DSLs 17 Where the idea of metamorphic DSL comes from?
  18. None
  19. •  Traditional dichotomy between internal DSL and external DSL (Fowler

    et al., 2010) – Fluent APIs – Internal DSLs – (deeply) embedded DSLs – External DSLs – What’s LINQ? •  Boundary between DSL and GPL is not that clear (Voelter et al., 2013) – What is and what is not a DSL is still a debate 19 #1 Diversity of terminology
  20. #1 The diversity of terminology shows the large spectrum of

    shapes DSLs can take
  21. •  Promises of DSL« improvement » in terms of – 

    usability, learnability, expressiveness, reusability, etc. •  Empirical study on the role of syntax –  C-style syntax induces problems in terms of usability for novices; language more or less intuitive for (non-)programmers (Stefik et al. 2014) –  Syntax issues with Java for students (Denny et al. 2011) –  PL usability: method namings/placement, use of identifiers, API design (Ellis et al., Styllos et al., Clarke, Montperrus et al., etc.) •  More specialized/sophicated tools/IDE can be derived from a DSL –  editors, compilers, debuggers 21 #2 Syntax and Environment Matter
  22. #2 As syntax and development environment matter, we should allow

    the user to choose the right shape of a DSL Meyerovich and Rabkin « Empirical analysis of programming language adoption » OOPSLA’13
  23. 23 #3 Language Workbenches Erdweg et al. SLE’13

  24. #3 The community of language engineering is providing more and

    more mature solutions for building DSLs – being external or internal. Developers of DSLs have now a variety of strategies to choose from and build an appropriate shape.
  25. Shaping Up DSL! •  Diversity of terminological clarifications •  Role

    of syntax and environment •  Developers can devise new DSL •  All suggest the idea of having different shapes of a DSL •  What is missing is a systematic solution for transitioning from one shape to another –  We would like to open a given artefact (expressed in a DSL) with another syntax and another environment –  Metamorphic DSL
  26. None
  27. In 2010, we have developed theoretical foundations, efficient algorithms for

    a certain number of operations (merge, aggregate, slice, refactor, synthesis, diff, etc.) over feature models (see Acher PhD thesis) Optional Mandatory Xor-Group Or-Group Organization of « features » (configuration options) into a hierarchy Specification of what combinations of features (configurations) are allowed
  28. FAMILIAR (Acher et al. 2013) •  In 2010, we have

    developed theoretical foundations, efficient algorithms for a certain number of operations (merge, aggregate, slice, refactor, synthesis, diff, etc.) over feature models •  One question that came across: how to provide the support (language + environment) so that people (partners, researchers, students, us) can use our operations?
  29. shape #0 Java (SPLAR API)

  30. #0 •  Too much verbose and technical, right? •  Don’t

    get it wrong: SPLAR is a great API (certainly the most popular) but… –  the audience (i.e., researchers in charge of developping/ benchmarking efficient algorithms) –  the design goal (i.e., extensible mechanisms to adapt reasoning mechanisms) does not fit our purpose •  We simply wanted to offer a way to merge two feature models and then, say, count the number of configuration and configure it –  Three lines right?
  31. OK 6 lines ;-)

  32. #1 •  Problem resolved? •  External nature of the DSL

    poses two kinds of problems – Integration of FAMILIAR to other applications – Limits of its expressiveness •  We have added foreach-like loop, if-then-else, and even ways to define reusable scripts… Also facilities for manipulating strings (like concatenation) •  At some points, we have even doubted that FAMILIAR was a DSL (~a kind of restricted GPL)
  33. #2 •  We shift to a fluent Java API – easier

    to integrate to other Java-based tools – foreach / if then else / string manipulation are already there
  34. #3 •  Suboptimal notation (e.g., still verbose) •  REPL

  35. What the hell is going on? •  3 attempts. THREE.

    •  But no superior / one-size-fits-all solution – We still like the external solution: (1) concise notation and the dedicated support ; (2) when communicating with other researchers, students, or partners – We still like the Java/Scala solution when we want to build variability-intensive, integrated applications
  36. WebFML We still like the external solution (1) concise notation

    and dedicated support ; (2) when communicating with other researchers, students, or partners
  37. WebFML Effective communication Understandability Learnability

  38. We still like the Java/Scala solution when we want to

    build variability-intensive, integrated applications Expressiveness Integration to other systems Tuning of internal details (solvers)
  39. What the hell is going on? •  3 attempts. THREE.

    •  But no superior / one-size-fits-all solution •  Are we all wrong? – No! Keeping all variants does make sense, with pros and cons depending on the « context » •  Metamorphic DSL – We would like to transition from one shape to the other
  40. shape #3 shape #2 shape #1 Scala (internal DSL) Java

    (internal DSL) FAMILIAR (external DSL) shape #0 Java (SPLAR API)
  41. Same kind of story with SQL •  Do you want

    to learn SQL with Java? •  Can you rule the world with only SQL? shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) Plain SQL (external DSL)
  42. Metamorphic DSL IDE 1 a DSL Specification a DSL Articfact

    IDE 2
  43. None
  44. Scenario #1 •  Help people to quickly learn the language

    and transition to another shape when needs be – NB: can be the same person!
  45. Scenario #2 •  People can use the more advanced support

    for understanding / debugging feature models – NB: can be the same person!
  46. Scenario #(1+2) •  People can use the more advanced support

    for understanding / debugging feature models – And back again!
  47. Scenario #1 •  Help people to quickly learn the language

    and transition to another shape when needs be – NB: can be the same person!
  48. Scenario #2 •  People can use the more advanced support

    for understanding / debugging SQL queries – NB: can be the same person!
  49. Scenario #(1+2) •  People can use the more advanced support

    for understanding / debugging SQL queries – And back again!
  50. Scenario #(Putting All Together) Say a company wants to develop

    a web configurator for assisting customers in the selection of products
  51. product manager mysql>'SELECT&*&FROM&products; +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |'productID'|'productCode'|'name''''''|'quantity'|'gps''''''''| +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |''''''1001'|'PEN'''''''''|'Car'Red'''|'''''5000'|'''''''true'| |''''''1002'|'PEN'''''''''|'Car'Blue''|'''''8000'|'''''''true'| |''''''1003'|'PEN'''''''''|'Car'Black'|'''''2000'|'''''''true'| |''''''1004'|'PEC'''''''''|'Car'2B''''|''''10000'|'''''''true'|

    |''''''1005'|'PEC'''''''''|'Car'2H''''|'''''8000'|'''''''false| |''''''1006'|'PEC'''''''''|'Car'HB''''|''''''''0'|'''''''false|'' +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ 6'rows'in'set'(0.02'sec)
  52. We also have an Eclipse version marketing engineer

  53. software engineer (Java API)

  54. software engineer (Java API) marketing engineer product manager

  55. Metamorphic DSLs FAMILIAR definition a SQL query a FAMILIAR script

    <<conformsTo>> <<editedWith>> <<generates>> SQL definition mysql>'SELECT&*&FROM&products; +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |'productID'|'productCode'|'name''''''|'quantity'|'gps''''''''| +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |''''''1001'|'PEN'''''''''|'Car'Red'''|'''''5000'|'''''''true'| |''''''1002'|'PEN'''''''''|'Car'Blue''|'''''8000'|'''''''true'| |''''''1003'|'PEN'''''''''|'Car'Black'|'''''2000'|'''''''true'| |''''''1004'|'PEC'''''''''|'Car'2B''''|''''10000'|'''''''true'| |''''''1005'|'PEC'''''''''|'Car'2H''''|'''''8000'|'''''''false| |''''''1006'|'PEC'''''''''|'Car'HB''''|''''''''0'|'''''''false|'' +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ 6'rows'in'set'(0.02'sec) product manager marketing engineer software engineer
  56. None
  57. Research Directions •  What and where are metamorphic DSLs? • 

    Empirical studies for investigating the role of syntax and environment •  Solution for building metamorphic DSL •  Solution in the large: does metamorphic DSL pay off?
  58. #1 Research Direction (metamorphic classification) •  What are metamorphic DSLs?

    – Precise definition/scope is missing •  Where are metamorphic DSLs? – Empirical observation and inventory ongoing •  Are all DSLs metamorphic?
  59. •  H0: Syntax and environment matter –  H1: suboptimal notation

    can decrease understandibility –  H2: non comprehensive environment support can drecrease usability or productivity –  … –  Hn: … •  But empirical evidence is missing –  e.g., JOOQ for new learners <<<<< plain SQL for new learners? –  e.g., Impact of the absence of profiling/benchmark tools #2 Research Direction (empirical studies)
  60. •  Solution for building metamorphic DSL –  Bi-directional transformations for

    moving from one shape to another (and back again) –  Engineering a new shape of a DSL (when the shape is missing) –  Reuse, « core » •  HCI issues –  Copy and paste? –  Open with? –  Features of projectional editors? –  Self-mechanism? #3 Research Direction (solution)
  61. •  Engineering metamorphic DSLs – Does it pay off for users

    and developers? •  From the users’ perspective « switching » can have a cognitive cost •  From the developers’ perspective it requires effort (hopefully little!) •  Metamorphic scenarios – Communication/socio-technical issues •  Case studies, controlled experiments #4 Research Direction (solution in the large)
  62. None
  63. •  Languages with specialized notation and services exist because a

    one size-fits-all-solution is unlikely = DSLs •  Follow up argument: a one-size-fits-all solution is unlikely even for a given DSL (domain/class of problem) –  Different shapes of a DSL –  How to transition from one shape to another? •  Both users, developers, and researchers want to shape up DSLs •  All based on practical experience (FAMILIAR) and observation (e.g., literature, SQL) Where the idea of Metamorphic DSL comes from?
  64. Metamorphic DSL •  Optimistic view of DSL diversity – We should

    embrace, promote, and support diversity in DSLs ; users are diverse, have different requirements, and are using DSLs in numerous different contexts •  DSLs should be self-adaptable to the most appropriate shape (including the corresponding IDE), according to a particular usage or task
  65. Metamorphic DSL •  Optimistic view of DSL diversity •  DSLs

    should be self-adaptable to the most appropriate shape (including the corresponding IDE), according to a particular usage or task •  A great future for DSL; additional research effort is needed – to further understand the concept of metamorphic DSL (what/where are they? why/when some shapes are more adequate?) – to provide effective solution for transitioning from one shape to another
  66. None
  67. None
  68. HTML Domain: web (markup) 68

  69. https://github.com/julienrf/glitter TCS Wyvern (Omar et al., OOPLSA’14) Scala

  70. None
  71. https://github.com/inukshuk/bibtex-ruby

  72. CSS Domain: web (styling) 72

  73. Makefile Domain: software building 73

  74. Lighthttpd configuration file Domain: web server (configuration) 74

  75. Graphviz Domain: graph (drawing) 75

  76. http://turingatemyhamster.co.uk/blog/?p=157 http://drdozer.github.io/graphviz-s/ https://github.com/glejeune/Ruby-Graphviz/

  77. Regular expression Domain: strings (pattern matching) 77

  78. Portable Game Notation (PGN)

  79. None
  80. None
  81. PGN and DSL promises Easy to read, write, exchange, process

    It is all about games information and moves (nothing more) Success (chess players/tools): 8 millions games
  82. « Alternatives » to PGN (other DSLs) •  For handling

    chess variants – e.g., Chess960 •  Proprietary extensions •  For recording a particular game position – Forsyth–Edwards Notation (FEN) rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/ RNBQKBNR b KQkq e3 0 1
  83. None
  84. I have a question Is it a domain-specific language!??! https://github.com/capicue/pgn

  85. Is it a DSL? YES, yes, yes !!! Just another

    « shape » of PGN
  86. FEN and PGN in Ruby

  87. PGN in Python? https://github.com/renatopp/pgnparser

  88. PGN in ????

  89. Internal Shapes External Shapes

  90. External Shapes (DSLs) Internal Shapes (DSLs)

  91. Metamorphic DSL

  92. None
  93. External DSLs vs Internal DSLs •  An external DSL is

    a completely separate language and has its own custom syntax/ tooling support (e.g., editor) •  An internal DSL is more or less a set of APIs written on top of a host language (e.g., Java). – Fluent interfaces 93
  94. External vs Internal DSL (SQL example) 94

  95. DSL (LINQ/C# example) 95

  96. Internal DSL •  « Using a host language (e.g., Java)

    to give the host language the feel of a particular language. » •  Fluent Interfaces –  « The more the use of the API has that language like flow, the more fluent it is » 96
  97. SQL in… Java DSL in GPL 97

  98. Regular expression in… Java DSL in GPL 98

  99. Internal DSLs vs External DSL •  Both internal and external

    DSLs have strengths and weaknesses (Fowler) –  learning curve, –  cost of building, –  programmer familiarity, –  communication with domain experts, –  mixing in the host language, –  strong expressiveness boundary •  Focus of the course – external DSL a completely separate language with its own custom syntax and tooling support (e.g., editor) 99
  100. None
  101. Actors Actors