Metamorphic Domain-Specific Languages

Slide 1

Slide 1 text

Metamorphic Domain-Specific Languages Mathieu Acher, Benoit Combemale, Philippe Collet

Slide 2

Slide 2 text

Acknowledgements Thomas Degueule, Guillaume Bécan, Olivier Barais, Julien Richard-Foy, Jean-Marc Jézéquel members and Jonathan Aldrich

Slide 3

Slide 3 text

Graphviz Make Matlab PGN Finite State Machine Domain-Specific Languages (DSLs)

Slide 4

Slide 4 text

« Another lesson we should have learned from the recent past is that the development of 'richer' or 'more powerful' programming languages was a mistake in the sense that these baroque monstrosities, these conglomerations of idiosyncrasies, are really unmanageable, both mechanically and mentally. I see a great future for very systematic and very modest programming languages » ACM Turing Lecture, « The Humble Programmer » Edsger W. Dijkstra 4 aka Domain- Specific Languages 1972

Slide 5

Slide 5 text

5 2011

Slide 6

Slide 6 text

6 2011 « Domain-specific languages are far more prevalent than anticipated »

Slide 7

Slide 7 text

Syntax + Services DSL = Specialized notation: Textual or Graphical Specific Vocabulary Idiomatic constructs Specialized tools/IDE: Editor with auto-completion, syntax highlighting, etc. Compiler Interpreter Debugger Profiler Syntax/Type Checker …

Slide 8

Slide 8 text

Domain-Specific Languages (DSLs) Why DSLs exist? Because there is no one-size-fits-all solution!

Slide 9

Slide 9 text

Domain-Specific Languages (DSLs) General Purpose Languages (e.g., Java, Scala, Haskell, Ruby) No one-size-fits-all solution! We need DSLs, whatever they’re

Slide 10

Slide 10 text

Domain-Specific Language (DSL) Even for a given domain and class of problem, there is no one-size-fits-all solution! (Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL)

Slide 11

Slide 11 text

Even for a given domain and class of problem, there is no one-size-fits-all solution! (Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL) Polymorphic DSLs: different shapes

Slide 12

Slide 12 text

(Java) (Plain text SQL) Notation and Services differ. Pros. Cons. in both sides.

Slide 13

Slide 13 text

(Java) (Clojure) (Scala) (C#) (PHP) (Plain text SQL) Metamorphic DSLs: moving from one shape to another

Slide 14

Slide 14 text

shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) Plain SQL (external DSL)

Slide 15

Slide 15 text

shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) FAMILIAR (external DSL) shape #0 Java (SPLAR API)

Slide 16

Slide 16 text

Metamorphic DSL Vision: software languages should be self- adaptable to the most appropriate shape (including the corresponding IDE) and according to a particular usage or task. shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) Plain SQL (external DSL)

Slide 17

Slide 17 text

•  Analysis of the DSL erea – Socio-technical aspects of DSLs suggest the idea of supporting different shapes •  4-years experience of developping different shapes of a DSL – FAMILIAR (for performing operations over feature models) •  Scenarios with Metamorphic DSLs 17 Where the idea of metamorphic DSL comes from?

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

•  Traditional dichotomy between internal DSL and external DSL (Fowler et al., 2010) – Fluent APIs – Internal DSLs – (deeply) embedded DSLs – External DSLs – What’s LINQ? •  Boundary between DSL and GPL is not that clear (Voelter et al., 2013) – What is and what is not a DSL is still a debate 19 #1 Diversity of terminology

Slide 20

Slide 20 text

#1 The diversity of terminology shows the large spectrum of shapes DSLs can take

Slide 21

Slide 21 text

•  Promises of DSL« improvement » in terms of –  usability, learnability, expressiveness, reusability, etc. •  Empirical study on the role of syntax –  C-style syntax induces problems in terms of usability for novices; language more or less intuitive for (non-)programmers (Stefik et al. 2014) –  Syntax issues with Java for students (Denny et al. 2011) –  PL usability: method namings/placement, use of identifiers, API design (Ellis et al., Styllos et al., Clarke, Montperrus et al., etc.) •  More specialized/sophicated tools/IDE can be derived from a DSL –  editors, compilers, debuggers 21 #2 Syntax and Environment Matter

Slide 22

Slide 22 text

#2 As syntax and development environment matter, we should allow the user to choose the right shape of a DSL Meyerovich and Rabkin « Empirical analysis of programming language adoption » OOPSLA’13

Slide 23

Slide 23 text

23 #3 Language Workbenches Erdweg et al. SLE’13

Slide 24

Slide 24 text

#3 The community of language engineering is providing more and more mature solutions for building DSLs – being external or internal. Developers of DSLs have now a variety of strategies to choose from and build an appropriate shape.

Slide 25

Slide 25 text

Shaping Up DSL! •  Diversity of terminological clarifications •  Role of syntax and environment •  Developers can devise new DSL •  All suggest the idea of having different shapes of a DSL •  What is missing is a systematic solution for transitioning from one shape to another –  We would like to open a given artefact (expressed in a DSL) with another syntax and another environment –  Metamorphic DSL

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

In 2010, we have developed theoretical foundations, efficient algorithms for a certain number of operations (merge, aggregate, slice, refactor, synthesis, diff, etc.) over feature models (see Acher PhD thesis) Optional Mandatory Xor-Group Or-Group Organization of « features » (configuration options) into a hierarchy Specification of what combinations of features (configurations) are allowed

Slide 28

Slide 28 text

FAMILIAR (Acher et al. 2013) •  In 2010, we have developed theoretical foundations, efficient algorithms for a certain number of operations (merge, aggregate, slice, refactor, synthesis, diff, etc.) over feature models •  One question that came across: how to provide the support (language + environment) so that people (partners, researchers, students, us) can use our operations?

Slide 29

Slide 29 text

shape #0 Java (SPLAR API)

Slide 30

Slide 30 text

#0 •  Too much verbose and technical, right? •  Don’t get it wrong: SPLAR is a great API (certainly the most popular) but… –  the audience (i.e., researchers in charge of developping/ benchmarking efficient algorithms) –  the design goal (i.e., extensible mechanisms to adapt reasoning mechanisms) does not fit our purpose •  We simply wanted to offer a way to merge two feature models and then, say, count the number of configuration and configure it –  Three lines right?

Slide 31

Slide 31 text

OK 6 lines ;-)

Slide 32

Slide 32 text

#1 •  Problem resolved? •  External nature of the DSL poses two kinds of problems – Integration of FAMILIAR to other applications – Limits of its expressiveness •  We have added foreach-like loop, if-then-else, and even ways to define reusable scripts… Also facilities for manipulating strings (like concatenation) •  At some points, we have even doubted that FAMILIAR was a DSL (~a kind of restricted GPL)

Slide 33

Slide 33 text

#2 •  We shift to a fluent Java API – easier to integrate to other Java-based tools – foreach / if then else / string manipulation are already there

Slide 34

Slide 34 text

#3 •  Suboptimal notation (e.g., still verbose) •  REPL

Slide 35

Slide 35 text

What the hell is going on? •  3 attempts. THREE. •  But no superior / one-size-fits-all solution – We still like the external solution: (1) concise notation and the dedicated support ; (2) when communicating with other researchers, students, or partners – We still like the Java/Scala solution when we want to build variability-intensive, integrated applications

Slide 36

Slide 36 text

WebFML We still like the external solution (1) concise notation and dedicated support ; (2) when communicating with other researchers, students, or partners

Slide 37

Slide 37 text

WebFML Effective communication Understandability Learnability

Slide 38

Slide 38 text

We still like the Java/Scala solution when we want to build variability-intensive, integrated applications Expressiveness Integration to other systems Tuning of internal details (solvers)

Slide 39

Slide 39 text

What the hell is going on? •  3 attempts. THREE. •  But no superior / one-size-fits-all solution •  Are we all wrong? – No! Keeping all variants does make sense, with pros and cons depending on the « context » •  Metamorphic DSL – We would like to transition from one shape to the other

Slide 40

Slide 40 text

shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) FAMILIAR (external DSL) shape #0 Java (SPLAR API)

Slide 41

Slide 41 text

Same kind of story with SQL •  Do you want to learn SQL with Java? •  Can you rule the world with only SQL? shape #3 shape #2 shape #1 Scala (internal DSL) Java (internal DSL) Plain SQL (external DSL)

Slide 42

Slide 42 text

Metamorphic DSL IDE 1 a DSL Speciﬁcation a DSL Articfact IDE 2

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Scenario #1 •  Help people to quickly learn the language and transition to another shape when needs be – NB: can be the same person!

Slide 45

Slide 45 text

Scenario #2 •  People can use the more advanced support for understanding / debugging feature models – NB: can be the same person!

Slide 46

Slide 46 text

Scenario #(1+2) •  People can use the more advanced support for understanding / debugging feature models – And back again!

Slide 47

Slide 47 text

Scenario #1 •  Help people to quickly learn the language and transition to another shape when needs be – NB: can be the same person!

Slide 48

Slide 48 text

Scenario #2 •  People can use the more advanced support for understanding / debugging SQL queries – NB: can be the same person!

Slide 49

Slide 49 text

Scenario #(1+2) •  People can use the more advanced support for understanding / debugging SQL queries – And back again!

Slide 50

Slide 50 text

Scenario #(Putting All Together) Say a company wants to develop a web configurator for assisting customers in the selection of products

Slide 51

Slide 51 text

product manager mysql>'SELECT&*&FROM&products; +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |'productID'|'productCode'|'name''''''|'quantity'|'gps''''''''| +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |''''''1001'|'PEN'''''''''|'Car'Red'''|'''''5000'|'''''''true'| |''''''1002'|'PEN'''''''''|'Car'Blue''|'''''8000'|'''''''true'| |''''''1003'|'PEN'''''''''|'Car'Black'|'''''2000'|'''''''true'| |''''''1004'|'PEC'''''''''|'Car'2B''''|''''10000'|'''''''true'| |''''''1005'|'PEC'''''''''|'Car'2H''''|'''''8000'|'''''''false| |''''''1006'|'PEC'''''''''|'Car'HB''''|''''''''0'|'''''''false|'' +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ 6'rows'in'set'(0.02'sec)

Slide 52

Slide 52 text

We also have an Eclipse version marketing engineer

Slide 53

Slide 53 text

software engineer (Java API)

Slide 54

Slide 54 text

software engineer (Java API) marketing engineer product manager

Slide 55

Slide 55 text

Metamorphic DSLs FAMILIAR deﬁnition a SQL query a FAMILIAR script <> <> <> SQL deﬁnition mysql>'SELECT&*&FROM&products; +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |'productID'|'productCode'|'name''''''|'quantity'|'gps''''''''| +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ |''''''1001'|'PEN'''''''''|'Car'Red'''|'''''5000'|'''''''true'| |''''''1002'|'PEN'''''''''|'Car'Blue''|'''''8000'|'''''''true'| |''''''1003'|'PEN'''''''''|'Car'Black'|'''''2000'|'''''''true'| |''''''1004'|'PEC'''''''''|'Car'2B''''|''''10000'|'''''''true'| |''''''1005'|'PEC'''''''''|'Car'2H''''|'''''8000'|'''''''false| |''''''1006'|'PEC'''''''''|'Car'HB''''|''''''''0'|'''''''false|'' +)))))))))))+)))))))))))))+)))))))))))+))))))))))+))))))))))))+ 6'rows'in'set'(0.02'sec) product manager marketing engineer software engineer

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

Research Directions •  What and where are metamorphic DSLs? •  Empirical studies for investigating the role of syntax and environment •  Solution for building metamorphic DSL •  Solution in the large: does metamorphic DSL pay off?

Slide 58

Slide 58 text

#1 Research Direction (metamorphic classification) •  What are metamorphic DSLs? – Precise definition/scope is missing •  Where are metamorphic DSLs? – Empirical observation and inventory ongoing •  Are all DSLs metamorphic?

Slide 59

Slide 59 text

•  H0: Syntax and environment matter –  H1: suboptimal notation can decrease understandibility –  H2: non comprehensive environment support can drecrease usability or productivity –  … –  Hn: … •  But empirical evidence is missing –  e.g., JOOQ for new learners <<<<< plain SQL for new learners? –  e.g., Impact of the absence of profiling/benchmark tools #2 Research Direction (empirical studies)

Slide 60

Slide 60 text

•  Solution for building metamorphic DSL –  Bi-directional transformations for moving from one shape to another (and back again) –  Engineering a new shape of a DSL (when the shape is missing) –  Reuse, « core » •  HCI issues –  Copy and paste? –  Open with? –  Features of projectional editors? –  Self-mechanism? #3 Research Direction (solution)

Slide 61

Slide 61 text

•  Engineering metamorphic DSLs – Does it pay off for users and developers? •  From the users’ perspective « switching » can have a cognitive cost •  From the developers’ perspective it requires effort (hopefully little!) •  Metamorphic scenarios – Communication/socio-technical issues •  Case studies, controlled experiments #4 Research Direction (solution in the large)

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

•  Languages with specialized notation and services exist because a one size-fits-all-solution is unlikely = DSLs •  Follow up argument: a one-size-fits-all solution is unlikely even for a given DSL (domain/class of problem) –  Different shapes of a DSL –  How to transition from one shape to another? •  Both users, developers, and researchers want to shape up DSLs •  All based on practical experience (FAMILIAR) and observation (e.g., literature, SQL) Where the idea of Metamorphic DSL comes from?

Slide 64

Slide 64 text

Metamorphic DSL •  Optimistic view of DSL diversity – We should embrace, promote, and support diversity in DSLs ; users are diverse, have different requirements, and are using DSLs in numerous different contexts •  DSLs should be self-adaptable to the most appropriate shape (including the corresponding IDE), according to a particular usage or task

Slide 65

Slide 65 text

Metamorphic DSL •  Optimistic view of DSL diversity •  DSLs should be self-adaptable to the most appropriate shape (including the corresponding IDE), according to a particular usage or task •  A great future for DSL; additional research effort is needed – to further understand the concept of metamorphic DSL (what/where are they? why/when some shapes are more adequate?) – to provide effective solution for transitioning from one shape to another

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

HTML Domain: web (markup) 68

Slide 69

Slide 69 text

https://github.com/julienrf/glitter TCS Wyvern (Omar et al., OOPLSA’14) Scala

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

https://github.com/inukshuk/bibtex-ruby

Slide 72

Slide 72 text

CSS Domain: web (styling) 72

Slide 73

Slide 73 text

Makefile Domain: software building 73

Slide 74

Slide 74 text

Lighthttpd configuration file Domain: web server (configuration) 74

Slide 75

Slide 75 text

Graphviz Domain: graph (drawing) 75

Slide 76

Slide 76 text

http://turingatemyhamster.co.uk/blog/?p=157 http://drdozer.github.io/graphviz-s/ https://github.com/glejeune/Ruby-Graphviz/

Slide 77

Slide 77 text

Regular expression Domain: strings (pattern matching) 77

Slide 78

Slide 78 text

Portable Game Notation (PGN)

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

PGN and DSL promises Easy to read, write, exchange, process It is all about games information and moves (nothing more) Success (chess players/tools): 8 millions games

Slide 82

Slide 82 text

« Alternatives » to PGN (other DSLs) •  For handling chess variants – e.g., Chess960 •  Proprietary extensions •  For recording a particular game position – Forsyth–Edwards Notation (FEN) rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/ RNBQKBNR b KQkq e3 0 1

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

I have a question Is it a domain-specific language!??! https://github.com/capicue/pgn

Slide 85

Slide 85 text

Is it a DSL? YES, yes, yes !!! Just another « shape » of PGN

Slide 86

Slide 86 text

FEN and PGN in Ruby

Slide 87

Slide 87 text

PGN in Python? https://github.com/renatopp/pgnparser

Slide 88

Slide 88 text

PGN in ????

Slide 89

Slide 89 text

Internal Shapes External Shapes

Slide 90

Slide 90 text

External Shapes (DSLs) Internal Shapes (DSLs)

Slide 91

Slide 91 text

Metamorphic DSL

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

External DSLs vs Internal DSLs •  An external DSL is a completely separate language and has its own custom syntax/ tooling support (e.g., editor) •  An internal DSL is more or less a set of APIs written on top of a host language (e.g., Java). – Fluent interfaces 93

Slide 94

Slide 94 text

External vs Internal DSL (SQL example) 94

Slide 95

Slide 95 text

DSL (LINQ/C# example) 95

Slide 96

Slide 96 text

Internal DSL •  « Using a host language (e.g., Java) to give the host language the feel of a particular language. » •  Fluent Interfaces –  « The more the use of the API has that language like flow, the more fluent it is » 96

Slide 97

Slide 97 text

SQL in… Java DSL in GPL 97

Slide 98

Slide 98 text

Regular expression in… Java DSL in GPL 98

Slide 99

Slide 99 text

Internal DSLs vs External DSL •  Both internal and external DSLs have strengths and weaknesses (Fowler) –  learning curve, –  cost of building, –  programmer familiarity, –  communication with domain experts, –  mixing in the host language, –  strong expressiveness boundary •  Focus of the course – external DSL a completely separate language with its own custom syntax and tooling support (e.g., editor) 99