Moe Status Update

This is a talk given at YAPC::NA 2013 in Austin Texas and is an update on the my Moe project.

Transcript

1. moe Stevan Little • YAPC::NA 2013 • Austin,TX Tuesday • 4 June 2013
2. So I announced the Moe project at this years Orlando Perl Workshop. For those of you who are not familiar with the project, my goal is to experiment with the idea of a new runtime for what I am calling an “Ultra Modern” Perl 5. So what does that mean exactly ...
3. My Moe Wishlist • A more consistent syntax and sane grammar • with a real AST • and better tooling support • A less insane runtime • proper MOP (everything is an object) • slimmer core • easy (non-XS) extension mechanism • On a modern VM platform • JVM / CLR / LLVM / V8 • cross language sharing So I created a simple wishlist of what I wanted the end game to look like (more of less).
4. And so, after the announcement, a number of very smart people offered me a lot of really good advice. I listened to every bit of it and actually spent maybe the first two weeks or so just thinking about things, talking to people about things and generally just mulling things over and over (and over and over) in my head. And in the end I came to a decision ...
5. STFU and code ... I just need to start coding, too much time thinking was hurting my head. I had a decent idea of what I wanted but I really just had to start writing code and feel my way through it. But I did know a couple things for sure
6. 5 first was that I wanted this to be starting from Perl 5, I wanted to remove some features and replace them with other features, execute on a vision that had stared with the p5-mop work. At it’s core I wanted the language to feel very much like Perl 5.
7. 6 I also knew I wanted to steal a bunch of ideas from Perl 6 (which I have been doing since I first wrote Moose in 2006). The reason being that Perl 6 itself was already doing exactly I just said; start with Perl 5, remove some stuff, add some stuff, tweak some stuff, etc.
8. 5½ so In a way, I guess you could call this Perl 5 and a half.
9. So I only have 20 minutes here so I am just going to dive in and start showing you Moe and what we have accomplished so far. But before I do that, I want to clarify something. Moe is what I am calling a <click> “thought experiement”. I mentioned in my talk at OPW that I wanted this to be Pugs- like, I said this not just for the -Ofun aspect, but also that one of the key goals of Pugs was to give the P6 language designers something they could actually use and try out. I think we need something like this for p5.
10. Moe is an attempt to show a way forward, not be a way forward. Moe is an attempt to show a way forward, not be a way forward. So keeping that in mind, I want to make a point here. I am not trying to port Perl to the JVM, or usurp Perl 6, I am simply trying to imagine what Perl 5 would look like it if evolved a little more, but not so much as Perl 6. I want to see if I can’t find a language middle ground that will bring some of the goodness that Perl 6 has, but still feel comfortable in the hands of a Perl 5 programmer.
11. Literals Str “Double” and ‘single’ Int 10 Num 3.14 Bool true && false Undef undef Pair foo => “bar” Hash { one => 1 } Array [ 1, 2, 3 ] Code -> ($x) { $x + 1 } Please notice that I am using the p5 “reference” syntax for arrays and hashes. For code we are actually borrowing the p6 pointy sub syntax, though we will likely also support the p5 “sub” style too. As I said, these are now all objects, which means there is no difference between a value and a reference, they are all references because they are all objects. Naturally, each of these is backed by a class which implements the methods that can be performed on each type of object. Additionally, there are no more implicit conversions anymore, they are explict (as we will see shortly). The only one exception is between Int and Num, they will usually just do the right thing.
12. Operators Logical && || ?: Comparison < > <= >= <=> lt gt le ge cmp Math + - / * % ** String ~ Methods . Coercion ? ~ + RegExpr =~ Range .. (get to regexpr operator and) I want to quickly mention that a lot of this work would not be possible without the contributions of Prakash Kailasa, he has consistently been on of the projects biggest contributors and has really helped pull a lot of features together. All the operators are also methods using the Perl 6 style naming convention. We don’t yet support defining your own operators, or overriding them, but those are both in the plans.
13. Operators Logical && || ?: Comparison < > <= >= <=> lt gt le ge cmp Math + - / * % ** String ~ Methods . Coercion ? ~ + RegExpr =~ Range .. infix:<&&> prefix:<?> postfix:<++> (get to regexpr operator and) I want to quickly mention that a lot of this work would not be possible without the contributions of Prakash Kailasa, he has consistently been on of the projects biggest contributors and has really helped pull a lot of features together. All the operators are also methods using the Perl 6 style naming convention. We don’t yet support defining your own operators, or overriding them, but those are both in the plans.
14. Statements if / elsif / else unless / elsif / else for / foreach while / until try / catch / finally match / case given / when We have support for if/elsif/else, unless/elsif/else, for/foreach (the Perl 5 style) and we have while/until. Additionally all of these are available as statement modifiers as well as normal statements. We currently parse try/catch/finally, but it is right now a noop. This is partially because I wanted to figure out the exception objects more before I down too heavily into this, and partially because I kind of want a nice way to catch exceptions, either with a match/case style statement or with a given/when style statement. Obviously both these add some complexity and new territory (match/case with the desconstructing bind and given/when with smartmatching)
15. Built ins say(*@msg) print(*@msg) warn(*@msg) caller($index?) sleep($seconds) system(*@cmd) eval($code) not($value) chr($number) ord($character) rand($seed?) die(*@msg) exit($status?) These are what we have now, they are all subroutines stored in the CORE package, which is sort of the “package of last resort” when looking up a subroutine. It should be noted that they have proper signatures which we check, and we will talk about that shortly ...
16. Built ins @ARGV %ENV @INC %INC $! $?RUNTIME $?PACKAGE $?CLASS $?SELF &?ROUTINE We also have some of the built in variables as well. We are going with the $! exception object and removed $@ entirely (another p6-ism). Yes, this means that all exceptions (even those thrown by die) will be proper exception objects. We also provide access to a number of special variables, and we’ve opted for the Perl 6 twigil format here instead of the __DOUBLE_UNDERSCORE__ version of Perl 5 (mostly cause they are really variables and not just funky barewords).
17. Built Ins push pop shift unshift map grep join sort splice keys values delete each exists open close eof -X unlink readline chop chomp uc lc ucfirst lcfirst hex length oct split sprintf substr atan2 cos exp log sin defined And all those built in functions to operate on the various data structures (arrays, hashes, files, strings, numbers, and misc values) all these are now methods on the respective objects. Now, this doesn’t mean we might not support some kind of built in function version of them, for instance ...
18. Built Ins push pop shift unshift map grep join sort splice keys values delete each exists open close eof -X unlink readline chop chomp uc lc ucfirst lcfirst hex length oct split sprintf substr atan2 cos exp log sin defined  And all those built in functions to operate on the various data structures (arrays, hashes, files, strings, numbers, and misc values) all these are now methods on the respective objects. Now, this doesn’t mean we might not support some kind of built in function version of them, for instance ...
19. Built Ins say “Hello World”; “Hello World”.say; say is defined as a function in the core package, but it is also defined as a method in the `Any` class (which I will explain shortly ... ) so you can do it whatever way suits you. So this means it is possible that we might do the same with all those other builtins. Of course, something missing from even that list was all the networking and *NIX functions, those would need to be written as a separate module, which is in line with Jesse Vincent’s original vision in his “Perl 5.16 and beyond” talk.
20. sub sum (@x, $acc = 0) { @x.length == 0 ? $acc : sum( @x.tail, @x.head + $acc ); } Subroutines Notice that we are not flattening the argument list, mostly cause we have proper signatures and because the @x is an object reference and not some magical flattening list. Additionally you can see that we are just calling methods on the @x array object.
21. Signatures Positional $x $x = 10 Optional $x? Slurpy *@x Named :$x NamedSlurpy *%x So lets dive into the signatures, the obvious ones are the positional ones, and as we saw in the last slides, <click> those can have a default value. <click> We also have optional parameters, which could be thought of as just a sugared version of default where the default value is undef. <click> Next we have slurpy params, which will gather all params after this and put them into an array object. <click> Next is named parameters which actually expect a Pair object whose key is the name of this variable minus the sigil. <click> and lastly there is Named Slurpy which really just slurps up the remaining params, just as the Slurpy does, but it expects them to be Pair objects.
22. Subroutines my &fib = -> ($n) { if ($n < 2) { $n } else { (&?ROUTINE.call($n - 1)) + (&?ROUTINE.call($n - 2)) } }; We also have anon-subroutines using the Perl 6 “pointy” sub. And again from Perl 6 (although we recently got this added to Perl 5 as well) the &? ROUTINE variable, which is the current executing subroutine. This is of course really helpful for avoiding circular references in your closures, not that it won’t let you still shoot yourself in the foot if you want.
23. Subroutines sub double ($x) { $x * 2 } my @y = @x.map( &double ); it is also possible to take a subroutine reference from any named subroutine, simply by using the name with the & sigil prefix. Assuming there is no &double variable already this will just create that reference for you so that you can pass it into a method like map.
24. Subroutines sub ok ($test, $msg = “”) is export { [ “ok“, $count, $msg ].join (“ ”); } Subroutines also have (very rudimentary) support for traits. The only one currently supported is `export`, which (as you might guess) tells Moe that this subroutine should be exported from it’s enclosing package when it is used.
25. package Test::More-0.01-cpan:MOE { sub ok ($test, $msg = “”) is export { [ “ok“, $count, $msg ].join (“ ”); } sub not_ok ($test, $msg = “”) is export { [ “not ok“, $count, $msg ].join (“ ”); } } Packages Currently the only form of package is the block package. Packages also have an associated version and authority, although there is currently no way to access them via the runtime (on the TODO list). What packages do have are a local lexical environment (for any ‘my’ variables declared in the package body), they do not however have support for ‘our’ variables (meaning internal variables that can be accessed outside of the package). Packages can also contain subroutines as you can see here. And packages can also contain sub packages, in fact, this package actually internally expands into ...
26. package Test { package More-0.01-cpan:MOE { sub ok ($test, $msg = “”) is export { [ “ok“, $count, $msg ].join (“ ”); } sub not_ok ($test, $msg = “”) is export { [ “not ok“, $count, $msg ].join (“ ”); } } } Packages ... this. As you can see the version and authority information is only attached to the More package. Packages can also contain classes, this is because packages are *not* classes anymore, instead we have a full object system in Moe, which brings me to ...
27. is subclass of is instance of instance class meta Object Foo $foo Class Classes First thing I want to point out is that the object system is properly bootstrapped. This means that Class is an instance of Class and Class is a subclass of Object, which itself is also an instance of Class. This also means ...
28. Classes class Foo { ... } say Foo.name; # prints Foo my $foo_class = Foo; my $foo_object = $foo_class.new; sub foo_class { Foo } my $foo_object = foo_class().new; ... that classes are first class things as well, you can call methods on them, you can store them in variables, return them from subroutines, etc.
29. Classes Object Class Any Undef Scalar Bool Str Int Num Exception Regex Array Hash Pair IO Code There is also an object hierarchy, which is based a lot on the Perl 6 hierarchy, but with a few omissions since we are aiming for a simpler language. I will note that there is no type checking done against these (beyond the checks implied by virtual method dispatch, which of course provides no real type safety). However we do type check the containers, so only scalars can be in variables with the $ sigil, arrays in variables with the @ sigil, % for hash and & for code.
30. Classes class Point { has $!x = 0; has $!y = 0; method x ($x?) { $!x = $x if $x.defined; $!x } method y ($y?) { $!y = $y if $y.defined; $!y } method clear { ($!x, $!y) = (0, 0); } } So here is your basic class, it has attributes and methods. Again we are borrowing from Perl 6 and using the twigils for attributes. Currently only the ! (private) version is supported, but eventually we will support the public version (and the autogenerated accessors that go with it, which will certainly mean that we will also support traits like `is ro` and `is rw` as well, but thats for later).
31. Classes class Point3D extends Point { has $!z = 0; method z ($z?) { $!z = $z if $z.defined; $!z } method clear { super; $!z = 0; } } We also support subclassing, again, borrowed from Perl 6 with the `extends` keyword. Additionally you can call the superclass method with just the `super` keyword (which is more like the p5-mop then p6). As you might have guess here, the object model is only single inheritance. Support for roles is planned, but not done yet.
32. Classes package Geometry { class Point { ... } class Point3d extends Point { ... } } my $p = Geometry::Point.new; ... and as I mentioned before classes can be put in packages, here is an example of that. They are addressed from outside the package with the :: qualifier, but internally they are called by their local name. You can think of classes as being very similar to subroutines in this situation. And given that, we might add support the export trait on classes as well, but not high on the list yet.
33. “foo” =~ m/^f/; “foo”.match(/^f/); “foo” =~ s/o/a/g; “foo”.subst(/o/, “a”); The Rest So those are kind of the core things, there are another few things I want to just show. First is the (very rudimentary) regexpr support. Now, this is by no means Perl regexpr, in fact this just forwards things to the Java regex engine at this point, but it is a base from which it start at least.
34. [ 1, 2, 3 ].map(-> { $_ + 1 }); { one => 1 }.keys.length; &adder.(2, 2); &adder.call(2, 2); &adder.apply([2, 2]); The Rest I know we saw earlier that it was possible to call methods on array and hash object, but I wanted to show that it can be done on the raw literals as well. Additionally I wanted to show how you can call an anon-subroutine, first with just the dot, the will the `call` method and finally you can use `apply` in which you are expected to pass an array for the arguments yourself. If you’ve done any funky Javascript stuff, this will be familiar to you.
35. my ($x, $y, $z) = (1, 2, 3); my @a = 1 .. 10; my @a = [ 1 .. 10 ]; # AoA my %x = @a; # BOOM! The Rest And then lastly, I just want to show a few other random features. The multiple assignment operator, the range operator (which produces a proper array, so you don’t need to wrap in square brackets, unless you want an array of an array that is. And lastly I just wanted to show an example of the sigil type checking mechanism. If you were to try to do this, you would get an exception (currently a runtime one, but eventually we plan on making this a compile time check).
36. Test Suite LIVE DEMO So, one of the things that Liz mentioned in her earlier talk was how the Pugs Perl 6 test suite is something that is still in use today and how that ultimately was a valuable by-product of that project. This again is something I hope that Moe can do, even if it suffers the same fate as Pugs did. <DANGER WILL ROBINSON! DANGER! LIVE DEMO TIME>
37. ... demo ....
38. • expose MOP to runtime • test, test, test! • implement Roles • implement try/catch & match/case • wrap Scala Actors & Java Threads What’s Next
39. Questions?