Scala: State of the Union

In this keynote, Kevin Webber, Enterprise Architect at Lightbend, will discuss the past, present, and future of Scala; where it has been and where it’s going. Yes, this talk will focus on the new language features introduced in 2.12, but also dive into an equally important aspect of Scala in 2016: the incredible community that’s grown up around Scala over the years. Kevin will frame the talk around his own journey to Scala and how both the language and the community made him a better developer and architect.

Transcript

1. Scala State of the Union Kevin Webber Enterprise Architect @ Lightbend e: kevin.webber@lightbend.com m: medium.com/@kvnwbbr t: @kvnwbbr
2. A bit about me » Enterprise Architect at Lightbend
3. A bit about me » Enterprise Architect at Lightbend » Started exploring Scala in 2008
4. A bit about me » Enterprise Architect at Lightbend » Started exploring Scala in 2008 » Former Lightbend customer (2012-2014)
5. Agenda » Where are we today? » Spoiler: 2.11.8
6. Agenda » Where are we today? » Spoiler: 2.11.8 » Where are we going? » 2.12, 2.13, and beyond (aka, dotty)
7. Agenda » Where are we today? » Spoiler: 2.11.8 » Where are we going? » 2.12, 2.13, and beyond (aka, dotty) » Where is the industry focused? » Scala is not an island » What have I learned over five years?
8. Where are we today?
10. Scala is active! » Significant contributions by » open-source community » partners (EPFL, Lightbend, Scala Center) » Releases average 2x per year
11. Scala repo stats
15. Companies driving growth
16. Companies driving growth? » Apple → jobs! » Coursera » IBM » Intel » Mesosphere
17. Companies driving growth? » Apple » Coursera → learning » IBM » Intel » Mesosphere
18. Companies driving growth? » Apple » Coursera » IBM → Big Data University » Intel » Mesosphere
19. Companies driving growth? » Apple » Coursera » IBM » Intel → Gearpump » Mesosphere
20. Companies driving growth? » Apple » Coursera » IBM » Intel → Gearpump » Mesosphere
21. Companies driving growth? » Apple » Coursera » IBM » Intel » Mesosphere → DC/OS
22. Business drivers? » Data is ruling the world » Metal is abstracted » Functional programming rules this space » Object-oriented programming is a familiar path
23. Scala projects driving growth
24. » Kafka (Confluent) » Distributed, publish-subscribe messaging system
26. » Kafka (Confluent) » Distributed, publish-subscribe messaging system » Spark (Apache) » General-purpose cluster computing
29. » Kafka (Confluent) » Distributed, publish-subscribe messaging system » Spark (Apache) » General-purpose cluster computing » Akka (Lightbend) » Distributed messaging toolkit
30. Akka » The actor model abstraction provides: » distribution » location transparency » isolation
31. Akka Streams » Higher-level abstraction on top of Akka » Per-event processing of streams » Back-pressure enabled by default
32. Why Scala? Example "one-liner" to implement per-event stream processing with Akka Streams and Kafka implicit val actorSystem = ActorSystem("ReactiveKafka") implicit val materializer = ActorMaterializer() val kafka = new ReactiveKafka(host = "localhost:9092", zooKeeperHost = "localhost:2181") val publisher = kafka.consume("lowercaseStrings", "groupName", new StringDecoder()) val subscriber = kafka.publish("uppercaseStrings", "groupName", new StringEncoder()) // consume lowercase strings from kafka and publish them transformed to uppercase Source(publisher).map(_.toUpperCase).to(Sink(subscriber)).run()
34. val g = RunnableGraph.fromGraph(GraphDSL.create() { implicit builder => import GraphDSL.Implicits._ // Source val A: Outlet[String] = builder.add(Source.fromIterator(() => flightDelayLines)).out // Flows val B: FlowShape[String, FlightEvent] = builder.add(csvToFlightEvent) val C: FlowShape[FlightEvent, FlightDelayRecord] = builder.add(filterAndConvert) val D: UniformFanOutShape[FlightDelayRecord, FlightDelayRecord] = builder.add(Broadcast[FlightDelayRecord](2)) val F: FlowShape[FlightDelayRecord, (String, Int, Int)] = builder.add(averageCarrierDelay) // Sinks val E: Inlet[Any] = builder.add(Sink.ignore).in val G: Inlet[Any] = builder.add(Sink.foreach(averageSink)).in // Graph A ~> B ~> C ~> D E <~ D G <~ F <~ D ClosedShape }) g.run()
35. Fast Data Architecture » Akka Streams » < 1 second latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
36. Big picture » An ecosystem is growing around Scala for: » Cloud computing » Stream processing » Fast data, big data, fast big data » Microservices » Scala is a natural fit in these spaces
37. Notable libraries » Argonaut → purely functional JSON » Cats → foundation for pure, typeful libraries » ScalaCheck → property-based testing » ScalaTest → test Scala, Java, and Scala.js » scalaz → purely functional data structures » Shapeless → generic programming library » Specs → executable software specifications
38. Scala Educational Opportunities
39. Educational opportunities » Carnegie-Mellon University » EPFL » Harvard University » Stanford University » University of Texas - Austin » University of Washington » West Point
41. Scala Center » Open source foundation for Scala
42. Scala Center » Open source foundation for Scala » Engage with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize
43. Scala Center » Open source foundation for Scala » Engage with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize » Help developers learn the language
44. Scala Language
45. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ, SAM, default) » New backend & optimizer » Q2 2016
46. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ, SAM, default) » New backend & optimizer » Q2 2016 » 2.13 ("modularization") » Focused on libraries, collections, and Spark
47. Scala.js 0.6.9 » Bringing Scala even closer to the JavaScript community » Current version is mostly about Scala 2.12 support and faster compilation » Closer and closer to production-grade
48. Scala Native » Low-level primitives » Pointers, structs, etc » Interop with C » extern objects, no runtime overhead » Compiled ahead-of-time via LLVM » Instant startup vs JIT
49. Scala in five years
50. dotty » Currently a platform to try out new language concepts and compiler technologies
51. dotty » Currently a platform to try out new language concepts and compiler technologies » Is the next Scala
52. dotty » Currently a platform to try out new language concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification
53. dotty » Currently a platform to try out new language concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification » Add new features & stay approachable by dropping other features » macros, general type projection, early initializers, existential types, etc
54. dotty features
55. Five years of Scala (A few things I've picked up)
56. 1. Always be learning » I dove into Scala because of this book:
57. Async boundary
58. Async boundary
59. Synchronous
60. Asynchronous
61. 2. You don't have to know everything ... » That time I had a job interview with Martin
62. ... but it can't hurt to try! » Buy this book and master: » Higher-order functions, currying, partial functions, functional data structures, handling errors without exceptions, strictness and laziness, purely functional state ...
63. 3. Learn to explain the value functional, immutable, asynchronous, non-blocking, real-time, highly-available, loosely coupled, scalable, fault-tolerant, concurrent, low latency, high throughput... How do you map the properties of Scala and functional programming to business value?
64. 4. Encourage, teach, mentor » We need developers of all experience levels, academic backgrounds, and professional backgrounds
65. Thank you! Kevin Webber, Enterprise Architect, Lightbend Email: kevin.webber@lightbend.com Medium: medium.com/@kvnwbbr Twitter: @kvnwbbr