Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scala: State of the Union

Scala: State of the Union

In this keynote, Kevin Webber, Enterprise Architect at Lightbend, will discuss the past, present, and future of Scala; where it has been and where it’s going. Yes, this talk will focus on the new language features introduced in 2.12, but also dive into an equally important aspect of Scala in 2016: the incredible community that’s grown up around Scala over the years. Kevin will frame the talk around his own journey to Scala and how both the language and the community made him a better developer and architect.

665a7ca82af87606f4fc83b3d94b5fd5?s=128

Kevin Webber

August 05, 2016
Tweet

More Decks by Kevin Webber

Other Decks in Technology

Transcript

  1. Scala State of the Union Kevin Webber Enterprise Architect @

    Lightbend e: kevin.webber@lightbend.com m: medium.com/@kvnwbbr t: @kvnwbbr
  2. A bit about me » Enterprise Architect at Lightbend

  3. A bit about me » Enterprise Architect at Lightbend »

    Started exploring Scala in 2008
  4. A bit about me » Enterprise Architect at Lightbend »

    Started exploring Scala in 2008 » Former Lightbend customer (2012-2014)
  5. Agenda » Where are we today? » Spoiler: 2.11.8

  6. Agenda » Where are we today? » Spoiler: 2.11.8 »

    Where are we going? » 2.12, 2.13, and beyond (aka, dotty)
  7. Agenda » Where are we today? » Spoiler: 2.11.8 »

    Where are we going? » 2.12, 2.13, and beyond (aka, dotty) » Where is the industry focused? » Scala is not an island » What have I learned over five years?
  8. Where are we today?

  9. None
  10. Scala is active! » Significant contributions by » open-source community

    » partners (EPFL, Lightbend, Scala Center) » Releases average 2x per year
  11. Scala repo stats

  12. None
  13. None
  14. None
  15. Companies driving growth

  16. Companies driving growth? » Apple → jobs! » Coursera »

    IBM » Intel » Mesosphere
  17. Companies driving growth? » Apple » Coursera → learning »

    IBM » Intel » Mesosphere
  18. Companies driving growth? » Apple » Coursera » IBM →

    Big Data University » Intel » Mesosphere
  19. Companies driving growth? » Apple » Coursera » IBM »

    Intel → Gearpump » Mesosphere
  20. Companies driving growth? » Apple » Coursera » IBM »

    Intel → Gearpump » Mesosphere
  21. Companies driving growth? » Apple » Coursera » IBM »

    Intel » Mesosphere → DC/OS
  22. Business drivers? » Data is ruling the world » Metal

    is abstracted » Functional programming rules this space » Object-oriented programming is a familiar path
  23. Scala projects driving growth

  24. » Kafka (Confluent) » Distributed, publish-subscribe messaging system

  25. None
  26. » Kafka (Confluent) » Distributed, publish-subscribe messaging system » Spark

    (Apache) » General-purpose cluster computing
  27. None
  28. None
  29. » Kafka (Confluent) » Distributed, publish-subscribe messaging system » Spark

    (Apache) » General-purpose cluster computing » Akka (Lightbend) » Distributed messaging toolkit
  30. Akka » The actor model abstraction provides: » distribution »

    location transparency » isolation
  31. Akka Streams » Higher-level abstraction on top of Akka »

    Per-event processing of streams » Back-pressure enabled by default
  32. Why Scala? Example "one-liner" to implement per-event stream processing with

    Akka Streams and Kafka implicit val actorSystem = ActorSystem("ReactiveKafka") implicit val materializer = ActorMaterializer() val kafka = new ReactiveKafka(host = "localhost:9092", zooKeeperHost = "localhost:2181") val publisher = kafka.consume("lowercaseStrings", "groupName", new StringDecoder()) val subscriber = kafka.publish("uppercaseStrings", "groupName", new StringEncoder()) // consume lowercase strings from kafka and publish them transformed to uppercase Source(publisher).map(_.toUpperCase).to(Sink(subscriber)).run()
  33. None
  34. val g = RunnableGraph.fromGraph(GraphDSL.create() { implicit builder => import GraphDSL.Implicits._

    // Source val A: Outlet[String] = builder.add(Source.fromIterator(() => flightDelayLines)).out // Flows val B: FlowShape[String, FlightEvent] = builder.add(csvToFlightEvent) val C: FlowShape[FlightEvent, FlightDelayRecord] = builder.add(filterAndConvert) val D: UniformFanOutShape[FlightDelayRecord, FlightDelayRecord] = builder.add(Broadcast[FlightDelayRecord](2)) val F: FlowShape[FlightDelayRecord, (String, Int, Int)] = builder.add(averageCarrierDelay) // Sinks val E: Inlet[Any] = builder.add(Sink.ignore).in val G: Inlet[Any] = builder.add(Sink.foreach(averageSink)).in // Graph A ~> B ~> C ~> D E <~ D G <~ F <~ D ClosedShape }) g.run()
  35. Fast Data Architecture » Akka Streams » < 1 second

    latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
  36. Big picture » An ecosystem is growing around Scala for:

    » Cloud computing » Stream processing » Fast data, big data, fast big data » Microservices » Scala is a natural fit in these spaces
  37. Notable libraries » Argonaut → purely functional JSON » Cats

    → foundation for pure, typeful libraries » ScalaCheck → property-based testing » ScalaTest → test Scala, Java, and Scala.js » scalaz → purely functional data structures » Shapeless → generic programming library » Specs → executable software specifications
  38. Scala Educational Opportunities

  39. Educational opportunities » Carnegie-Mellon University » EPFL » Harvard University

    » Stanford University » University of Texas - Austin » University of Washington » West Point
  40. None
  41. Scala Center » Open source foundation for Scala

  42. Scala Center » Open source foundation for Scala » Engage

    with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize
  43. Scala Center » Open source foundation for Scala » Engage

    with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize » Help developers learn the language
  44. Scala Language

  45. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ,

    SAM, default) » New backend & optimizer » Q2 2016
  46. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ,

    SAM, default) » New backend & optimizer » Q2 2016 » 2.13 ("modularization") » Focused on libraries, collections, and Spark
  47. Scala.js 0.6.9 » Bringing Scala even closer to the JavaScript

    community » Current version is mostly about Scala 2.12 support and faster compilation » Closer and closer to production-grade
  48. Scala Native » Low-level primitives » Pointers, structs, etc »

    Interop with C » extern objects, no runtime overhead » Compiled ahead-of-time via LLVM » Instant startup vs JIT
  49. Scala in five years

  50. dotty » Currently a platform to try out new language

    concepts and compiler technologies
  51. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala
  52. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification
  53. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification » Add new features & stay approachable by dropping other features » macros, general type projection, early initializers, existential types, etc
  54. dotty features

  55. Five years of Scala (A few things I've picked up)

  56. 1. Always be learning » I dove into Scala because

    of this book:
  57. Async boundary

  58. Async boundary

  59. Synchronous

  60. Asynchronous

  61. 2. You don't have to know everything ... » That

    time I had a job interview with Martin
  62. ... but it can't hurt to try! » Buy this

    book and master: » Higher-order functions, currying, partial functions, functional data structures, handling errors without exceptions, strictness and laziness, purely functional state ...
  63. 3. Learn to explain the value functional, immutable, asynchronous, non-blocking,

    real-time, highly-available, loosely coupled, scalable, fault-tolerant, concurrent, low latency, high throughput... How do you map the properties of Scala and functional programming to business value?
  64. 4. Encourage, teach, mentor » We need developers of all

    experience levels, academic backgrounds, and professional backgrounds
  65. Thank you! Kevin Webber, Enterprise Architect, Lightbend Email: kevin.webber@lightbend.com Medium:

    medium.com/@kvnwbbr Twitter: @kvnwbbr