$30 off During Our Annual Pro Sale. View Details »

Scala: State of the Union

Scala: State of the Union

In this keynote, Kevin Webber, Enterprise Architect at Lightbend, will discuss the past, present, and future of Scala; where it has been and where it’s going. Yes, this talk will focus on the new language features introduced in 2.12, but also dive into an equally important aspect of Scala in 2016: the incredible community that’s grown up around Scala over the years. Kevin will frame the talk around his own journey to Scala and how both the language and the community made him a better developer and architect.

Kevin Webber

August 05, 2016
Tweet

More Decks by Kevin Webber

Other Decks in Technology

Transcript

  1. Scala
    State of the Union
    Kevin Webber
    Enterprise Architect @
    Lightbend
    e: [email protected]
    m: medium.com/@kvnwbbr
    t: @kvnwbbr

    View Slide

  2. A bit about me
    » Enterprise Architect at Lightbend

    View Slide

  3. A bit about me
    » Enterprise Architect at Lightbend
    » Started exploring Scala in 2008

    View Slide

  4. A bit about me
    » Enterprise Architect at Lightbend
    » Started exploring Scala in 2008
    » Former Lightbend customer (2012-2014)

    View Slide

  5. Agenda
    » Where are we today?
    » Spoiler: 2.11.8

    View Slide

  6. Agenda
    » Where are we today?
    » Spoiler: 2.11.8
    » Where are we going?
    » 2.12, 2.13, and beyond (aka, dotty)

    View Slide

  7. Agenda
    » Where are we today?
    » Spoiler: 2.11.8
    » Where are we going?
    » 2.12, 2.13, and beyond (aka, dotty)
    » Where is the industry focused?
    » Scala is not an island
    » What have I learned over five years?

    View Slide

  8. Where are we today?

    View Slide

  9. View Slide

  10. Scala is active!
    » Significant contributions by
    » open-source community
    » partners (EPFL, Lightbend, Scala Center)
    » Releases average 2x per year

    View Slide

  11. Scala repo stats

    View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. Companies driving
    growth

    View Slide

  16. Companies
    driving
    growth?
    » Apple → jobs!
    » Coursera
    » IBM
    » Intel
    » Mesosphere

    View Slide

  17. Companies
    driving
    growth?
    » Apple
    » Coursera → learning
    » IBM
    » Intel
    » Mesosphere

    View Slide

  18. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM → Big Data University
    » Intel
    » Mesosphere

    View Slide

  19. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel → Gearpump
    » Mesosphere

    View Slide

  20. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel → Gearpump
    » Mesosphere

    View Slide

  21. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel
    » Mesosphere → DC/OS

    View Slide

  22. Business
    drivers?
    » Data is ruling the world
    » Metal is abstracted
    » Functional programming
    rules this space
    » Object-oriented programming
    is a familiar path

    View Slide

  23. Scala projects
    driving growth

    View Slide

  24. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system

    View Slide

  25. View Slide

  26. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system
    » Spark (Apache)
    » General-purpose cluster computing

    View Slide

  27. View Slide

  28. View Slide

  29. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system
    » Spark (Apache)
    » General-purpose cluster computing
    » Akka (Lightbend)
    » Distributed messaging toolkit

    View Slide

  30. Akka
    » The actor model abstraction
    provides:
    » distribution
    » location transparency
    » isolation

    View Slide

  31. Akka Streams
    » Higher-level abstraction on
    top of Akka
    » Per-event processing of
    streams
    » Back-pressure enabled by
    default

    View Slide

  32. Why Scala?
    Example "one-liner" to implement per-event stream
    processing with Akka Streams and Kafka
    implicit val actorSystem = ActorSystem("ReactiveKafka")
    implicit val materializer = ActorMaterializer()
    val kafka = new ReactiveKafka(host = "localhost:9092", zooKeeperHost = "localhost:2181")
    val publisher = kafka.consume("lowercaseStrings", "groupName", new StringDecoder())
    val subscriber = kafka.publish("uppercaseStrings", "groupName", new StringEncoder())
    // consume lowercase strings from kafka and publish them transformed to uppercase
    Source(publisher).map(_.toUpperCase).to(Sink(subscriber)).run()

    View Slide

  33. View Slide

  34. val g = RunnableGraph.fromGraph(GraphDSL.create() {
    implicit builder =>
    import GraphDSL.Implicits._
    // Source
    val A: Outlet[String] = builder.add(Source.fromIterator(() => flightDelayLines)).out
    // Flows
    val B: FlowShape[String, FlightEvent] = builder.add(csvToFlightEvent)
    val C: FlowShape[FlightEvent, FlightDelayRecord] = builder.add(filterAndConvert)
    val D: UniformFanOutShape[FlightDelayRecord, FlightDelayRecord] = builder.add(Broadcast[FlightDelayRecord](2))
    val F: FlowShape[FlightDelayRecord, (String, Int, Int)] = builder.add(averageCarrierDelay)
    // Sinks
    val E: Inlet[Any] = builder.add(Sink.ignore).in
    val G: Inlet[Any] = builder.add(Sink.foreach(averageSink)).in
    // Graph
    A ~> B ~> C ~> D
    E <~ D
    G <~ F <~ D
    ClosedShape
    })
    g.run()

    View Slide

  35. Fast Data
    Architecture
    » Akka Streams
    » < 1 second latency per-
    event processing
    » Spark Streaming
    » 1-10+ second latency for
    mini-batch event
    processing
    » Kafka captures the complete
    event stream

    View Slide

  36. Big picture
    » An ecosystem is growing
    around Scala for:
    » Cloud computing
    » Stream processing
    » Fast data, big data,
    fast big data
    » Microservices
    » Scala is a natural fit in
    these spaces

    View Slide

  37. Notable libraries
    » Argonaut → purely functional JSON
    » Cats → foundation for pure, typeful libraries
    » ScalaCheck → property-based testing
    » ScalaTest → test Scala, Java, and Scala.js
    » scalaz → purely functional data structures
    » Shapeless → generic programming library
    » Specs → executable software specifications

    View Slide

  38. Scala Educational
    Opportunities

    View Slide

  39. Educational opportunities
    » Carnegie-Mellon University
    » EPFL
    » Harvard University
    » Stanford University
    » University of Texas - Austin
    » University of Washington
    » West Point

    View Slide

  40. View Slide

  41. Scala Center
    » Open source foundation for
    Scala

    View Slide

  42. Scala Center
    » Open source foundation for
    Scala
    » Engage with developers in
    the open-source community
    to improve the language
    » Modularization (more to
    come in 2.13)
    » Smaller core to
    standardize & stabilize

    View Slide

  43. Scala Center
    » Open source foundation for
    Scala
    » Engage with developers in
    the open-source community
    to improve the language
    » Modularization (more to
    come in 2.13)
    » Smaller core to
    standardize & stabilize
    » Help developers learn the
    language

    View Slide

  44. Scala Language

    View Slide

  45. Scala 2.1x
    » 2.12 ("compiler release")
    » Java 8 (λ, SAM, default)
    » New backend & optimizer
    » Q2 2016

    View Slide

  46. Scala 2.1x
    » 2.12 ("compiler release")
    » Java 8 (λ, SAM, default)
    » New backend & optimizer
    » Q2 2016
    » 2.13 ("modularization")
    » Focused on libraries,
    collections, and Spark

    View Slide

  47. Scala.js 0.6.9
    » Bringing Scala even closer
    to the JavaScript community
    » Current version is mostly
    about Scala 2.12 support
    and faster compilation
    » Closer and closer to
    production-grade

    View Slide

  48. Scala Native
    » Low-level primitives
    » Pointers, structs, etc
    » Interop with C
    » extern objects, no
    runtime overhead
    » Compiled ahead-of-time via
    LLVM
    » Instant startup vs JIT

    View Slide

  49. Scala in five years

    View Slide

  50. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies

    View Slide

  51. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala

    View Slide

  52. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala
    » Focus is mainly on simplification

    View Slide

  53. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala
    » Focus is mainly on simplification
    » Add new features & stay approachable by dropping
    other features
    » macros, general type projection, early
    initializers, existential types, etc

    View Slide

  54. dotty features

    View Slide

  55. Five years of Scala
    (A few things I've picked up)

    View Slide

  56. 1. Always be learning
    » I dove into Scala because of this book:

    View Slide

  57. Async boundary

    View Slide

  58. Async boundary

    View Slide

  59. Synchronous

    View Slide

  60. Asynchronous

    View Slide

  61. 2. You don't have to know
    everything ...
    » That time I had a job interview with Martin

    View Slide

  62. ... but it can't
    hurt to try!
    » Buy this book and master:
    » Higher-order functions,
    currying, partial
    functions, functional
    data structures,
    handling errors without
    exceptions, strictness
    and laziness, purely
    functional state ...

    View Slide

  63. 3. Learn to explain the value
    functional, immutable, asynchronous, non-blocking,
    real-time, highly-available, loosely coupled,
    scalable, fault-tolerant, concurrent, low latency,
    high throughput...
    How do you map the properties of Scala and functional
    programming to business value?

    View Slide

  64. 4. Encourage, teach, mentor
    » We need developers of all experience levels,
    academic backgrounds, and professional backgrounds

    View Slide

  65. Thank you!
    Kevin Webber, Enterprise Architect, Lightbend
    Email: [email protected]
    Medium: medium.com/@kvnwbbr
    Twitter: @kvnwbbr

    View Slide