Scala: State of the Union

In this keynote, Kevin Webber, Enterprise Architect at Lightbend, will discuss the past, present, and future of Scala; where it has been and where it’s going. Yes, this talk will focus on the new language features introduced in 2.12, but also dive into an equally important aspect of Scala in 2016: the incredible community that’s grown up around Scala over the years. Kevin will frame the talk around his own journey to Scala and how both the language and the community made him a better developer and architect.

Transcript

1. Scala
   State of the Union
   Kevin Webber
   Enterprise Architect @
   Lightbend
   e: kevin.webber@lightbend.com
   m: medium.com/@kvnwbbr
   t: @kvnwbbr
   

   View Slide

2. A bit about me
   » Enterprise Architect at Lightbend
   

   View Slide

3. A bit about me
   » Enterprise Architect at Lightbend
   » Started exploring Scala in 2008
   

   View Slide

4. A bit about me
   » Enterprise Architect at Lightbend
   » Started exploring Scala in 2008
   » Former Lightbend customer (2012-2014)
   

   View Slide

5. Agenda
   » Where are we today?
   » Spoiler: 2.11.8
   

   View Slide

6. Agenda
   » Where are we today?
   » Spoiler: 2.11.8
   » Where are we going?
   » 2.12, 2.13, and beyond (aka, dotty)
   

   View Slide

7. Agenda
   » Where are we today?
   » Spoiler: 2.11.8
   » Where are we going?
   » 2.12, 2.13, and beyond (aka, dotty)
   » Where is the industry focused?
   » Scala is not an island
   » What have I learned over five years?
   

   View Slide

8. Where are we today?
   

   View Slide

9. View Slide

10. Scala is active!
    » Significant contributions by
    » open-source community
    » partners (EPFL, Lightbend, Scala Center)
    » Releases average 2x per year
    

    View Slide

11. Scala repo stats
    

    View Slide

12. View Slide

13. View Slide

14. View Slide

15. Companies driving
    growth
    

    View Slide

16. Companies
    driving
    growth?
    » Apple → jobs!
    » Coursera
    » IBM
    » Intel
    » Mesosphere
    

    View Slide

17. Companies
    driving
    growth?
    » Apple
    » Coursera → learning
    » IBM
    » Intel
    » Mesosphere
    

    View Slide

18. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM → Big Data University
    » Intel
    » Mesosphere
    

    View Slide

19. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel → Gearpump
    » Mesosphere
    

    View Slide

20. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel → Gearpump
    » Mesosphere
    

    View Slide

21. Companies
    driving
    growth?
    » Apple
    » Coursera
    » IBM
    » Intel
    » Mesosphere → DC/OS
    

    View Slide

22. Business
    drivers?
    » Data is ruling the world
    » Metal is abstracted
    » Functional programming
    rules this space
    » Object-oriented programming
    is a familiar path
    

    View Slide

23. Scala projects
    driving growth
    

    View Slide

24. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system
    

    View Slide

25. View Slide

26. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system
    » Spark (Apache)
    » General-purpose cluster computing
    

    View Slide

27. View Slide

28. View Slide

29. » Kafka (Confluent)
    » Distributed, publish-subscribe messaging system
    » Spark (Apache)
    » General-purpose cluster computing
    » Akka (Lightbend)
    » Distributed messaging toolkit
    

    View Slide

30. Akka
    » The actor model abstraction
    provides:
    » distribution
    » location transparency
    » isolation
    

    View Slide

31. Akka Streams
    » Higher-level abstraction on
    top of Akka
    » Per-event processing of
    streams
    » Back-pressure enabled by
    default
    

    View Slide

32. Why Scala?
    Example "one-liner" to implement per-event stream
    processing with Akka Streams and Kafka
    implicit val actorSystem = ActorSystem("ReactiveKafka")
    implicit val materializer = ActorMaterializer()
    val kafka = new ReactiveKafka(host = "localhost:9092", zooKeeperHost = "localhost:2181")
    val publisher = kafka.consume("lowercaseStrings", "groupName", new StringDecoder())
    val subscriber = kafka.publish("uppercaseStrings", "groupName", new StringEncoder())
    // consume lowercase strings from kafka and publish them transformed to uppercase
    Source(publisher).map(_.toUpperCase).to(Sink(subscriber)).run()
    

    View Slide

33. View Slide

34. val g = RunnableGraph.fromGraph(GraphDSL.create() {
    implicit builder =>
    import GraphDSL.Implicits._
    // Source
    val A: Outlet[String] = builder.add(Source.fromIterator(() => flightDelayLines)).out
    // Flows
    val B: FlowShape[String, FlightEvent] = builder.add(csvToFlightEvent)
    val C: FlowShape[FlightEvent, FlightDelayRecord] = builder.add(filterAndConvert)
    val D: UniformFanOutShape[FlightDelayRecord, FlightDelayRecord] = builder.add(Broadcast[FlightDelayRecord](2))
    val F: FlowShape[FlightDelayRecord, (String, Int, Int)] = builder.add(averageCarrierDelay)
    // Sinks
    val E: Inlet[Any] = builder.add(Sink.ignore).in
    val G: Inlet[Any] = builder.add(Sink.foreach(averageSink)).in
    // Graph
    A ~> B ~> C ~> D
    E <~ D
    G <~ F <~ D
    ClosedShape
    })
    g.run()
    

    View Slide

35. Fast Data
    Architecture
    » Akka Streams
    » < 1 second latency per-
    event processing
    » Spark Streaming
    » 1-10+ second latency for
    mini-batch event
    processing
    » Kafka captures the complete
    event stream
    

    View Slide

36. Big picture
    » An ecosystem is growing
    around Scala for:
    » Cloud computing
    » Stream processing
    » Fast data, big data,
    fast big data
    » Microservices
    » Scala is a natural fit in
    these spaces
    

    View Slide

37. Notable libraries
    » Argonaut → purely functional JSON
    » Cats → foundation for pure, typeful libraries
    » ScalaCheck → property-based testing
    » ScalaTest → test Scala, Java, and Scala.js
    » scalaz → purely functional data structures
    » Shapeless → generic programming library
    » Specs → executable software specifications
    

    View Slide

38. Scala Educational
    Opportunities
    

    View Slide

39. Educational opportunities
    » Carnegie-Mellon University
    » EPFL
    » Harvard University
    » Stanford University
    » University of Texas - Austin
    » University of Washington
    » West Point
    

    View Slide

40. View Slide

41. Scala Center
    » Open source foundation for
    Scala
    

    View Slide

42. Scala Center
    » Open source foundation for
    Scala
    » Engage with developers in
    the open-source community
    to improve the language
    » Modularization (more to
    come in 2.13)
    » Smaller core to
    standardize & stabilize
    

    View Slide

43. Scala Center
    » Open source foundation for
    Scala
    » Engage with developers in
    the open-source community
    to improve the language
    » Modularization (more to
    come in 2.13)
    » Smaller core to
    standardize & stabilize
    » Help developers learn the
    language
    

    View Slide

44. Scala Language
    

    View Slide

45. Scala 2.1x
    » 2.12 ("compiler release")
    » Java 8 (λ, SAM, default)
    » New backend & optimizer
    » Q2 2016
    

    View Slide

46. Scala 2.1x
    » 2.12 ("compiler release")
    » Java 8 (λ, SAM, default)
    » New backend & optimizer
    » Q2 2016
    » 2.13 ("modularization")
    » Focused on libraries,
    collections, and Spark
    

    View Slide

47. Scala.js 0.6.9
    » Bringing Scala even closer
    to the JavaScript community
    » Current version is mostly
    about Scala 2.12 support
    and faster compilation
    » Closer and closer to
    production-grade
    

    View Slide

48. Scala Native
    » Low-level primitives
    » Pointers, structs, etc
    » Interop with C
    » extern objects, no
    runtime overhead
    » Compiled ahead-of-time via
    LLVM
    » Instant startup vs JIT
    

    View Slide

49. Scala in five years
    

    View Slide

50. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    

    View Slide

51. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala
    

    View Slide

52. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala
    » Focus is mainly on simplification
    

    View Slide

53. dotty
    » Currently a platform to try out new language
    concepts and compiler technologies
    » Is the next Scala
    » Focus is mainly on simplification
    » Add new features & stay approachable by dropping
    other features
    » macros, general type projection, early
    initializers, existential types, etc
    

    View Slide

54. dotty features
    

    View Slide

55. Five years of Scala
    (A few things I've picked up)
    

    View Slide

56. 1. Always be learning
    » I dove into Scala because of this book:
    

    View Slide

57. Async boundary
    

    View Slide

58. Async boundary
    

    View Slide

59. Synchronous
    

    View Slide

60. Asynchronous
    

    View Slide

61. 2. You don't have to know
    everything ...
    » That time I had a job interview with Martin
    

    View Slide

62. ... but it can't
    hurt to try!
    » Buy this book and master:
    » Higher-order functions,
    currying, partial
    functions, functional
    data structures,
    handling errors without
    exceptions, strictness
    and laziness, purely
    functional state ...
    

    View Slide

63. 3. Learn to explain the value
    functional, immutable, asynchronous, non-blocking,
    real-time, highly-available, loosely coupled,
    scalable, fault-tolerant, concurrent, low latency,
    high throughput...
    How do you map the properties of Scala and functional
    programming to business value?
    

    View Slide

64. 4. Encourage, teach, mentor
    » We need developers of all experience levels,
    academic backgrounds, and professional backgrounds
    

    View Slide

65. Thank you!
    Kevin Webber, Enterprise Architect, Lightbend
    Email: kevin.webber@lightbend.com
    Medium: medium.com/@kvnwbbr
    Twitter: @kvnwbbr
    

    View Slide