Save 37% off PRO during our Black Friday Sale! »

Pros and cons of Scala as a server-side programming language at ShiftForward

Pros and cons of Scala as a server-side programming language at ShiftForward

ShiftForward has been using Scala almost since its inception, in 2011. During the past few years, both the language and the way ShiftForward uses it has evolved. This talk aims to explain ShiftForward's experiences with Scala: why we decided to go with it in the first place, what benefits we take from it, how it affects our programming style and what issues we have encountered in using it.

7724a3ee1890f271f424878b0524ae15?s=128

Joao Azevedo

July 02, 2016
Tweet

Transcript

  1. PROS AND CONS OF SCALA AS A SERVER-SIDE PROGRAMMING LANGUAGE

    AT SHIFTFORWARD / Joao Azevedo @jcazevedo http://jcazevedo.net/commit2016
  2. ABOUT ME Graduated from FEUP in 2010; Worked on a

    crew-scheduling application for a railway company at SISCOG from 2010 to 2011; Worked on mobile application development and signal analysis tools at Fraunhofer Portugal from 2011 to 2012; At Shi Forward since 2012, working on distributed, responsive and scalable systems for the online advertising industry.
  3. SHIFTFORWARD: WHY SCALA IN THE FIRST PLACE?

  4. STATICALLY TYPED WITH DECENT TYPE INFERENCE Specifying program invariants at

    compile time: Increases development speed; Removes the need for some classes of tests. Not having to declare every single type removes excessive verbosity.
  5. JVM Platform independence (predictable runtime environment) eases the deployment of

    services; The ecosystem is very rich: libraries and tools.
  6. CONCURRENCY Thread-based concurrency (from Java); Actor-based concurrency (inspired by Erlang,

    available in Akka); Parallel collections; Controllable immutability.
  7. SCALA PROS: THE TYPE SYSTEM

  8. SCALA CLASS HIERARCHY AND UNIFIED TYPES

  9. PARAMETRIC POLYMORPHISM d e f d u p [ T

    ] ( x : T , n : I n t ) : L i s t [ T ] = i f ( n = = 0 ) N i l e l s e x : : d u p ( x , n - 1 ) s c a l a > d u p ( 3 , 3 ) r e s 0 : L i s t [ I n t ] = L i s t ( 3 , 3 , 3 ) s c a l a > d u p ( " t h r e e " , 3 ) r e s 1 : L i s t [ S t r i n g ] = L i s t ( t h r e e , t h r e e , t h r e e )
  10. TYPE INFERENCE d e f i d [ T ]

    ( x : T ) = x s c a l a > v a l x = i d ( 3 2 2 ) x : I n t = 3 2 2 s c a l a > v a l x = i d ( " h e y " ) x : S t r i n g = h e y
  11. FUNCTIONS AS TYPES d e f f o o [

    A , B ] ( l : L i s t [ A ] , f : A = > B ) : L i s t [ B ] = i f ( l . l e n g t h = = 0 ) N i l e l s e f ( l . h e a d ) : : f o o ( l . t a i l , f ) s c a l a > f o o ( L i s t ( 1 , 2 , 3 ) , { x : I n t = > x * 2 } ) r e s 0 : L i s t [ I n t ] = L i s t ( 2 , 4 , 6 )
  12. TRAITS Collection of fields and behaviors that can be extended

    or mixed in to classes. t r a i t C a r { v a l b r a n d : S t r i n g d e f e m i t B r a n d : S t r i n g = " M y b r a n d i s " + b r a n d } t r a i t S h i n y { v a l s h i n e R e f l e c t i o n : I n t } c l a s s B M W e x t e n d s C a r w i t h S h i n y { v a l b r a n d = " B M W " v a l s h i n e R e f l e c t i o n = 1 2 } s c a l a > v a l b m w = n e w B M W b m w : B M W = B M W @ 2 d a 8 e d 8 0 s c a l a > b m w . e m i t B r a n d r e s 0 : S t r i n g = M y b r a n d i s B M W
  13. SCALA PROS: DEFINING DOMAIN OBJECTS

  14. CASE CLASSES c a s e c l a s

    s U s e r ( i d : I n t , n a m e : S t r i n g , a g e : I n t ) Can be constructed without using n e w ; Automatically have equality and hash code methods; Nice t o S t r i n g methods; Pattern matching.
  15. CASE CLASSES c a s e c l a s

    s U s e r ( i d : I n t , n a m e : S t r i n g , a g e : I n t ) c a s e c l a s s S i t e ( u s e r s : L i s t [ U s e r ] ) v a l s i t e 1 = S i t e ( L i s t ( U s e r ( 0 , " a l i c e " , 3 7 ) , U s e r ( 1 , " b o b " , 4 6 ) ) ) v a l s i t e 2 = S i t e ( L i s t ( U s e r ( 0 , " a l i c e " , 3 7 ) , U s e r ( 1 , " b o b " , 4 6 ) ) ) s c a l a > s i t e 1 . t o S t r i n g r e s 0 : S t r i n g = S i t e ( L i s t ( U s e r ( 0 , a l i c e , 3 7 ) , U s e r ( 1 , b o b , 4 6 ) ) ) s c a l a > s i t e 1 = = s i t e 2 r e s 1 : B o o l e a n = t r u e
  16. PATTERN MATCHING d e f a v e r a

    g e A g e ( s i t e : S i t e ) = { d e f a g e S u m ( u s e r s : L i s t [ U s e r ] ) : I n t = u s e r s m a t c h { c a s e U s e r ( _ , _ , a g e ) : : r e s t = > a g e + a g e S u m ( r e s t ) c a s e N i l = > 0 } a g e S u m ( s i t e . u s e r s ) / s i t e . u s e r s . l e n g t h } v a l s i t e 1 = S i t e ( L i s t ( U s e r ( 0 , " a l i c e " , 3 7 ) , U s e r ( 1 , " b o b " , 4 6 ) ) ) s c a l a > a v e r a g e A g e ( s i t e 1 ) r e s 0 : I n t = 4 1
  17. EXAMPLE: MODELING JSON s e a l e d t

    r a i t J s V a l u e { d e f t o J s o n S t r i n g : S t r i n g } c a s e c l a s s J s N u m b e r ( n : I n t ) e x t e n d s J s V a l u e { d e f t o J s o n S t r i n g = n . t o S t r i n g } c a s e c l a s s J s S t r i n g ( s : S t r i n g ) e x t e n d s J s V a l u e { d e f t o J s o n S t r i n g = " \ " " + s + " \ " " } c a s e c l a s s J s A r r a y ( v a l u e s : A r r a y [ J s V a l u e ] ) e x t e n d s J s V a l u e { d e f t o J s o n S t r i n g = { v a l e l e m e n t s S t r i n g = v a l u e s . m a p ( _ . t o J s o n S t r i n g ) . m k S t r i n g ( " , " ) " [ " + e l e m e n t s S t r i n g + " ] " } }
  18. EXAMPLE: MODELING JSON c a s e c l a

    s s J s O b j e c t ( v a l u e s : M a p [ S t r i n g , J s V a l u e ] ) e x t e n d s J s V a l u e { d e f t o J s o n S t r i n g = { v a l e l e m e n t s = v a l u e s . m a p { c a s e ( k , v ) = > " \ " " + k + " \ " : " + v . t o J s o n S t r i n g } " { " + e l e m e n t s . m k S t r i n g ( " , " ) + " } " } } v a l u s e r J s o n = J s O b j e c t ( M a p ( " i d " - > J s N u m b e r ( 0 ) , " n a m e " - > J s S t r i n g ( " a l i c e " ) , " a g e " - > J s N u m b e r ( 3 7 ) ) ) s c a l a > u s e r J s o n . t o J s o n S t r i n g r e s 0 : S t r i n g = { " i d " : 0 , " n a m e " : " a l i c e " , " a g e " : 3 7 }
  19. SCALA PROS: MONADS

  20. A WAY TO ABSTRACT COMPUTATIONS t r a i t

    M [ A ] d e f u n i t [ A ] : A = > M [ A ] = ? ? ? d e f f l a t M a p [ A , B ] : M [ A ] = > ( A = > M [ B ] ) = > M [ B ] = ? ? ?
  21. MONADIC OPERATIONS EVERYWHERE s c a l a > L

    i s t ( 1 , 2 , 3 ) . f l a t M a p { x = > L i s t ( x - 1 , x , x + 1 ) } r e s 0 : L i s t [ I n t ] = L i s t ( 0 , 1 , 2 , 1 , 2 , 3 , 2 , 3 , 4 ) s c a l a > L i s t ( 1 , 2 , 3 ) . m a p { x = > x * 2 } r e s 1 : L i s t [ I n t ] = L i s t ( 2 , 4 , 6 )
  22. (FLAT)MAP IS THE GLUE v a l l 1 =

    L i s t ( 1 , 2 ) v a l l 2 = L i s t ( 8 , 9 ) l 1 f l a t M a p { v 1 = > l 2 m a p { v 2 = > v 1 * v 2 } } r e s 0 : L i s t [ I n t ] = L i s t ( 8 , 9 , 1 6 , 1 8 )
  23. FOR COMPREHENSIONS v a l l 1 = L i

    s t ( 1 , 2 ) v a l l 2 = L i s t ( 8 , 9 ) f o r { v 1 < - l 1 v 2 < - l 2 } y i e l d v 1 * v 2 r e s 0 : L i s t [ I n t ] = L i s t ( 8 , 9 , 1 6 , 1 8 )
  24. REUSABLE COMPONENTS d e f f ( v : I

    n t ) = v * 2 v a l l = L i s t ( 1 , 2 , 3 , 4 ) v a l o = S o m e ( 4 ) v a l t = T r y ( 5 ) s c a l a > l . m a p ( f ) r e s 0 : L i s t [ I n t ] = L i s t ( 2 , 4 , 6 , 8 ) s c a l a > o . m a p ( f ) r e s 1 : O p t i o n [ I n t ] = S o m e ( 8 ) s c a l a > t . m a p ( f ) r e s 2 : s c a l a . u t i l . T r y [ I n t ] = S u c c e s s ( 1 0 )
  25. SCALA PROS: TYPE CLASSES

  26. PREREQUISITE: IMPLICIT PARAMETERS d e f a d d e

    r ( a : I n t ) ( i m p l i c i t b : I n t ) = a + b s c a l a > a d d e r ( 2 ) < c o n s o l e > : 1 3 : e r r o r : c o u l d n o t f i n d i m p l i c i t v a l u e f o r p a r a m e t e r b : I n t a d d e r ( 2 ) ^ s c a l a > i m p l i c i t v a l x = 5 x : I n t = 5 s c a l a > a d d e r ( 2 ) r e s 0 : I n t = 7
  27. PREREQUISITE: IMPLICIT CONVERSIONS d e f d o u b

    l e ( v : I n t ) = v * 2 i m p l i c i t d e f s t r i n g T o I n t ( s : S t r i n g ) = s . t o I n t s c a l a > d o u b l e ( " 1 2 3 4 " ) r e s 0 : I n t = 2 4 6 8
  28. AD HOC POLYMORPHISM t r a i t J s

    o n F o r m a t [ A ] { d e f w r i t e ( v a l u e : A ) : J s V a l u e d e f r e a d ( j s o n : J s V a l u e ) : A } d e f t o J s o n [ A ] ( a : A ) ( i m p l i c i t f o r m a t : J s o n F o r m a t [ A ] ) : S t r i n g = { f o r m a t . w r i t e ( a ) . t o J s o n S t r i n g } i m p l i c i t o b j e c t I n t J s o n F o r m a t e x t e n d s J s o n F o r m a t [ I n t ] { d e f w r i t e ( v a l u e : I n t ) = J s N u m b e r ( v a l u e ) d e f r e a d ( j s o n : J s V a l u e ) = j s o n m a t c h { c a s e J s N u m b e r ( v a l u e ) = > v a l u e c a s e _ = > t h r o w n e w E x c e p t i o n ( " U n e x p e c t e d J S O N t y p e " ) } } s c a l a > t o J s o n ( 1 ) r e s 7 : S t r i n g = 1 s c a l a > t o J s o n ( " I ' m a s t r i n g ! " ) < c o n s o l e > : 1 7 : e r r o r : c o u l d n o t f i n d i m p l i c i t v a l u e f o r p a r a m e t e r f o r m a t : J s o n F o r m a t [ S t r i n g ] t o J s o n ( " I ' m a s t r i n g ! " ) ^
  29. EXTENSION METHODS t r a i t J s o

    n W r i t a b l e [ A ] { d e f t o J s o n : S t r i n g } i m p l i c i t d e f t o J s o n W r i t e a b l e [ A ] ( v : A ) ( i m p l i c i t f o r m a t : J s o n F o r m a t [ A ] ) = n e w J s o n W r i t a b l e [ A ] { d e f t o J s o n = f o r m a t . w r i t e ( v ) . t o J s o n S t r i n g } s c a l a > 1 . t o J s o n r e s 0 : S t r i n g = 1 s c a l a > U s e r ( 0 , " a l i c e " , 3 7 ) . t o J s o n r e s 1 : S t r i n g = { " i d " : 0 , " n a m e " : " a l i c e " , " a g e " : 3 7 }
  30. SCALA PROS: CONCURRENCY

  31. FUTURE s c a l a > F u t

    u r e { / * e x p e n s i v e c o m p u t a t i o n * / 2 } r e s 0 : s c a l a . c o n c u r r e n t . F u t u r e [ I n t ] = s c a l a . c o n c u r r e n t . i m p l . P r o m i s e $ D e f a u l t P r o m i s e @ 2 c c 4 4 a d s c a l a > r e s 0 . o n C o m p l e t e ( p r i n t l n ) S u c c e s s ( 2 )
  32. COMPOSING FUTURES

  33. FUTURE IS A MONAD v a l u s e

    r F u t u r e : F u t u r e [ U s e r ] = u s e r I n f o ( r e q u e s t . u s e r n a m e ) v a l g e o F u t u r e : F u t u r e [ G e o ] = u s e r G e o ( r e q u e s t . i p ) v a l t p D a t a : F u t u r e [ D a t a ] = u s e r F u t u r e . f l a t M a p ( u s e r = > t h i r d P a r t y D a t a ( u s e r . i d ) ) v a l i n c o m e I n f o : F u t u r e [ D o u b l e ] = g e o F u t u r e . f l a t M a p ( g e o = > a v e r a g e I n c o m e ( g e o ) ) f o r { u s e r < - u s e r F u t u r e d a t a < - t p D a t a a v g I n c o m e < - i n c o m e I n f o } y i e l d A u g m e n t e d U s e r ( u s e r , d a t a , a v g I n c o m e )
  34. AKKA c l a s s M y A c

    t o r e x t e n d s A c t o r { d e f r e c e i v e = { c a s e v a l u e : S t r i n g = > p r i n t l n ( " r e c e i v e d " + v a l u e ) c a s e _ = > p r i n t l n ( " r e c e i v e d u n k n o w n m e s s a g e " ) } } s c a l a > v a l a c t o r = s y s t e m . a c t o r O f ( P r o p s ( n e w M y A c t o r ) ) a c t o r : a k k a . a c t o r . A c t o r R e f = A c t o r [ a k k a : / / d e f a u l t / u s e r / $ b # 1 1 3 7 2 1 1 9 8 4 ] s c a l a > a c t o r ! " h e l l o " r e c e i v e d h e l l o s c a l a > a c t o r ! 1 r e c e i v e d u n k n o w n m e s s a g e
  35. AKKA c l a s s D o u b

    l e r e x t e n d s A c t o r { d e f r e c e i v e = { c a s e v : I n t = > s e n d e r ! ( v * 2 ) c a s e _ = > p r i n t l n ( " r e c e i v e d u n k n o w n m e s s a g e " ) } } s c a l a > v a l r e s = a c t o r ? 3 r e s : s c a l a . c o n c u r r e n t . F u t u r e [ A n y ] = S u c c e s s ( 6 )
  36. AKKA: WHAT ELSE? The message-driven programming model promotes asynchronous interfaces;

    Supervision strategies enable fault tolerance; Location transparency promotes scalability; Persistency enables internal state recovery.
  37. LARGE-SCALE DATA PROCESSING: APACHE SPARK R D D is another

    monad! v a l t e x t F i l e = s c . t e x t F i l e ( " h d f s : / / . . . " ) v a l c o u n t s = t e x t F i l e . f l a t M a p ( l i n e = > l i n e . s p l i t ( " " ) ) . m a p ( w o r d = > ( w o r d , 1 ) ) . r e d u c e B y K e y ( _ + _ ) c o u n t s . s a v e A s T e x t F i l e ( " h d f s : / / . . . " )
  38. OTHER SCALA PROS Power to create DSLs; Macros; Higher kinded

    types.
  39. SCALA CONS: LEARNING CURVE

  40. QUICK TO START USING, HARD TO MASTER When coming from

    an OO-background, it's easy to write OO code in Scala.
  41. FUNCTIONAL IN THE SMALL, OO IN THE LARGE Pure functions,

    referential transparency and immutability are applied in small, isolated areas.
  42. MORE FUNCTIONAL ABSTRACTIONS ; ; . Scalaz Shapeless Cats

  43. OO OR FUNCTIONAL DESIGN? Do I design it using typeclasses

    and ad hoc polymorphism or using subtyping and parametric polymorphism?
  44. SCALA CONS: COMPILATION TIMES

  45. STARTUP OVERHEAD Scalac has lots of classes which have to

    be loaded and jit- compiled. The long startup time is mitigated by using incremental compilation.
  46. COMPILATION SPEED Type inference; Implicit resolution; Functional idioms generate many

    more classes per given file size than Java.
  47. COMPILATION SPEED

  48. COMPILATION SPEED

  49. COMPILATION SPEED

  50. SCALA CONS: BUILD SYSTEM

  51. SBT IS WIDELY USED Simple (?) build tool: a type-safe

    programmable build configuration.
  52. SLIGHTLY OVERENGINEERED AND CONVOLUTED A build definition consists of one

    or more S e t t i n g s; A S e t t i n g describes a transformation of the build description; S e t t i n g s are scoped: project, configuration and task axis; A T a s k is a computation of values or side effects; T a s k s are also assigned to keys and can depend on settings and other tasks; T a s k K e y [ T ] returns S e t t i n g [ T a s k [ T ] ] .
  53. DEPENDENCY RESOLUTION IS SLOW The dependency resolution (ivy) is single

    threaded, and not cached by default; There are alternatives: coursier, a pure Scala dependency resolver, is around 5 times faster resolving dependencies in a multi-project (28 projects) build (5 vs 25 minutes).
  54. SCALA CONS: IDE AND TOOLING

  55. IDES IntelliJ IDEA; Scala IDE (for Eclipse); ENSIME (for Emacs,

    Atom, Vim and Sublime).
  56. RAPIDLY EVOLVING ECOSYSTEM IDE plugins are not as sophisticated (particularly

    when compared with Java), so developers might get frustrated. Work on the Scala presentation compiler (used by Scala IDE and ENSIME) and other tools makes us look with optimism to the future.
  57. QUESTIONS?