Scala - методы организации потока управления: пространство между императивностью и декларатированностью

Dd3f18c87b851137000c7427d7bd5d32?s=47 fwdays
August 14, 2014

Scala - методы организации потока управления: пространство между императивностью и декларатированностью

Руслан Шевченко

Dd3f18c87b851137000c7427d7bd5d32?s=128

fwdays

August 14, 2014
Tweet

Transcript

  1. SCALA - organizing control flow (between imperative and declarative approaches)

    Ruslan Shevchenko GoSave ! ruslan@shevchenko.kiev.ua @rssh1 https://github.com/rssh
  2. Styles of control flow organizations: ! • Future-s … •

    async/await • Actors • Channels • Reactive Collections • DSLs for Computation Plans
  3. Control Flow val x = readX val y = readY

    val z = x*y scala X Y Z Imperative = we explicitly set one
  4. directed by evaluation strategy z = x*y where x =

    readX y = readX haskell X Y Z let x = readX y = readX in x*y
  5. Control flow: what we need ? ! • Manage multi{core,machine}

    control- flows. ! • Optimize resource utilization ( see reactive manifesto http://www.reactivemanifesto.org/ )
  6. Reactivity ! • Ugly situation in industry ! • In

    ideal world - {operation system/ language VM} must care about resource utilization, human - about logic ! • Our world is far from ideal
  7. Control Flow val x = readX val y = readY

    val z = x*y Can we readX and Y in parallel X Y Z
  8. Low level @volatile var x = _ val xThread =

    new Thread( public void run() { x = readX } ).start(); ! @volatile var y = _ val yThread = new Thread( public void run() { y = readY } ).start(); yThread.join() xThread.join() z = x+y As GOTO from 60-s
  9. Low level with thread pool val xTask = new Task(

    public X run() { readX } ); pool.submit(xTask) ! val yTask = new Task( public void run() { readY } ) pool.submit(yTask) ! z = xTask.get()+yTask.get() X Y Z
  10. Scala Future X Y Z val x = Future{ readX

    } val y = Future{ readY } val z = Await.result(x, 1 minute) + Await.result(y, 1 minute)
  11. Future ! • Future[T] = pointer to the value in

    the future ! • isComplete: Boolean ! • Await.result(feature, duration): T ! • onComplete(Try[T] => U) : Unit
  12. Future { readX } object Future { ! def apply(body:

    =>T) : Future[T] = ………… ! } Call by name syntax = [call by function .. ] by name - term from Algol 68 Own functions like control flow constructions
  13. Future-s are composable. ! • map: X=>Y Future[X] => Future[Y]

    ! • map[Y](x:Future[X])(f:X=>Y): Future[Y] ! • flatMap: • X => Future[Y] Future[X]=>Future[Y] ! • flatMap[Y](x:Future[X])(f:X=>Future[Y]): Future[Y]
  14. Scala Future val x = Future{ readX } val y

    = Future{ readY } val z = Future{ … after X and Y } val z = Future{ readX } flatMap { x => readY map ( y=> x+y ) } ! for{ x <- Future{ readY }, y <- Future{ readY }) yield x+y ! Using monadic syntax:
  15. Future ! • Good for simple cases ! • Hard

    when we want to implement complex logic !
  16. SIP22 (Async/Await) ! • when we want to implement complex

    logic ! • Ozz style -> (limited) F# ->(limited) C# ! • in scala as library: https://github.com/ scala/async !
  17. SIP22 (Async/Await) ! • async(body: T):Future[T] ! • await(future: Future[T]):

    T • can be used only inside async ! • async macro rewrite body as state machine. !
  18. Async/Await val x = Future{ readX } val y =

    Future{ readY } val z = Future{ … after X and Y } val z = async{ val x = Future{ readX } val y = Future{ readY } await(x) + await(y) }
  19. Async/Await val z = async{ val x = Future{ readX

    } val y = Future{ readY } await(x) + await(y) } val z = { val s1 = Future{ readX(); } val s2 = Future{ readY(); } val s3 = if (s1.isComplete) if (s2.isComplete) { Promise.success(s1.result + s2.result) } else { s2 map s3 } } else s1 map s3 s3 // Just show the idea, not actual
  20. Akka http://www.akka.io ! • Erlang-style concurrency ! • Actor -

    active object • send/receive message • have local state • available by name in akka cluster. ! ! ! !
  21. Actor ! class EventsProcessor extends Actor { def receive =

    { case Event(msg) => println(msg) case Echo(msg) => sender ! Echo(msg) case Ping => sender ! Pong case Stop => context.stop(self) } ! }
  22. Akka ! • tell(x:Any)=>Unit — send and forget actor !

    x ! ! • ack(x:Any)=>Future[Any] — send and receive future to answer actor ? x ! !
  23. Actor Mailbox Processor (with state) 1

  24. Akka ! • Scheduler ! • Event Bus ! •

    Common patterns • Load balancing • Throttling messages • ….. ! !
  25. Akka ! • Scale on cluster ! • Recovery !

    • Persistent Queue ! !
  26. Go-like channels ! • Come from Go language • http://golang.org/

    ! • Implementation as library and async. • (yet not ready) - https://github.com/rssh/scala-gopher branch “async/unsugared.”
  27. Rx channels ! • Reactive extensions • http://rxscala.github.io/ ! •

    Rx collection call you. !
  28. Rx scala ! trait Observer[E] { def onNext(e: E) def

    onError(e: Throwable) def onCompleted() ! } ! ! ! trait Observable[E] { ! def subscribe(o:Observer) ! ………. ! map, flatMap, zip, filter … } !
  29. Computation plan DSL ! • Collections operations can be distributed

    ! • Simple form: .par for( x <- collection.par) yield x+1 ! ! • Same idea for hadoop map/reduce !
  30. Computation plan DSL for Hadoop ! • Scalding • https://github.com/twitter/scalding

    ! • Scoobi • http://nicta.github.io/scoobi/ ! • Scrunch • http://crunch.apache.org/scrunch.html
  31. Scoobi ! val lines = fromTextFile("hdfs://in/...") ! val counts =

    lines.mapFlatten(_.split(" ")) .map(word => (word, 1)) .groupByKey .combine(Sum.int) ! counts.toTextFile(\“hdfs://out/…", overwrite=true).persist(ScoobiConfiguration()) map, groupByKey, combine => Map/Reduce tasks
  32. Mahout: computation plan DSL for Spark ! • https://mahout.apache.org/ !

    • Scalable machine learning library. • R-like matrix operations • Optimizer for algebraic expression
  33. Machout ! // R-like operations (linear algebra) ! val g

    = bt.t %*% bt - c - c.t + (s_q cross s_q) * (xi dot xi) ! drmA.mapBlock(ncol = r) { case (keys, blockA) => val blockY = blockA %*% Matrices.symmetricUniformView(n, r, omegaSeed) keys -> blockY } Match operations => computations plans on backend
  34. Scala: organization of control flow ! • Many styles. No

    one is better. ! • Low-level: futures & callbacks ! • Middle-level: actors, channels, streams ! • Hight-level: declarative DSL
  35. Scala: organization of control flow ! • Thanks for attention.

    ! • Questions (?) ! • //Ruslan Shevchenko <ruslan@shevchenko.kiev.ua> !