Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running Scala on AWS Lambda in a Snappy Way

Running Scala on AWS Lambda in a Snappy Way

Shunsuke Tadokoro

June 28, 2019
Tweet

More Decks by Shunsuke Tadokoro

Other Decks in Technology

Transcript

  1. Running Scala on AWS Lambda in a snappy way ScalaMatsuri

    2019 @todokr BizReach, Inc. "84-BNCEBͰ4DBMBΛαΫαΫಈ͔͢ and fun!
  2. The opinions expressed here do not represent
 those of any

    organization. ͜͜Ͱࣔ͞Ε͍ͯΔݟղ͸ɺ
 ॴଐ૊৫ͷݟղΛ൓ө͢Δ΋ͷͰ͸͋Γ·ͤΜ
  3. Who? Shunsuke Tadokoro @todokr ‣A software engineer working at BizReach,

    Inc. ‣Love: Scala, Clojure, Emacs, etc. ‣Contributed: Play Framework, ScalikeJDBC, etc. ‣Wrote: ాॴॣ༎
 גࣜձࣾϏζϦʔνͷιϑτ΢ΣΞΤϯδχΞ
  4. sbt-project-switcher ‣ A sbt plugin to switch sub projects quickly⚡

    TCUͷαϒϓϩδΣΫτΛαΫαΫ੾Γସ͑ΔϓϥάΠϯ
  5. ×

  6. AWS Lambda ‣a serverless computing service what we call 'FaaS'

    ‣Runtimes ‣ Python ‣ Node.js ‣ Java8 ‣ Go ‣ C# ‣ PowerShell ‣ Ruby ‣ custom runtime ‣Just write function and execute ͍ΘΏΔ'BB4ͱݺ͹ΕΔαʔόʔϨείϯϐϡʔςΟϯάαʔϏε
 ͨͩؔ਺Λॻ͍࣮ͯߦ͢Δ͚ͩ
  7. AWS Lambda ‣ Fully managed ‣ No server management ‣

    High Availability & Scalable ‣ So, you can focus on creating value ϑϧϚωʔδυɿ αʔόʔͷ؅ཧ͕ෆཁɺϏδωεՁ஋ΛੜΉ͜ͱʹूதͰ͖Δ
  8. AWS Lambda ‣ Fully managed ‣ No server management ‣

    High Availability & Scalable ‣ So, you can focus on creating value ‣ Low cost ‣ Free tier ‣ Pay only for compute time ‣ No charge when code is not running ௿ίετɿ ແྉ࿮͋ΓɺίϯϐϡʔςΟϯά࣌ؒʹରͯ͠՝ۚ
  9. I hope to run Scala on AWS Lambda! ‣ No

    context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ೴಺ίϯςΩετεΠονͷ཈੍ͷͨΊ
 ͜Ε͸1ZUIPO ͋Ε͸/PEF ͋ͬͪ͸(Pͱ͍͏ͷ͸ͳΔ΂͘ආ͚͍ͨ
  10. I hope to run Scala on AWS Lambda! ‣ No

    context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ‣ Types ‣ Lambda handler is rarely maintained ‣ Information richness comes from types save time
 to understand code -BNCEBϋϯυϥ͸Ұ౓ॻ͔ΕΔͱ์ஔ͞Ε͕ͪ
 ܕͷදݱྗ͸ίʔυͷཧղΛॿ͚ͯ͘ΕΔ
  11. I hope to run Scala on AWS Lambda! ‣ No

    context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ‣ Types ‣ Lambda handler is rarely maintained ‣ Information richness comes from types save time
 to understand code ‣ Test ‣ Able to use testing library you get used to ׳ΕͨςετϥΠϒϥϦΛ࢖͏͜ͱ͕Ͱ͖Δ
  12. However... ‣ It's often said that 
 "Running JVM language

    on Lambda is too slow!" ‣ So called "cold-start problem" ʮ-BNCEBͰ+7.ݴޠ͸஗͍ʯͱ͠͹͠͹ݴΘΕΔ
  13. However... ‣ It's often said that 
 "Running JVM language

    on Lambda is too slow!" ‣ So called "cold-start problem" ‣ There is limitation of function package size ‣ Jar size: up to 50 MB ‣ Need to care about libraries to depend on ؔ਺ύοέʔδͷαΠζ͸.#Ҏ಺ͱ͍͏੍ݶ΋ଘࡏ͢Δ
  14. So, questions are... ‣ How slow Scala on AWS Lambda

    is? ‣ How can we make package size small? ‣ How can we run Scala on AWS Lambda in a snappy way? Ͳͷ͘Β͍஗͍ͷ͔ʁͲ͏ͨ͠ΒύοέʔδαΠζΛখ͘͞Ͱ͖Δ͔ʁ
 Ͳ͏ͨ͠Β-BNCEBͰ4DBMBΛαΫαΫಈ͔͢͜ͱ͕Ͱ͖Δ͔ʁ
  15. "
 Hello world (avg duration & memory usage of 5

    times) ·ͣ͸)FMMPXPSME͔Β࣮ݧ
  16. Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello

    { def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.#
 ࠷େϝϞϦ࢖༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms
  17. Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello

    { def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.#
 ࠷େϝϞϦ࢖༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms
  18. Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello

    { def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.#
 ࠷େϝϞϦ࢖༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms
  19. Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello

    { def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.#
 ࠷େϝϞϦ࢖༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms
  20. 2nd trial... TODO screen shot 2nd trial is blazing fast!

    ճ໨ͷॴཁ࣌ؒ͸NTͱ଎͍
  21. 2nd trial... TODO screen shot 2nd trial is blazing fast!

    1st trial took 8 seconds, so called "cold-start" problem ॳճ͸ඵɺ͍ΘΏΔʮίʔϧυελʔτʯ໰୊
  22. What is cold-start? We should know Lambda lifecycle! ؔ਺ύοέʔδͷμ΢ϯϩʔυ Download

    code 1. Download the function package 
 (zip or jar, file on S3) you wrote
  23. What is cold-start? We should know Lambda lifecycle! ίʔυ࣮ߦ༻ͷαϯυϘοΫείϯςφͷىಈ Download

    code Start new container 2. Start the sandbox container where your code will be executed
  24. What is cold-start? We should know Lambda lifecycle! ϥϯλΠϜͷىಈ
 +BWBͷ৔߹ɺ+7.ͷىಈ΍ΫϥεϑΝΠϧͷϩʔυͳͲ΋

    Download code Start new container Bootstrap runtime 3. Bootstrap runtime to run your code, 
 including JVM spin-up, load class files, etc.
  25. What is cold-start? We should know Lambda lifecycle! ؔ਺ύοέʔδͷ࣮ߦ Download

    code Start new container Bootstrap runtime Run code 4. Run your function package code and
 affect the world
  26. What is cold-start? We should know Lambda lifecycle! ίϯςφΛϑϦʔζ͠ɺ࣍ճͷݺͼग़͠Λ଴ͭ
 ͨͩ͠ɺ࣍ճͷݺͼग़͕͠ಉ͡ΠϯελϯεͰ࣮ߦ͞ΕΔอূ͸ͳ͍

    Download code Start new container Bootstrap runtime Run code Freeze the container 5. Freeze the container and wait for a next invocation However there's no guarantee it will run on the same instance. 
 Container may be removed.
  27. Cold-start is a process that follows 
 "download" to "run".

    Note that downloading jar and spinning-up
 JVM relatively take time. What is cold-start? We should know Lambda lifecycle! ίʔϧυελʔτ͸ίʔυͷμ΢ϯϩʔυ͔Β࣮ߦ·ͰΛܦΔ
 KBSͷμ΢ϯϩʔυ΍+7.ͷىಈ͸ൺֱత͕͔͔࣌ؒΔॲཧ Download code Start new container Bootstrap runtime Run code Freeze the container
  28. Warm-start is a process which reuses the same instance, so

    there's no spin-up time. This is why each invocations after the first 
 is fast. What is cold-start? We should know Lambda lifecycle! ΢ΥʔϜελʔτͱ͸-BNCBΠϯελϯε͕࠶ར༻͞ΕΔ࣮ߦ
 ͜Ε͕ճ໨Ҏ߱ͷݺͼग़͕͠଎͔ͬͨཧ༝ Download code Start new container Bootstrap runtime Run code Freeze the container
  29. first-time-only misunderstanding ‣ Not the first time only! ‣ If

    there is no available container, cold-start happens ‣ Lambda containers have the limitation that they can handle 
 only single request at a time ‣ If two requests come in at once, two containers are required 
 to handle these requests ‣ If there is no available one, it starts from container creation ෳ਺ಉ࣌ϦΫΤετͳͲɺ࣮ߦՄೳͳίϯςφ͕ଘࡏ͠ͳ͍৔߹ʹ΋ى ͖Δ
  30. first-time-only misunderstanding ‣ Not the first time only! ‣ If

    there is no available container, cold-start happens ‣ Lambda containers have the limitation that they can handle 
 only single request at a time ‣ If two requests come in at once, two containers are required 
 to handle these requests ‣ If there is no available one, it starts from container creation ‣ When a large request spike comes, the throughput problem occurs or it exceed Lambda's concurrency limitation େ͖ͳϦΫΤετͷεύΠΫ͕དྷͨ৔߹ʹɺεϧʔϓοτ͕໰୊ʹͳͬͨΓ -BNCEBͷಉ࣮࣌ߦ਺ͷ੍ݶʹͻ͔͔ͬͬͯ͠·͏͜ͱ΋
  31. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size -BNCEBʹׂΓ౰ͯΒΕͨϝϞϦαΠζ
 ଞͷϥϯλΠϜʹൺ΂ɺ+7.͸ىಈʹൺֱతଟ͘ͷϝϞϦΛඞཁͱ͢Δ
  32. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size -BNCEBʹׂΓ౰ͯΒΕͨϝϞϦαΠζ
 ଞͷϥϯλΠϜʹൺ΂ɺ+7.͸ىಈʹൺֱతଟ͘ͷϝϞϦΛඞཁͱ͢Δ
  33. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ؔ਺ύοέʔδͷαΠζ
 େ͖͚Ε͹େ͖͍΄Ͳɺμ΢ϯϩʔυʹ΋͕͔͔࣌ؒΔ
  34. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ؔ਺ύοέʔδͷαΠζ
 େ͖͚Ε͹େ͖͍΄Ͳɺμ΢ϯϩʔυʹ΋͕͔͔࣌ؒΔ
  35. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ‣ Number of class files to be loaded ‣ Large number of classes to be loaded and initialized introduces
 large amount of latency ΫϥεϑΝΠϧͷ਺
 ϩʔυ͓ΑͼॳظԽ͞ΕΔΫϥεϑΝΠϧ͕ଟ͚Ε͹ɺͦͷ෼஗͘ͳΔ
  36. Factors increase cold-start latency ‣ Attached memory size ‣ Memory

    footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ‣ Number of class files to be loaded ‣ Large number of classes to be loaded and initialized introduces large amount of latency ‣ VPC, etc. ͦͷଞɺ71$ͳͲ΋Өڹ
  37. Another factor in Scala app ‣ Scala class files tend

    to be numerous ‣ These class files are loaded and initialized when it used 4DBMBͷΫϥεϑΝΠϧ͸਺͕ଟ͘ͳΓ͕ͪ
 ར༻ͷࡍʹ͸͜ΕΒ΋ϩʔυɾॳظԽ͞ΕΔ $ tree . /unzipped-jar/scala scala/ ├── AnyVal.class ├── AnyValCompanion.class ├── App.class ├── Array$$anon$2.class ├── Array$.class ... │ ├── Regex$Replacement.class │ ├── Regex.class │ └── UnanchoredRegex.class └── volatile.class 34 directories, 2503 files $ tree ./ ... |____Hello$.class |____Hello.class |____Request.class |____Request$.class ... | |____PartialFunction$anonfun$apply$1.class | |____SttpBackendOptions$Proxy$$anon$1.class ...
  38. Scala on Lambda problems ‣ Java runtime is at a

    disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ىಈ࣌ͷϝϞϦϑοτϓϦϯτ΍μ΢ϯϩʔυ͢ΔKBSͷαΠζɺ
 ΫϥεϑΝΠϧͷϩʔυͱॳظԽͳͲͷ఺Ͱɺ+BWBϥϯλΠϜ͸ෆར
  39. Scala on Lambda problems ‣ Java runtime is at a

    disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ‣ Scala's another disadvantage ‣ Scala class files tend to be numerous 4DBMBͷΫϥεϑΝΠϧ͸਺͕ଟ͘ͳΓ͕ͪͳͷ΋ෆརͳ఺
  40. Scala on Lambda problems ‣ Java runtime is at a

    disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ‣ Scala's another disadvantage ‣ Scala class files tend to be numerous ‣ Thus, Scala cold-start in Java runtime takes longer than other language runtime ‣ even "Hello world", it took 8s ‣ when in warm-start, 0.5ms Αͬͯɺ4DBMBY+BWBϥϯλΠϜͷίʔϧυελʔτ͸
 ଞͷϥϯλΠϜΑΓ΋͕͔͔࣌ؒͬͯ͠·͏
  41. hmm... so how can we run Scala snappy on Lambda?

    ͏ʔΉɺ͡Ό͋Ͳ͏ͨ͠ΒαΫαΫಈ͔ͤΔͩΖ͏͔ʁ
  42. Run Scala on Lambda without JVM ‣ However JVM is

    great, the short-lived Lambda instance doesn't take advantage of the characteristics of that ‣ Such as dynamic class loading or HotSpot JIT compile, etc. +7.͸Ғେ͕ͩɺੜଘظ͕ؒ୹͍-BNCEBΠϯελϯεͰ͸
 ͦͷྑ͍ಛੑ͕׆͔ͮ͠Β͍
  43. Run Scala on Lambda without JVM ‣ However JVM is

    great, the short-lived Lambda instance doesn't take advantage of the characteristics of that ‣ Such as dynamic class loading or HotSpot JIT compile, etc. ‣ It might be awesome if it runs without JVM! ‣ Not to wait JVM spin-up and loading classes ‣ Make executable package size small ‣ Keep memory footprint small +7.ͳ͠Ͱ࣮ߦͰ͖ͨΒྑ͍݁ՌʹͳΔ͔΋͠Εͳ͍ʂ
 +7.ͷεϐϯΞοϓΛ଴ͨͣɺؔ਺ύοέʔδͱϝϞϦ࢖༻ྔΛখ͘͞
  44. How? ‣ Scala.js with Node.js runtime ‣ Scala Native with

    Custom runtime 4DBMB/BUJWFΛΧελϜϥϯλΠϜͰಈ͔͢
  45. How? ‣ Scala.js with Node.js runtime ‣ Scala Native with

    Custom runtime ‣ Native image built by GraalVM with Custom runtime (SBBM7.Ͱੜ੒ͨ͠ωΠςΟϒΠϝʔδΛΧελϜϥϯλΠϜͰಈ͔͢
  46. Scala.js ‣ A statically typed AltJS ‣ Generate JavaScript code

    from Scala ‣ Provided as a sbt compiler plugin // plugins.sbt addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.26") // build.sbt enablePlugins(ScalaJSPlugin) // Greet.scala object Greet extends App { println("Hello from Scala.js!") } ੩తܕ෇͚"MU+4ͷҰछͰɺ4DBMBͷίʔυ͔Β+BWB4DSJQUΛੜ੒͢Δ
 TCUͷίϯύΠϥϓϥάΠϯͱͯ͠ఏڙ͞Ε͍ͯΔ
  47. class of ES6 & Scala.js class Person(val firstName: String, val

    lastName: String) { def fullName(): String = s"$firstName $lastName" } class Person { constructor(firstName, lastName) { this.firstName = firstName; this.lastName = lastName; } fullName() { return `${this.firstName} ${this.lastName}`; } } ES6 Scala.js
  48. Scala.js facade for Node.js ‣ scalajs-io/nodejs https://github.com/scalajs-io/nodejs ‣ A facade

    library provides Node.js API interface ‣ Enable type hinting and checking // before (not type safe) import js.Dynamic def getToken(): UndefOr[String] = Dynamic.global.process.env.ACCESS_TOKEN.asInstanceOf[UndefOr[String]] // after import io.scalajs.nodejs.{Environment, process} def getToken2(): Environment = process.env TDBMBKTJPOPEFKTͱ͍͏/PEFKTͷϑΝαʔυϥΠϒϥϦ͕͋Δ
 /PEFKTͷ"1*Λ҆શʹݺͼग़͢͜ͱ͕Ͱ͖Δ
  49. Scala Native ‣ Generate executable binary file from Scala ‣

    Provided as a sbt compiler plugin // plugins.sbt addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.3.8") // build.sbt scalaVersion := "2.11.12" enablePlugins(ScalaNativePlugin) // Greet.scala object Greet extends App { println("Hello from Scala Native!") } 4DBMBͷίʔυ͔Β࣮ߦՄೳͳόΠφϦϑΝΠϧΛੜ੒͢Δ
 TCUͷίϯύΠϥϓϥάΠϯͱͯ͠ఏڙ͞Ε͍ͯΔ
  50. Scala Native ‣ Try generating executable and running without JVM

    # compile & generate machine code $ sbt nativeLink # execute $ time ./target/scala-2.11/greet-out Hello from Scala Native! real 0m0.019s user 0m0.004s sys 0m0.007s όΠφϦΛੜ੒࣮ͯ͠ߦͯ͠ΈΔ
  51. AWS Lambda custom runtime ‣ Any language that can run

    in the Lambda execution environment is available ‣ Simple HTTP based interface ‣ to get the event payload for a new invocation ‣ to return back the response from the function -BNCEBͷ؀ڥͰ࣮ߦͰ͖ΔݴޠͳΒԿͰ΋ར༻Մೳ ϦΫΤετϨεϙϯεͷͨΊͷγϯϓϧͳ)551ΠϯλϑΣʔεΛ࣋ͭ
  52. GraalVM ‣ Polyglot VM developed by Oracle ‣ Enable to

    run JVM langs, JS, Python, Ruby, etc. with zero overhead ‣ SubstrateVM ‣ A GraalVM component that allows AOT compilation of Java code
 into executable native images ‣ Community Edition / Enterprise Edition 0SBDMF͕։ൃ͍ͯ͠Δ1PMZHMPU7.
 4VCTUSBUF7.͸࣮ߦՄೳͳωΠςΟϒΠϝʔδΛੜ੒͢Δίϯϙʔωϯτ
  53. Java frameworks use SubstrateVM ‣ Quarkus https://quarkus.io ‣ A "Supersonic

    Subatomic Java" framework ‣ Micronaut Framework https://micronaut.io ‣ A full-stack framework for microservice and serverless applications ‣ Project Helidon https://helidon.io ‣ A collection of Java libraries for writing microservices (SBBM7.ͰωΠςΟϒΠϝʔδΛੜ੒͢Δ+BWBϑϨʔϜϫʔΫ
  54. Hello world in Scala.js // build.sbt lazy val commonSettings =

    Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ build.sbt
  55. Hello world in Scala.js $PNNPO+4ελΠϧͰϞδϡʔϧΛੜ੒͢Δ // build.sbt lazy val commonSettings

    = Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ Emit CommonJS style modules
  56. Hello world in Scala.js ੜ੒͞Εͨ+4͕഑ஔ͞ΕΔύεΛఆٛ // build.sbt lazy val commonSettings

    = Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ Define the path which is generated JS placed
  57. Hello world in Scala.js // Hello.scala import scala.scalajs.js import scala.scalajs.js.annotation.JSExportTopLevel

    @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Hello.scala
  58. Hello world in Scala.js +4ͷΦϒδΣΫτΛܕ҆શʹѻ͏ͨΊͷϑΝαʔυܕ // Hello.scala import scala.scalajs.js import

    scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ A facade type to handle JS object type safely
  59. Hello world in Scala.js ίʔϧόοΫؔ਺ͷܕΛఆٛ // Hello.scala import scala.scalajs.js import

    scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Define the type of callback function
  60. Hello world in Scala.js ͕͜͜ϋϯυϥͷओཁͳύʔτ // Hello.scala import scala.scalajs.js import

    scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ This is the main part of handler
  61. Hello world in Scala.js ΦϒδΣΫτΛHMPCBMʹΤΫεϙʔτ͢ΔΞϊςʔγϣϯ // Hello.scala import scala.scalajs.js import

    scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ A annotation exports that object to the JS global scope
  62. Hello world in Scala.js ϦΫΤετ͔ΒϝοηʔδΛ૊ΈཱͯɺίʔϧόοΫؔ਺ΛݺͿ // Hello.scala import scala.scalajs.js import

    scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Create message from request and invoke callback function
  63. Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU

    %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT
  64. Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU

    %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT
  65. Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU

    %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT
  66. Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU

    %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT
  67. Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU

    %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT So fast! ΊͬͪΌ଎͍ʂ
  68. Hello world in Scala Native Building executable for Lambda is

    complex a bit -BNCEB޲͚࣮ߦՄೳϑΝΠϧΛ4DBMB/BUJWFͰ࡞Δͷ͸গʑෳࡶ
  69. Hello world in Scala Native Building executable for Lambda is

    complex a bit ‣ Write Scala Native code 4DBMB/BUJWFͷίʔυΛॻ͘
  70. Hello world in Scala Native Building executable for Lambda is

    complex a bit ‣ Write Scala Native code ‣ Build the Amazon Linux based docker container ‣ Install required libs such as LLVM, curl, openssl, re2, etc. ‣ Publish those as a Lambda Layer "NB[PO-JOVYϕʔεͷ%PDLFSίϯςφΛϏϧυ͢Δɻ ඞཁͳϥΠϒϥϦΛΠϯετʔϧ͠ɺͦΕΒΛ-BNCEB-BZFSͱͯ͠഑ஔ
  71. Hello world in Scala Native Building executable for Lambda is

    complex a bit ‣ Write Scala Native code ‣ Build the Amazon Linux based docker container ‣ Install required libs such as LLVM, curl, openssl, re2, etc. ‣ Publish those as a Lambda Layer ‣ Build executable with the docker container and deploy %PDLFSΛ࢖࣮ͬͯߦՄೳϑΝΠϧΛϏϧυ͠ɺσϓϩΠ
  72. Hello world in Scala Native ‣ build.sbt // build.sbt lazy

    val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin)
  73. Hello world in Scala Native ‣ Settings for Scala version,

    linkage option, compile option, etc. // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4DMBMBόʔδϣϯ΍ϦϯέʔδɺίϯύΠϧͷΦϓγϣϯΛઃఆ
  74. Hello world in Scala Native ‣ Libraries written for Scala

    Native are available // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4DBMB/BUJWFʹରԠ͍ͯ͠ΔϥΠϒϥϦͳΒར༻͕Մೳ
  75. Hello world in Scala Native ‣ STTP: HTTP lib, to

    invoke Custom runtime API ‣ Argonaut: JSON lib, to handle JSON request / response // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4551)551ϥΠϒϥϦ ΧελϜϥϯλΠϜ"1*Λݺͼग़͢༻
 "SHPOBVU+40/ϥΠϒϥϦ ϦΫΤετϨεϙϯεͷ+40/༻
  76. Hello world in Scala Native ‣ Hello.scala import com.softwaremill.sttp._ import

    argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String)
  77. Hello world in Scala Native ‣ Initialize: get runtime api

    path, etc. import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ॳظԽॲཧϥϯλΠϜ"1*ͷύεऔಘͳͲ
  78. Hello world in Scala Native ‣ Process 1 request with

    1 loop import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ϧʔϓͰϦΫΤετΛॲཧ͢Δ
  79. Hello world in Scala Native ‣ Fetch event from next

    API and parse as request import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) OFYU"1*Λୟ͖ɺ݁ՌΛ3FRVFTUΦϒδΣΫτͱͯ͠ύʔε
  80. Hello world in Scala Native ‣ Send build message to

    response API import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ૊ΈཱͯͨϝοηʔδΛϨεϙϯε"1*ʹૹΔ
  81. Hello world in Scala Native Build code with docker FROM

    amazonlinux:latest ENV SCALA_VERSION="2.11.12" WORKDIR /build/ RUN yum install -y java-1.8.0 && \ yum install -y tar.x86_64 && \ yum install -y gzip gunzip RUN curl -O http://downloads.typesafe.com/scala/${SCALA_VERSION}/scala-${SCALA_VERSION}.tgz RUN tar -xzvf scala-${SCALA_VERSION}.tgz && \ rm -rf scala-${SCALA_VERSION}.tgz && \ echo "export SCALA_HOME=/home/ec2-user/scala-${SCALA_VERSION}" >> ~/.bashrc && \ echo "export PATH=$PATH:/home/ec2-user/scala-${SCALA_VERSION}/bin:/opt/llvm-3.9.0/bin" >> ~/.bashrc && \ source ~/.bashrc #SBT RUN curl https://bintray.com/sbt/rpm/rpm | tee /etc/yum.repos.d/bintray-sbt-rpm.repo RUN yum install -y sbt WORKDIR /build/runtime/ RUN yum install -y -q yum-utils RUN yum-config-manager --enable epel > /dev/null RUN yum -y update RUN yum -y group install "development tools" RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm RUN echo $'[alonid-llvm-3.9.0] \n\ name=Copr repo for llvm-3.9.0 owned by alonid \n\ baseurl=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/epel-7-$basearch/ \n\ type=rpm-md \n\ skip_if_unavailable=True \n\ gpgcheck=1 \n\ gpgkey=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/pubkey.gpg \n\ repo_gpgcheck=0 \n\ enabled=1 \n\ enabled_metadata=1' >> /etc/yum.repos.d/epel.repo RUN yum install -y clang-3.9.0 RUN yum install -y llvm-3.9.0 llvm-3.9.0-devel RUN yum install -y zip which libunwind libunwind-devel python-pip jq libcurl-devel RUN mkdir -p /build/runtime/lib/ && cp /usr/lib64/libunwind.so /build/runtime/lib/libunwind.so.8 && cp /usr/lib64/libunwind-x86_64.so.8 /build/runtime/lib/libunwind-x86_64.so.8 RUN yum install -y libidn libidn-devel && cp /usr/lib64/libidn.so.11 lib/libidn.so.11 RUN yum install -y git make openssl-devel RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_62_0/curl-7.62.0.tar.gz && tar -zxvf curl-7.62.0.tar.gz RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz && tar -zxvf OpenSSL_1_0_2l.tar.gz RUN git clone https://github.com/google/re2.git WORKDIR /build/runtime/re2/ ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ENV CXX=/opt/llvm-3.9.0/bin/clang++ LDFLAGS="-static-libstdc++" RUN make && make install RUN cp /usr/local/lib/libre2.so.0 /build/runtime/lib/libre2.so.0 WORKDIR /build/runtime/openssl-OpenSSL_1_0_2l RUN ./config --prefix=/opt/lib/ssl --openssldir=/opt/lib/ssl shared zlib RUN make && make install WORKDIR /build/runtime/curl-7.62.0/ ENV LD_LIBRARY_PATH=/opt/lib:/usr/lib64:$LD_LIBRARY_PATH RUN mkdir -p /build/runtime/lib/ssl && cp -r /opt/lib/ssl/lib /build/runtime/lib/ssl/lib RUN ./configure --prefix=$(/build/runtime) --with-ssl=/opt/lib/ssl && make && make install # libs RUN cp /lib/libcurl.so.4 /build/runtime/lib/libcurl.so.4 && cp /opt/lib/ssl/lib/libcrypto.so.1.0.0 /build/runtime/lib/libcrypto.so.1.0.0 && cp /opt/lib/ssl/lib/libssl.so.1.0.0 /build/runtime/lib/libssl.so.1.0.0 WORKDIR /build/runtime/ RUN rm OpenSSL_1_0_2l.tar.gz curl-7.62.0.tar.gz RUN mkdir -p /build/main WORKDIR /build/main ENV LD_LIBRARY_PATH=/opt/lib/ssl/lib:/usr/local/lib:$LD_LIBRARY_PATH ADD build.sbt /build/main/ ADD project/ /build/main/ CMD tail -f /dev/null
  82. Hello world in Scala Native Build code with docker ڊେ

    FROM amazonlinux:latest ENV SCALA_VERSION="2.11.12" WORKDIR /build/ RUN yum install -y java-1.8.0 && \ yum install -y tar.x86_64 && \ yum install -y gzip gunzip RUN curl -O http://downloads.typesafe.com/scala/${SCALA_VERSION}/scala-${SCALA_VERSION}.tgz RUN tar -xzvf scala-${SCALA_VERSION}.tgz && \ rm -rf scala-${SCALA_VERSION}.tgz && \ echo "export SCALA_HOME=/home/ec2-user/scala-${SCALA_VERSION}" >> ~/.bashrc && \ echo "export PATH=$PATH:/home/ec2-user/scala-${SCALA_VERSION}/bin:/opt/llvm-3.9.0/bin" >> ~/.bashrc && \ source ~/.bashrc #SBT RUN curl https://bintray.com/sbt/rpm/rpm | tee /etc/yum.repos.d/bintray-sbt-rpm.repo RUN yum install -y sbt WORKDIR /build/runtime/ RUN yum install -y -q yum-utils RUN yum-config-manager --enable epel > /dev/null RUN yum -y update RUN yum -y group install "development tools" RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm RUN echo $'[alonid-llvm-3.9.0] \n\ name=Copr repo for llvm-3.9.0 owned by alonid \n\ baseurl=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/epel-7-$basearch/ \n\ type=rpm-md \n\ skip_if_unavailable=True \n\ gpgcheck=1 \n\ gpgkey=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/pubkey.gpg \n\ repo_gpgcheck=0 \n\ enabled=1 \n\ enabled_metadata=1' >> /etc/yum.repos.d/epel.repo RUN yum install -y clang-3.9.0 RUN yum install -y llvm-3.9.0 llvm-3.9.0-devel RUN yum install -y zip which libunwind libunwind-devel python-pip jq libcurl-devel RUN mkdir -p /build/runtime/lib/ && cp /usr/lib64/libunwind.so /build/runtime/lib/libunwind.so.8 && cp /usr/lib64/libunwind-x86_64.so.8 /build/runtime/lib/libunwind-x86_64.so.8 RUN yum install -y libidn libidn-devel && cp /usr/lib64/libidn.so.11 lib/libidn.so.11 RUN yum install -y git make openssl-devel RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_62_0/curl-7.62.0.tar.gz && tar -zxvf curl-7.62.0.tar.gz RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz && tar -zxvf OpenSSL_1_0_2l.tar.gz RUN git clone https://github.com/google/re2.git WORKDIR /build/runtime/re2/ ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ENV CXX=/opt/llvm-3.9.0/bin/clang++ LDFLAGS="-static-libstdc++" RUN make && make install RUN cp /usr/local/lib/libre2.so.0 /build/runtime/lib/libre2.so.0 WORKDIR /build/runtime/openssl-OpenSSL_1_0_2l RUN ./config --prefix=/opt/lib/ssl --openssldir=/opt/lib/ssl shared zlib RUN make && make install WORKDIR /build/runtime/curl-7.62.0/ ENV LD_LIBRARY_PATH=/opt/lib:/usr/lib64:$LD_LIBRARY_PATH RUN mkdir -p /build/runtime/lib/ssl && cp -r /opt/lib/ssl/lib /build/runtime/lib/ssl/lib RUN ./configure --prefix=$(/build/runtime) --with-ssl=/opt/lib/ssl && make && make install # libs RUN cp /lib/libcurl.so.4 /build/runtime/lib/libcurl.so.4 && cp /opt/lib/ssl/lib/libcrypto.so.1.0.0 /build/runtime/lib/libcrypto.so.1.0.0 && cp /opt/lib/ssl/lib/libssl.so.1.0.0 /build/runtime/lib/libssl.so.1.0.0 WORKDIR /build/runtime/ RUN rm OpenSSL_1_0_2l.tar.gz curl-7.62.0.tar.gz RUN mkdir -p /build/main WORKDIR /build/main ENV LD_LIBRARY_PATH=/opt/lib/ssl/lib:/usr/local/lib:$LD_LIBRARY_PATH ADD build.sbt /build/main/ ADD project/ /build/main/ CMD tail -f /dev/null Oh, huge scala-on-lambda-snappy/scalanative/Dockerfile
  83. Hello world in Scala Native ‣ Build docker container &

    build executable %PDLFSίϯςφͷϏϧυͱɺ࣮ߦՄೳϑΝΠϧͷϏϧυ # bootstrap $ cd scala-on-lambda-snappy/scalanative $ docker build -t native-builder-image . $ docker run --name native-builder -dt native-builder-image $ docker cp ./ native-builder:/build/main/ $ docker exec -it native-builder sbt 'project hello' nativeLink $ docker cp native-builder:/build/main/hello/target/scala-2.11/hello-out \ hello/bootstrap $ zip -Dj hello.zip ./hello/bootstrap # Deploy this zip # runtime layer $ docker cp native-builder:/build/runtime/ ./runtime/ $ cd ./runtime $ zip -r runtime.zip . # Publish this zip as Lambda Layer
  84. Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF

    $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT
  85. Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF

    $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT
  86. Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF

    $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT
  87. Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF

    $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT
  88. Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF

    $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT Yay! ΍ͬͨͶʂ
  89. Hello world in GraalVM native image Build is easier than

    Scala Native a bit 4DBMB/BUJWFͷϏϧυΑΓ͸গ͠؆୯
  90. Hello world in GraalVM native image Build is easier than

    Scala Native a bit ‣ Write Scala code 4DBMBͷίʔυΛॻ͘
  91. Hello world in GraalVM native image Build is easier than

    Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly TCUBTTFNCMZͰGBUKBSΛϏϧυ͢Δ
  92. Hello world in GraalVM native image Build is easier than

    Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly ‣ Build the GraalVM docker container ‣ Community Edition, in this sample (SBBM7.ͷ%PDLFSίϯςφΛϏϧυ͢Δ
  93. Hello world in GraalVM native image Build is easier than

    Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly ‣ Build the GraalVM docker container ‣ Community Edition, in this sample ‣ Build executable from jar with the docker and deploy %PDLFSΛ࢖ͬͯKBS͔Β࣮ߦՄೳϑΝΠϧΛϏϧυ͠ɺσϓϩΠ͢Δ
  94. Hello world in GraalVM native image ‣ build.sbt // build.sbt

    lazy val commonSettings = Seq( scalaVersion := "2.12.8" ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %% "core" % "1.5.11", "io.argonaut" %% "argonaut" % "6.2.2" ) )
  95. Hello world in GraalVM native image ‣ Almost the same

    as Scala Native's Hello world 4DBMB/BUJWFͷ)FMMPXPSMEͱ΄΅ಉ͡ import argonaut._; import Argonaut._; import com.softwaremill.sttp._ object Hello { implicit val backend = HttpURLConnectionBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") def main(args: Array[String]): Unit = { val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("lambda-runtime-aws-request-id").head val request = next.unsafeBody.decodeOption[Request].get val responseJson = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post(uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response") .body(responseJson).send() } } } case class Request(name: String)
  96. Hello world in GraalVM native image ‣ First, build fat-jar

    ‣ Then, build native image from jar with GraalVM ·ͣGBUKBSΛϏϧυ
 ࣍ʹ(SBBM7.ͰKBS͔ΒOBUJWFJNBHFΛϏϧυ $ cd scala-on-lambda-snappy/graalvm # build fat-jar $ sbt 'project hello' assembly # build native image $ docker build -t graal-builder-image . $ docker run --name graal-builder -dt graal-builder-image $ docker cp ./hello/target/scala-2.12/hello-assembly-0.1-SNAPSHOT.jar \ graal-builder:/hello.jar $ docker exec -it graal-builder native-image -jar hello.jar \ -H:EnableURLProtocols=http,https --no-server $ docker cp graal-builder:/hello ./hello/bootstrap $ zip -Dj hello.zip ./hello/bootstrap
  97. Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F

    .BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT
  98. Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F

    .BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT
  99. Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F

    .BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT
  100. Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F

    .BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT
  101. Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F

    .BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .#  NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT Good! ͍͍Ͷʂ
  102. AWS integration with Scala.js ‣ Use AWS SDK for JavaScript

    ‣ There is some tips to use the SDK type safely ‣ In this example, use DynamoDB Document Client "844%,GPS+BWB4DSJQUΛ࢖͏͕ɺܕ҆શʹѻ͏ʹ͸গʑίπ͕͍Δ ͜ͷྫͰ͸%ZOBNP%#ͷ%PDVNFOU$MJFOUΛ࢖͏
  103. Querying DynamoDB with Scala.js ‣ AWS.scala // AWS.scala @JSImport("aws-sdk", JSImport.Namespace)

    @js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } }
  104. Querying DynamoDB with Scala.js ‣ Import AWS SDK for JavaScript

    // AWS.scala @JSImport("aws-sdk", JSImport.Namespace) @js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } } "844%,GPS+BWB4DSJQUͷΠϯϙʔτ
  105. Querying DynamoDB with Scala.js ‣ Facade for DocumentClient to handle

    type safely // AWS.scala @JSImport("aws-sdk", JSImport.Namespace) @js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } } %PDVNFOU$MJFOUΛܕ҆શʹѻ͏ͨΊͷϑΝαʔυ
  106. Querying DynamoDB with Scala.js ‣ DynamoAccessor.scala object DynamoAccessor { val

    documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } }
  107. Querying DynamoDB with Scala.js ‣ Initialize DocumentClient object DynamoAccessor {

    val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } %PDVNFOU$MJFOUͷΠχγϟϥΠζ
  108. Querying DynamoDB with Scala.js ‣ Prepare param (using Dynamic, so

    not type safe...) object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ύϥϝʔλͷ༻ҙ %ZOBNJDΛ࢖͍ͬͯΔͷͰܕ҆શͰ͸ͳ͍
  109. Querying DynamoDB with Scala.js ‣ To make it type safe,

    define type like the Request trait object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ܕ҆શʹ͢ΔͳΒɺ3FRVFTUτϨΠτͷΑ͏ʹܕΛఆٛ͢Δ
  110. Querying DynamoDB with Scala.js ‣ Submit a query with the

    client object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ΫϥΠΞϯτΛ࢖ͬͯΫΤϦΛ౤͛Δ
  111. Facade library for aws-sdk ‣ exoego/aws-sdk-scalajs-facade 
 https://github.com/exoego/aws-sdk-scalajs-facade ‣ Scala.js

    type facade for aws-sdk-js ‣ A fork of balshor/aws-sdk-scalajs-facade ‣ It's also good for learning how to write facade type by yourself FYPFHPBXTTELTDBMBKTGBDBEFͱ͍͏ϑΝαʔυϥΠϒϥϦ͕͋Δ
 ϑΝαʔυͷॻ͖ํΛֶͿͷʹ΋Αͦ͞͏
  112. AWS with Scala Native? ‣ Currently, there's no Scala Native

    library of AWS SDK ‣ So need to implement a client which invokes AWS REST API ‣ cf. rusoto: A third-party AWS SDK for Rust https://rusoto.org 4DBMB/BUJWF൛ͷ"844%,GPS+BWB͸ݱঢ়ͳ͍
 ΋͠"84αʔϏεΛ࢖͍͍ͨ৔߹ɺΫϥΠΞϯτΛ࣮૷͢Δඞཁ͕͋Δ
  113. AWS with GraalVM? ‣ GraalVM is able to build native

    images from Java, however... (SBBM7.͸+BWB͔ΒOBUJWFJNBHFΛϏϧυͰ͖Δ͕
  114. AWS with GraalVM? ‣ GraalVM is able to build native

    images from Java, however... ‣ There is some limitations of SubstrateVM 4VCTUSBUF7.ͷ͍͔ͭ͘ͷ੍ݶ͕͋Δ
  115. AWS with GraalVM? ‣ GraalVM is able to build native

    images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading
 https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ྫ͑͹ɺDMBTTͷಈతͳMPBEJOHVOMPBEJOH͸αϙʔτ͞Ε͍ͯͳ͍
  116. AWS with GraalVM? ‣ GraalVM is able to build native

    images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading
 https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ‣ AWS SDK for Java uses commons-logging which uses ClassLoader... "844%,GPS+BWBͷDPNNPOTMPHHJOH͕$MBTT-PBEFSΛ࢖͍ͬͯΔ
  117. AWS with GraalVM? ‣ GraalVM is able to build native

    images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading
 https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ‣ AWS SDK for Java uses commons-logging which uses ClassLoader... "844%,GPS+BWBͷDPNNPOTMPHHJOH͕$MBTT-PBEFSΛ࢖͍ͬͯΔ
  118. Kuromoji ‣ atilika / kuromoji https://github.com/atilika/kuromoji ‣ A Japanese morphological

    analyzer written in Java ‣ Its name is derived from "ࠇจࣈ" ‣ A traditional small wooden knife to cut Japanese sweets in tea ceremony ,VSPNPKJ+BWB੡ͷ೔ຊޠܗଶૉղੳϥΠϒϥϦ
 ໊લ͸࿨՛ࢠΛ੾Γ෼͚Δʮࠇจࣈʯʹ༝དྷ͢ΔʢΒ͍͠ʣ
  119. Kuromoji ‣ atilika / kuromoji https://github.com/atilika/kuromoji ‣ A Japanese morphological

    analyzer written in Java ‣ Its name is derived from "ࠇจࣈ" ‣ A traditional small wooden knife to cut Japanese sweets in tea ceremony ‣ Let's create a morphological analysis API with this! ؆୯ͳܗଶૉղੳ"1*Λ࡞ͬͯΈΑ͏ʂ
  120. Kuromoji with GraalVM ‣ Kuromoji.scala object Kuromoji { implicit val

    backend = HttpURLConnectionBackend() def main(args: Array[String]): Unit = { val runtime = System.getenv("AWS_LAMBDA_RUNTIME_API") val tokenizer = new Tokenizer() while (true) { val req = sttp.get(uri"http://$runtime/2018-06-01/runtime/invocation/next").send() val requestId = req.headers("lambda-runtime-aws-request-id").head val text = Parse.parseWith(req.unsafeBody, _.field("text").flatMap(_.string).get, identity) val results = tokenizer.tokenize(text).asScala.map { token => jString(s"${token.getSurface}: ${token.getAllFeatures}") } sttp.post(uri"http://$runtime/2018-06-01/runtime/invocation/$requestId/response") .body(Json.array(results :_*).nospaces).send() } } }
  121. Kuromoji with GraalVM ‣ Parse text with Kuromoji & tokenize

    object Kuromoji { implicit val backend = HttpURLConnectionBackend() def main(args: Array[String]): Unit = { val runtime = System.getenv("AWS_LAMBDA_RUNTIME_API") val tokenizer = new Tokenizer() while (true) { val req = sttp.get(uri"http://$runtime/2018-06-01/runtime/invocation/next").send() val requestId = req.headers("lambda-runtime-aws-request-id").head val text = Parse.parseWith(req.unsafeBody, _.field("text").flatMap(_.string).get, identity) val results = tokenizer.tokenize(text).asScala.map { token => jString(s"${token.getSurface}: ${token.getAllFeatures}") } sttp.post(uri"http://$runtime/2018-06-01/runtime/invocation/$requestId/response") .body(Json.array(results :_*).nospaces).send() } } } ,VSPNPKJΛ࢖ͬͯςΩετΛύʔε͠ɺτʔΫϯԽ͢Δ
  122. Other examples in repo ‣ Takeuchi function ‣ As known

    as "Tarai-mawashi Function" ‣ High load function; when takeuchi(2n, n, 0), O(n^n) ‣ Slack notification of S3 put with Scala Native ͦͷଞͷαϯϓϧ https://github.com/todokr/scala-on-lambda-snappy
  123. Scala.js ‣ Pros ‣ Very small code size (hello.zip: 4KB)

    ‣ Integration with AWS services ‣ Cons ‣ Need to write some boilerplaty facade for type safe ‣ Need to know Node.js API, ecosystem, manners, etc... ‣ I use this for ‣ DynamoDB accessor ௕ॴίʔυαΠζͷখ͞͞ɺ"84αʔϏεͱͷ࿈ܞ ୹ॴϘΠϥʔϓϨʔτతϑΝαʔυ /PEFKTͷ஌͕ࣝඞཁ
  124. Scala Native ‣ Pros ‣ Relatively small package size (hello.zip:

    1.3M) ‣ Cons ‣ Tricky to build for Lambda ‣ Need to check if the library is supported for Scala Native ௕ॴൺֱతখ͍͞ύοέʔδαΠζ
 ୹ॴ-BNCEB޲͚ͷϏϧυʹίπ͕͍Δɺ࢖͍͍ͨϥΠϒϥϦ͕
 4DBMB/BUJWFʹରԠ͍ͯ͠Δ͔֬ೝ͢Δඞཁ͕͋Δ
  125. Native image built by GraalVM ‣ Pros ‣ Easy to

    use Scala/Java libraries ‣ Cons ‣ Relatively large package size (hello.zip: 7.3MB) ‣ Tricky to build a bit ‣ Some limitation of AOT compilation ௕ॴ4DBMB΍+BWBͷϥΠϒϥϦΛ؆୯ʹ࢖͑Δ
 ୹ॴύοέʔδ͕ͦͦ͜͜େ͖͍ Ϗϧυʹগʑίπ͕͍Δ 
 "05ίϯύΠϧͷ੍ݶ͕͋Δ
  126. My case: EC Discount Notifier cron Cloudwatch Events Crawler Price

    history Amazon etc. Notifier ࢲͷར༻έʔε&$ͷ஋Լ͛৘ใΛ௨஌
  127. My case: EC Discount Notifier cron Cloudwatch Events Crawler Price

    history Amazon etc. Notifier Ϋϩʔϥʔ͸(SBBM7.OBUJWFJNBHF
 +TPVQ +BWBϥΠϒϥϦ Λ࢖͑Δ͜ͱ΍ɺจࣈίʔυͷѻ͍Ͱ༗ར GraalVM native-image - Using Jsoup (Java library) - Charset
  128. My case: EC Discount Notifier cron Cloudwatch Events Crawler Price

    history Amazon etc. Notifier /PUJpFS͸4DBMB+4Y/PEFKTϥϯλΠϜ 4%,͕࢖͑ΔͷͰɺ%ZOBNP%#΁ͷΞΫηε͕༰қ GraalVM native-image - Using Jsoup (Java library) - Charset ScalaJS x Node.js -Accessing DynamoDB
  129. Java improvement for the container age ‣ Java continue to

    evolve! ‣ jlink (Java9) ‣ A command-line tool to create customized small JRE খ͞ͳ+3&Λ࡞ΔͨΊͷKMJOL
  130. Java improvement for the container age ‣ Java continue to

    evolve! ‣ jlink (Java9) ‣ A command-line tool to create customized small JRE ‣ Class-Data Sharing (JEP 310, 341) ‣ storing the preprocessed class metadata to file ‣ loading classes from CDS file improves startup performance ىಈΛߴ଎Խ͢Δ$MBTT%BUB4IBSJOH
  131. ‣ Run Scala on Lambda without JVM is not so

    simple 4DBMBΛ+7.ͳ͠Ͱಈ͔͢ͷ͸ͦΜͳʹγϯϓϧͰ͸ͳ͍
  132. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ͔͠͠ɺಛผͳঢ়گԼͰ͸ͱͯ΋ύϫϑϧ
  133. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ͳͷͰɺࢲͨͪ͸τϨʔυΦϑΛߟ͑Δඞཁ͕͋Δ
  134. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service 
 => Scala.js with Node.js runtime "84αʔϏεͱͷ࿈ܞ͕ඞཁ/PEFKTϥϯλΠϜͰ4DBMBKT
  135. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service 
 => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy 
 => Native Image with GraalVM +BWBͷϥΠϒϥϦΛ࢖͍ɺαΫαΫಈ͔͢ඞཁ͕͋Δ
 (SBBM7.ͰOBUJWFJNBHF
  136. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service 
 => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy 
 => Native Image with GraalVM ‣ Invoked constantly and throughput is not so important 
 => Just use Java8 runtime ఆظతͳݺͼग़͕͋͠Γɺεϧʔϓοτ͕ॏཁͰͳ͍
 ૉ௚ʹ+BWBϥϯλΠϜΛ࢖͓͏
  137. ‣ Run Scala on Lambda without JVM is not so

    simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service 
 => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy 
 => Native Image with GraalVM ‣ Invoked constantly and throughput is not so important 
 => Just use Java8 runtime ‣ _ => Give up and use other runtime such as Python or Node.js ͦΕҎ֎ͳΒɺఘΊͯ1ZUIPO΍/PEFKTͳͲଞͷϥϯλΠϜΛ࢖͓͏
  138. Thank you! Contact(via) match { case GitHub => "@todokr" case

    Twitter => "@todokr" case Facebook => "shusnuke.tadokoro" case _ => Dynamic.talk } ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ