Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What I learned by creating 'Scala on Rails' #trbmeetup

Kazuhiro Sera
September 06, 2017

What I learned by creating 'Scala on Rails' #trbmeetup

Kazuhiro Sera

September 06, 2017
Tweet

More Decks by Kazuhiro Sera

Other Decks in Programming

Transcript

  1. What I learned
    by creating 'Scala on Rails'
    Tokyo Rubyist Meetup #trbmeetup
    Kazuhiro Sera @seratch

    View Slide

  2. RubyKaigi - Not Accepted ;-(

    View Slide

  3. Thanks, @pwim :-)

    View Slide

  4. Who Am I

    View Slide

  5. Kazuhiro Sera @seratch
    - Backend Engineer on Web services
    - From Hiroshima, living in Tokyo for 16+ years
    - A Scala enthusiast since 2011
    - OSS: ScalikeJDBC, Skinny, Scalatra, json4s, etc
    - Organizer of Scala meetups (inactive now)
    - Software Engineer / Engineering Manager at SmartNews (2016 - )
    - Java/Kotlin/Scala for back-end APIs
    - Software Engineer at M3 (2009 - 2016)
    - Java/Scala for back-end APIs
    - Ruby on Rails for front-end web app

    View Slide

  6. Things to know about Scala

    View Slide

  7. Key Concepts
    - Scala official website
    - Reactive Manifesto
    - Lightbend Reactive Platform Overview

    View Slide

  8. History of Scala
    - Born in 2004
    - Creator: Prof. Martin Odersky from EPFL
    - EPFL: École Polytechnique Fédérale de Lausanne
    - Java generics designer
    - 2006/03: Scala 2.0 (practically design completed)
    - 2011/05: Lightbend, Inc. (formerly Typesafe, Inc.)
    - 2012/12: Scala 2.10
    - Future, macros, string interpolation, 2.x bin-compatibility
    - 2016/10: Scala 2.12
    - (Someday in the future) Scala 3 with Dotty (dotty.epfl.ch)?

    View Slide

  9. Who uses Scala?
    - Twitter (Twitter OSS List, Finagle in the lead)
    - Surprised the industry by “Twitter on Scala (2009)”
    - LinkedIn (Lightbend case study)
    - Tumblr (Colossus, Talk at Scala by the Bay 2016)
    - Netflix (Netfix Scala OSS List)
    - Credit Karma (Lightbend case study)
    - The Guardian (Lightbend case study)
    - What startups or tech companies are using Scala? (Quora)

    View Slide

  10. Scala OSS Chronology
    - 2009: Akka, Lift
    - 2010: Finagle
    - 2011: Typesafe, Inc.
    - 2012: Scala 2.10, Play 2.0
    - 2013: Reactive Manifesto
    - 2014: Apache Spark 1.0
    - 2015: akka-http
    - 2016: Lightbend, Inc.
    - 2017: sbt 1.0

    View Slide

  11. Akka, Play from Lightbend (Typsafe)
    - Akka (akka.io)
    - An implementation of actor model on the JVM
    - akka-streams: Reactive Streams implementation built upon Akka
    - akka-http: Full server/client HTTP stack
    - Play Framework (playframework.com)
    - Play 1: a “Java on Rails” (built in Java)
    - Play 2: Framework to build non-blocking HTTP services
    - Built upon akka-http (formerly, Netty)

    View Slide

  12. Finagle from Twitter
    - Pioneer OSS in the Scala world
    - Finagle: A Protocol-Agnostic RPC System
    - “an extensible RPC system for the JVM”
    - Standardized the concept of Future API before others
    - HTTP/Thrift server toolkit
    - The basis of huge backend at Twitter
    - Twitter Server
    - Finatra
    - Finch

    View Slide

  13. Scala for Big Data: Apache Spark, Flink
    - Apache Spark (spark.apache.org)
    - RDD, SQL, GraphX, MLlib, Streaming
    - Amazon EMR
    - Databricks
    - IBM Spark Technology Center
    - Apache Flink (flink.apache.org)
    - Streaming-first, Continuous processing
    - In particular, becoming popular in Europe

    View Slide

  14. In 2014, Play was already famous.
    Why did I create yet another one?

    View Slide

  15. What I like about Scala
    - Scala’s good points
    - Type-safety (statically-typed)
    - Expressiveness (object-oriented + functional)
    - Concurrency support (Future API)
    - Akka’s Actor Model
    - Finagle from Twitter
    - In particular, I like ...
    - Type-safety (statically-typed)
    - Expressiveness (object-oriented + functional)

    View Slide

  16. Type-safety + expressiveness for...
    - Solving common business problems
    - Building and operating small/medium scale web apps
    - Non-blocking is not required
    - Dealing with real world data / legacy APIs
    - Maintainability for long living systems
    - Compile time checks
    - Making code well-documented with static types
    - Keeping it easy-to-understand for anybody
    - Never enforce users efforts when upgrading

    View Slide

  17. What I started
    - ScalikeJDBC
    - A tidy and type-safe SQL database library
    - JDBC assets: MySQL, PostgreSQL, Redshift, Presto, etc
    - Completely free from SQL injection vulnerability
    - 796 GitHub stargazers (2011/11 - 2017/9)
    - Skinny Framework
    - A full-stack Web framework - “Scala on Rails”
    - Intentionally, being a Rails follower to be friendly
    - Scaffolding, ORM, DB migration, Mail, OAuth, Testing, etc.
    - 639 GitHub stargazers (2013/9 - 2017/9)

    View Slide

  18. The Overview of
    Skinny Framework

    View Slide

  19. Named after a talk at RubyKaigi
    “Refactoring Fat Models with Patterns” by Bryan Helmkamp at RubyKaigi 2013
    https://vimeo.com/68611168

    View Slide

  20. Full-stack Web Framework
    - Scaffolding gerator - generates CRUD endpoints/web pages
    - Validation rule DSL - easy-to-understand & extendable
    - O/R mapper - built upon ScalikeJDBC
    - Database migration - using Flyway
    - Template engine -Scalate by default
    - Mail module - built upon JavaMail
    - OAuth integration (Facebook, Google, Twitter, GitHub, etc)
    - Unit testing support - with embedded Jetty HTTP server

    View Slide

  21. Portable Modules

    View Slide

  22. Getting Started on macOS
    # Setup global “skinny” script via Homebrew
    brew install skinny
    # Create a blank project
    skinny new todolist
    cd ./todolist
    # Generate CRUD scaffolding
    skinny g scaffold tasks task \
    title:String description:Option[String] dueDate:LocalDate
    skinny db:migrate
    skinny run
    open localhost:8080/tasks

    View Slide

  23. Auto-generated CRUD pages

    View Slide

  24. Auto-generated JSON/XML endpoints
    http://localhost:8080/tasks.json http://localhost:8080/tasks.xml

    View Slide

  25. Controller
    class TasksController extends ApplicationController {
    protectFromForgery()
    beforeAction() {
    // do something for all actions
    }
    def showAll = {
    set(“item”, Task.findAll()) // same as set value to instance fields in Rails
    render(“/tasks/index”)
    }
    }

    View Slide

  26. Validation Rules
    def creationForm = validation(createParams,
    paramKey(“title”) is required & maxLength(20),
    parmaKey(“velocity_point”) is numeric,
    paramKey(“due_date”) is required & dateFormat
    )
    object numeric extends ValidationRule {
    def name = “numeric”
    def isValid(v: Any) = isEmpty(v) || isNumeric(v)
    }

    View Slide

  27. Action Method in Controller
    def create() = {
    if (form.validate()) {
    val paramsToSave = params.permit(strongParams: _*)
    val id = Task.createWithStrongParameters(paramsToSave)
    redirect(s“/tasks/${id}”)
    } else {
    status = 400
    render(“/tasks/new”) // Error messages are already bound in the view
    }
    }

    View Slide

  28. Entity data + Data Access Object
    // Entity data class
    case class Task(id: Long, title: String, dueDate: Option[LocalDate])
    // Data access object (≒ Rails ActiveRecord model)
    object Task extends SkinnyCRUDMapper[Task] {
    override lazy val tableName = “tasks”
    override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn)
    }
    Task.findAll()
    Task.where(‘title -> “Think different”).limit(1).offset(0).apply()
    Task.where(sqls.isNotNull(t.dueDate)).count()

    View Slide

  29. ORM - Associations
    case class Task(
    id: Long,
    title: String,
    assigneeId: Long,
    assignee: Option[Assignee] = None)
    object Task extends SkinnyCRUDMapper[Task] {
    val assigneeRef = belongsTo[Assignee](Assignee, (t, a) => t.copy(assignee = a))
    // ... other lines omitted
    }
    Task.joins(Task.assigneeRef).findAll() // join query, no N+1

    View Slide

  30. ORM - Eager Loading
    case class Task(
    id: Long,
    title: String,
    assigneeId: Long,
    assignee: Option[Assignee] = None)
    object Task extends SkinnyCRUDMapper[Task] {
    val assigneeRef = belongsTo[Assignee](Assignee, ...).includes[Assignee](merge = … )
    // ... other lines omitted
    }
    Task.includes(Task.assigneeRef).findAll() // 2 qureies, no N+1

    View Slide

  31. What I learned
    by creating ‘Scala on Rails’

    View Slide

  32. Findings while building a “Rails”
    - (1) Fun to follow the idiomatic way
    - Rails is a collection of idioms in Web development
    - No need to worry about the design of basic concepts
    - (2) Some people don’t prefer the Rails way like “Omakase”
    - Allowing work around for corner cases
    - Difficult to change people’s preferences
    - (3) Decided to have several differences from Ruby on Rails
    - Don’t use instance fields to pass values to view templates
    - Separated validations from models
    - Don’t support routes.rb (mainly due to maintenance costs)

    View Slide

  33. ORM with immutability + static types
    - (1) Difficulty with dynamic resolutions of associations
    - Good Point: By design, it’s hard to issue N+1 queries
    - Bad Point: Not so handy for quick-prototyping
    - (2) Immutable data structure vs Datastore in the wild
    - Rows in database table are mutable
    - auto_generated PK field
    - Long or Option[Long] (= can be null/nil)
    - Decided not to support insertion with an entity (so far)

    View Slide

  34. (1) N+1 never happens without intention
    case class Assignee(id: Long, name: String)
    case class Task(id: Long, assigneeId: Long) {
    lazy val assignee: Option[Assignee] = Assignee.findById(assigneeId)
    }
    Task.findAll().foreach { task =>
    task.assignee // issue a query to fetch an assignee entity
    }

    View Slide

  35. (2) Insertion
    // Entity data class
    case class Task(id: Long, title: String, dueDate: Option[LocalDate])
    // Data access object (≒ Rails ActiveRecord model)
    object Task extends SkinnyCRUDMapper[Task] {
    override lazy val tableName = “tasks”
    override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn)
    }
    Task.createWithAttributes(‘title -> “Preparing for a talk”)

    View Slide

  36. Catching the wave
    - The moving trends
    - 2011: Type-safety (+ Actor model)
    - 2017: Reactive
    - Turning points in the history
    - 2012/12: Scala 2.10.0 with Future APIs
    - 2013/8: I started the Skinny project
    - 2014/3: Skinny 1.0.0
    - 2016/3: Typesafe was renamed to Lightbend
    - Symbolic decision: Reactiveness > Type-safety

    View Slide

  37. The Future of Skinny

    View Slide

  38. The beauty of backward compatibilities
    - No breaking changes
    - No extra costs without benefits
    - Except for compatibility with Rails API (renaming, etc)
    - Servlet, JDBC have never introduced breaking changes
    - Placing an importance on the trust
    - The trust should be a unique value in Scala
    - To be chosen again for next project at ease

    View Slide

  39. Keep ORM independent from Web
    - “We don’t use Servlet but need a good ORM!”
    - Steep learning curve for other libraries
    - Skinny ORM is widely accepted in non-Skinny projects
    - Keep being portable and flexible
    - So few dependencies (a JDBC driver, slf4j-api, joda-time)
    - No magic, simply built upon JDBC drivers

    View Slide

  40. Conclusion

    View Slide

  41. Conclusion
    - Scala = Not only reactive but type-safety + expressiveness
    - Rails = a collection of idioms in Web app development
    - Making a yet another Rails = a lot of fun!
    - “Skinny” was named after a RubyKaigi talk
    - Be aware of the moving trends

    View Slide