What I learned by creating 'Scala on Rails' #trbmeetup

https://trbmeetup.doorkeeper.jp/events/63939

Transcript

1. What I learned
   by creating 'Scala on Rails'
   Tokyo Rubyist Meetup #trbmeetup
   Kazuhiro Sera @seratch
   

   View Slide

2. RubyKaigi - Not Accepted ;-(
   

   View Slide

3. Thanks, @pwim :-)
   

   View Slide

4. Who Am I
   

   View Slide

5. Kazuhiro Sera @seratch
   - Backend Engineer on Web services
   - From Hiroshima, living in Tokyo for 16+ years
   - A Scala enthusiast since 2011
   - OSS: ScalikeJDBC, Skinny, Scalatra, json4s, etc
   - Organizer of Scala meetups (inactive now)
   - Software Engineer / Engineering Manager at SmartNews (2016 - )
   - Java/Kotlin/Scala for back-end APIs
   - Software Engineer at M3 (2009 - 2016)
   - Java/Scala for back-end APIs
   - Ruby on Rails for front-end web app
   

   View Slide

6. Things to know about Scala
   

   View Slide

7. Key Concepts
   - Scala official website
   - Reactive Manifesto
   - Lightbend Reactive Platform Overview
   

   View Slide

8. History of Scala
   - Born in 2004
   - Creator: Prof. Martin Odersky from EPFL
   - EPFL: École Polytechnique Fédérale de Lausanne
   - Java generics designer
   - 2006/03: Scala 2.0 (practically design completed)
   - 2011/05: Lightbend, Inc. (formerly Typesafe, Inc.)
   - 2012/12: Scala 2.10
   - Future, macros, string interpolation, 2.x bin-compatibility
   - 2016/10: Scala 2.12
   - (Someday in the future) Scala 3 with Dotty (dotty.epfl.ch)?
   

   View Slide

9. Who uses Scala?
   - Twitter (Twitter OSS List, Finagle in the lead)
   - Surprised the industry by “Twitter on Scala (2009)”
   - LinkedIn (Lightbend case study)
   - Tumblr (Colossus, Talk at Scala by the Bay 2016)
   - Netflix (Netfix Scala OSS List)
   - Credit Karma (Lightbend case study)
   - The Guardian (Lightbend case study)
   - What startups or tech companies are using Scala? (Quora)
   

   View Slide

10. Scala OSS Chronology
    - 2009: Akka, Lift
    - 2010: Finagle
    - 2011: Typesafe, Inc.
    - 2012: Scala 2.10, Play 2.0
    - 2013: Reactive Manifesto
    - 2014: Apache Spark 1.0
    - 2015: akka-http
    - 2016: Lightbend, Inc.
    - 2017: sbt 1.0
    

    View Slide

11. Akka, Play from Lightbend (Typsafe)
    - Akka (akka.io)
    - An implementation of actor model on the JVM
    - akka-streams: Reactive Streams implementation built upon Akka
    - akka-http: Full server/client HTTP stack
    - Play Framework (playframework.com)
    - Play 1: a “Java on Rails” (built in Java)
    - Play 2: Framework to build non-blocking HTTP services
    - Built upon akka-http (formerly, Netty)
    

    View Slide

12. Finagle from Twitter
    - Pioneer OSS in the Scala world
    - Finagle: A Protocol-Agnostic RPC System
    - “an extensible RPC system for the JVM”
    - Standardized the concept of Future API before others
    - HTTP/Thrift server toolkit
    - The basis of huge backend at Twitter
    - Twitter Server
    - Finatra
    - Finch
    

    View Slide

13. Scala for Big Data: Apache Spark, Flink
    - Apache Spark (spark.apache.org)
    - RDD, SQL, GraphX, MLlib, Streaming
    - Amazon EMR
    - Databricks
    - IBM Spark Technology Center
    - Apache Flink (flink.apache.org)
    - Streaming-first, Continuous processing
    - In particular, becoming popular in Europe
    

    View Slide

14. In 2014, Play was already famous.
    Why did I create yet another one?
    

    View Slide

15. What I like about Scala
    - Scala’s good points
    - Type-safety (statically-typed)
    - Expressiveness (object-oriented + functional)
    - Concurrency support (Future API)
    - Akka’s Actor Model
    - Finagle from Twitter
    - In particular, I like ...
    - Type-safety (statically-typed)
    - Expressiveness (object-oriented + functional)
    

    View Slide

16. Type-safety + expressiveness for...
    - Solving common business problems
    - Building and operating small/medium scale web apps
    - Non-blocking is not required
    - Dealing with real world data / legacy APIs
    - Maintainability for long living systems
    - Compile time checks
    - Making code well-documented with static types
    - Keeping it easy-to-understand for anybody
    - Never enforce users efforts when upgrading
    

    View Slide

17. What I started
    - ScalikeJDBC
    - A tidy and type-safe SQL database library
    - JDBC assets: MySQL, PostgreSQL, Redshift, Presto, etc
    - Completely free from SQL injection vulnerability
    - 796 GitHub stargazers (2011/11 - 2017/9)
    - Skinny Framework
    - A full-stack Web framework - “Scala on Rails”
    - Intentionally, being a Rails follower to be friendly
    - Scaffolding, ORM, DB migration, Mail, OAuth, Testing, etc.
    - 639 GitHub stargazers (2013/9 - 2017/9)
    

    View Slide

18. The Overview of
    Skinny Framework
    

    View Slide

19. Named after a talk at RubyKaigi
    “Refactoring Fat Models with Patterns” by Bryan Helmkamp at RubyKaigi 2013
    https://vimeo.com/68611168
    

    View Slide

20. Full-stack Web Framework
    - Scaffolding gerator - generates CRUD endpoints/web pages
    - Validation rule DSL - easy-to-understand & extendable
    - O/R mapper - built upon ScalikeJDBC
    - Database migration - using Flyway
    - Template engine -Scalate by default
    - Mail module - built upon JavaMail
    - OAuth integration (Facebook, Google, Twitter, GitHub, etc)
    - Unit testing support - with embedded Jetty HTTP server
    

    View Slide

21. Portable Modules
    

    View Slide

22. Getting Started on macOS
    # Setup global “skinny” script via Homebrew
    brew install skinny
    # Create a blank project
    skinny new todolist
    cd ./todolist
    # Generate CRUD scaffolding
    skinny g scaffold tasks task \
    title:String description:Option[String] dueDate:LocalDate
    skinny db:migrate
    skinny run
    open localhost:8080/tasks
    

    View Slide

23. Auto-generated CRUD pages
    

    View Slide

24. Auto-generated JSON/XML endpoints
    http://localhost:8080/tasks.json http://localhost:8080/tasks.xml
    

    View Slide

25. Controller
    class TasksController extends ApplicationController {
    protectFromForgery()
    beforeAction() {
    // do something for all actions
    }
    def showAll = {
    set(“item”, Task.findAll()) // same as set value to instance fields in Rails
    render(“/tasks/index”)
    }
    }
    

    View Slide

26. Validation Rules
    def creationForm = validation(createParams,
    paramKey(“title”) is required & maxLength(20),
    parmaKey(“velocity_point”) is numeric,
    paramKey(“due_date”) is required & dateFormat
    )
    object numeric extends ValidationRule {
    def name = “numeric”
    def isValid(v: Any) = isEmpty(v) || isNumeric(v)
    }
    

    View Slide

27. Action Method in Controller
    def create() = {
    if (form.validate()) {
    val paramsToSave = params.permit(strongParams: _*)
    val id = Task.createWithStrongParameters(paramsToSave)
    redirect(s“/tasks/${id}”)
    } else {
    status = 400
    render(“/tasks/new”) // Error messages are already bound in the view
    }
    }
    

    View Slide

28. Entity data + Data Access Object
    // Entity data class
    case class Task(id: Long, title: String, dueDate: Option[LocalDate])
    // Data access object (≒ Rails ActiveRecord model)
    object Task extends SkinnyCRUDMapper[Task] {
    override lazy val tableName = “tasks”
    override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn)
    }
    Task.findAll()
    Task.where(‘title -> “Think different”).limit(1).offset(0).apply()
    Task.where(sqls.isNotNull(t.dueDate)).count()
    

    View Slide

29. ORM - Associations
    case class Task(
    id: Long,
    title: String,
    assigneeId: Long,
    assignee: Option[Assignee] = None)
    object Task extends SkinnyCRUDMapper[Task] {
    val assigneeRef = belongsTo[Assignee](Assignee, (t, a) => t.copy(assignee = a))
    // ... other lines omitted
    }
    Task.joins(Task.assigneeRef).findAll() // join query, no N+1
    

    View Slide

30. ORM - Eager Loading
    case class Task(
    id: Long,
    title: String,
    assigneeId: Long,
    assignee: Option[Assignee] = None)
    object Task extends SkinnyCRUDMapper[Task] {
    val assigneeRef = belongsTo[Assignee](Assignee, ...).includes[Assignee](merge = … )
    // ... other lines omitted
    }
    Task.includes(Task.assigneeRef).findAll() // 2 qureies, no N+1
    

    View Slide

31. What I learned
    by creating ‘Scala on Rails’
    

    View Slide

32. Findings while building a “Rails”
    - (1) Fun to follow the idiomatic way
    - Rails is a collection of idioms in Web development
    - No need to worry about the design of basic concepts
    - (2) Some people don’t prefer the Rails way like “Omakase”
    - Allowing work around for corner cases
    - Difficult to change people’s preferences
    - (3) Decided to have several differences from Ruby on Rails
    - Don’t use instance fields to pass values to view templates
    - Separated validations from models
    - Don’t support routes.rb (mainly due to maintenance costs)
    

    View Slide

33. ORM with immutability + static types
    - (1) Difficulty with dynamic resolutions of associations
    - Good Point: By design, it’s hard to issue N+1 queries
    - Bad Point: Not so handy for quick-prototyping
    - (2) Immutable data structure vs Datastore in the wild
    - Rows in database table are mutable
    - auto_generated PK field
    - Long or Option[Long] (= can be null/nil)
    - Decided not to support insertion with an entity (so far)
    

    View Slide

34. (1) N+1 never happens without intention
    case class Assignee(id: Long, name: String)
    case class Task(id: Long, assigneeId: Long) {
    lazy val assignee: Option[Assignee] = Assignee.findById(assigneeId)
    }
    Task.findAll().foreach { task =>
    task.assignee // issue a query to fetch an assignee entity
    }
    

    View Slide

35. (2) Insertion
    // Entity data class
    case class Task(id: Long, title: String, dueDate: Option[LocalDate])
    // Data access object (≒ Rails ActiveRecord model)
    object Task extends SkinnyCRUDMapper[Task] {
    override lazy val tableName = “tasks”
    override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn)
    }
    Task.createWithAttributes(‘title -> “Preparing for a talk”)
    

    View Slide

36. Catching the wave
    - The moving trends
    - 2011: Type-safety (+ Actor model)
    - 2017: Reactive
    - Turning points in the history
    - 2012/12: Scala 2.10.0 with Future APIs
    - 2013/8: I started the Skinny project
    - 2014/3: Skinny 1.0.0
    - 2016/3: Typesafe was renamed to Lightbend
    - Symbolic decision: Reactiveness > Type-safety
    

    View Slide

37. The Future of Skinny
    

    View Slide

38. The beauty of backward compatibilities
    - No breaking changes
    - No extra costs without benefits
    - Except for compatibility with Rails API (renaming, etc)
    - Servlet, JDBC have never introduced breaking changes
    - Placing an importance on the trust
    - The trust should be a unique value in Scala
    - To be chosen again for next project at ease
    

    View Slide

39. Keep ORM independent from Web
    - “We don’t use Servlet but need a good ORM!”
    - Steep learning curve for other libraries
    - Skinny ORM is widely accepted in non-Skinny projects
    - Keep being portable and flexible
    - So few dependencies (a JDBC driver, slf4j-api, joda-time)
    - No magic, simply built upon JDBC drivers
    

    View Slide

40. Conclusion
    

    View Slide

41. Conclusion
    - Scala = Not only reactive but type-safety + expressiveness
    - Rails = a collection of idioms in Web app development
    - Making a yet another Rails = a lot of fun!
    - “Skinny” was named after a RubyKaigi talk
    - Be aware of the moving trends
    

    View Slide