What I learned by creating 'Scala on Rails' #trbmeetup

https://trbmeetup.doorkeeper.jp/events/63939

Transcript

1. What I learned by creating 'Scala on Rails' Tokyo Rubyist Meetup #trbmeetup Kazuhiro Sera @seratch
2. RubyKaigi - Not Accepted ;-(
3. Thanks, @pwim :-)
4. Who Am I
5. Kazuhiro Sera @seratch - Backend Engineer on Web services - From Hiroshima, living in Tokyo for 16+ years - A Scala enthusiast since 2011 - OSS: ScalikeJDBC, Skinny, Scalatra, json4s, etc - Organizer of Scala meetups (inactive now) - Software Engineer / Engineering Manager at SmartNews (2016 - ) - Java/Kotlin/Scala for back-end APIs - Software Engineer at M3 (2009 - 2016) - Java/Scala for back-end APIs - Ruby on Rails for front-end web app
6. Things to know about Scala
7. Key Concepts - Scala official website - Reactive Manifesto - Lightbend Reactive Platform Overview
8. History of Scala - Born in 2004 - Creator: Prof. Martin Odersky from EPFL - EPFL: École Polytechnique Fédérale de Lausanne - Java generics designer - 2006/03: Scala 2.0 (practically design completed) - 2011/05: Lightbend, Inc. (formerly Typesafe, Inc.) - 2012/12: Scala 2.10 - Future, macros, string interpolation, 2.x bin-compatibility - 2016/10: Scala 2.12 - (Someday in the future) Scala 3 with Dotty (dotty.epfl.ch)?
9. Who uses Scala? - Twitter (Twitter OSS List, Finagle in the lead) - Surprised the industry by “Twitter on Scala (2009)” - LinkedIn (Lightbend case study) - Tumblr (Colossus, Talk at Scala by the Bay 2016) - Netflix (Netfix Scala OSS List) - Credit Karma (Lightbend case study) - The Guardian (Lightbend case study) - What startups or tech companies are using Scala? (Quora)
10. Scala OSS Chronology - 2009: Akka, Lift - 2010: Finagle - 2011: Typesafe, Inc. - 2012: Scala 2.10, Play 2.0 - 2013: Reactive Manifesto - 2014: Apache Spark 1.0 - 2015: akka-http - 2016: Lightbend, Inc. - 2017: sbt 1.0
11. Akka, Play from Lightbend (Typsafe) - Akka (akka.io) - An implementation of actor model on the JVM - akka-streams: Reactive Streams implementation built upon Akka - akka-http: Full server/client HTTP stack - Play Framework (playframework.com) - Play 1: a “Java on Rails” (built in Java) - Play 2: Framework to build non-blocking HTTP services - Built upon akka-http (formerly, Netty)
12. Finagle from Twitter - Pioneer OSS in the Scala world - Finagle: A Protocol-Agnostic RPC System - “an extensible RPC system for the JVM” - Standardized the concept of Future API before others - HTTP/Thrift server toolkit - The basis of huge backend at Twitter - Twitter Server - Finatra - Finch
13. Scala for Big Data: Apache Spark, Flink - Apache Spark (spark.apache.org) - RDD, SQL, GraphX, MLlib, Streaming - Amazon EMR - Databricks - IBM Spark Technology Center - Apache Flink (flink.apache.org) - Streaming-first, Continuous processing - In particular, becoming popular in Europe
14. In 2014, Play was already famous. Why did I create yet another one?
15. What I like about Scala - Scala’s good points - Type-safety (statically-typed) - Expressiveness (object-oriented + functional) - Concurrency support (Future API) - Akka’s Actor Model - Finagle from Twitter - In particular, I like ... - Type-safety (statically-typed) - Expressiveness (object-oriented + functional)
16. Type-safety + expressiveness for... - Solving common business problems - Building and operating small/medium scale web apps - Non-blocking is not required - Dealing with real world data / legacy APIs - Maintainability for long living systems - Compile time checks - Making code well-documented with static types - Keeping it easy-to-understand for anybody - Never enforce users efforts when upgrading
17. What I started - ScalikeJDBC - A tidy and type-safe SQL database library - JDBC assets: MySQL, PostgreSQL, Redshift, Presto, etc - Completely free from SQL injection vulnerability - 796 GitHub stargazers (2011/11 - 2017/9) - Skinny Framework - A full-stack Web framework - “Scala on Rails” - Intentionally, being a Rails follower to be friendly - Scaffolding, ORM, DB migration, Mail, OAuth, Testing, etc. - 639 GitHub stargazers (2013/9 - 2017/9)
18. The Overview of Skinny Framework
19. Named after a talk at RubyKaigi “Refactoring Fat Models with Patterns” by Bryan Helmkamp at RubyKaigi 2013 https://vimeo.com/68611168
20. Full-stack Web Framework - Scaffolding gerator - generates CRUD endpoints/web pages - Validation rule DSL - easy-to-understand & extendable - O/R mapper - built upon ScalikeJDBC - Database migration - using Flyway - Template engine -Scalate by default - Mail module - built upon JavaMail - OAuth integration (Facebook, Google, Twitter, GitHub, etc) - Unit testing support - with embedded Jetty HTTP server
21. Portable Modules
22. Getting Started on macOS # Setup global “skinny” script via Homebrew brew install skinny # Create a blank project skinny new todolist cd ./todolist # Generate CRUD scaffolding skinny g scaffold tasks task \ title:String description:Option[String] dueDate:LocalDate skinny db:migrate skinny run open localhost:8080/tasks
23. Auto-generated CRUD pages
24. Auto-generated JSON/XML endpoints http://localhost:8080/tasks.json http://localhost:8080/tasks.xml
25. Controller class TasksController extends ApplicationController { protectFromForgery() beforeAction() { // do something for all actions } def showAll = { set(“item”, Task.findAll()) // same as set value to instance fields in Rails render(“/tasks/index”) } }
26. Validation Rules def creationForm = validation(createParams, paramKey(“title”) is required & maxLength(20), parmaKey(“velocity_point”) is numeric, paramKey(“due_date”) is required & dateFormat ) object numeric extends ValidationRule { def name = “numeric” def isValid(v: Any) = isEmpty(v) || isNumeric(v) }
27. Action Method in Controller def create() = { if (form.validate()) { val paramsToSave = params.permit(strongParams: _*) val id = Task.createWithStrongParameters(paramsToSave) redirect(s“/tasks/${id}”) } else { status = 400 render(“/tasks/new”) // Error messages are already bound in the view } }
28. Entity data + Data Access Object // Entity data class case class Task(id: Long, title: String, dueDate: Option[LocalDate]) // Data access object (≒ Rails ActiveRecord model) object Task extends SkinnyCRUDMapper[Task] { override lazy val tableName = “tasks” override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn) } Task.findAll() Task.where(‘title -> “Think different”).limit(1).offset(0).apply() Task.where(sqls.isNotNull(t.dueDate)).count()
29. ORM - Associations case class Task( id: Long, title: String, assigneeId: Long, assignee: Option[Assignee] = None) object Task extends SkinnyCRUDMapper[Task] { val assigneeRef = belongsTo[Assignee](Assignee, (t, a) => t.copy(assignee = a)) // ... other lines omitted } Task.joins(Task.assigneeRef).findAll() // join query, no N+1
30. ORM - Eager Loading case class Task( id: Long, title: String, assigneeId: Long, assignee: Option[Assignee] = None) object Task extends SkinnyCRUDMapper[Task] { val assigneeRef = belongsTo[Assignee](Assignee, ...).includes[Assignee](merge = … ) // ... other lines omitted } Task.includes(Task.assigneeRef).findAll() // 2 qureies, no N+1
31. What I learned by creating ‘Scala on Rails’
32. Findings while building a “Rails” - (1) Fun to follow the idiomatic way - Rails is a collection of idioms in Web development - No need to worry about the design of basic concepts - (2) Some people don’t prefer the Rails way like “Omakase” - Allowing work around for corner cases - Difficult to change people’s preferences - (3) Decided to have several differences from Ruby on Rails - Don’t use instance fields to pass values to view templates - Separated validations from models - Don’t support routes.rb (mainly due to maintenance costs)
33. ORM with immutability + static types - (1) Difficulty with dynamic resolutions of associations - Good Point: By design, it’s hard to issue N+1 queries - Bad Point: Not so handy for quick-prototyping - (2) Immutable data structure vs Datastore in the wild - Rows in database table are mutable - auto_generated PK field - Long or Option[Long] (= can be null/nil) - Decided not to support insertion with an entity (so far)
34. (1) N+1 never happens without intention case class Assignee(id: Long, name: String) case class Task(id: Long, assigneeId: Long) { lazy val assignee: Option[Assignee] = Assignee.findById(assigneeId) } Task.findAll().foreach { task => task.assignee // issue a query to fetch an assignee entity }
35. (2) Insertion // Entity data class case class Task(id: Long, title: String, dueDate: Option[LocalDate]) // Data access object (≒ Rails ActiveRecord model) object Task extends SkinnyCRUDMapper[Task] { override lazy val tableName = “tasks” override def extract(rs: WrappedResultSet, rn: ResultName[Task]) = autoConstruct(rs, rn) } Task.createWithAttributes(‘title -> “Preparing for a talk”)
36. Catching the wave - The moving trends - 2011: Type-safety (+ Actor model) - 2017: Reactive - Turning points in the history - 2012/12: Scala 2.10.0 with Future APIs - 2013/8: I started the Skinny project - 2014/3: Skinny 1.0.0 - 2016/3: Typesafe was renamed to Lightbend - Symbolic decision: Reactiveness > Type-safety
37. The Future of Skinny
38. The beauty of backward compatibilities - No breaking changes - No extra costs without benefits - Except for compatibility with Rails API (renaming, etc) - Servlet, JDBC have never introduced breaking changes - Placing an importance on the trust - The trust should be a unique value in Scala - To be chosen again for next project at ease
39. Keep ORM independent from Web - “We don’t use Servlet but need a good ORM!” - Steep learning curve for other libraries - Skinny ORM is widely accepted in non-Skinny projects - Keep being portable and flexible - So few dependencies (a JDBC driver, slf4j-api, joda-time) - No magic, simply built upon JDBC drivers
40. Conclusion
41. Conclusion - Scala = Not only reactive but type-safety + expressiveness - Rails = a collection of idioms in Web app development - Making a yet another Rails = a lot of fun! - “Skinny” was named after a RubyKaigi talk - Be aware of the moving trends