@seratch • My name is Kaz (Kazuhiro Sera) • Has been happy with Scala for 8 years • Love open source software projects • Started ScalikeJDBC, Skinny Framework, AWScala, etc • Took over the maintenance of json4s, Scalate • Gave this talk at Scale By The Bay 2018 https://www.youtube.com/ watch?v=nN7yMChrpMM
What is ScalikeJDBC? • My most long-lasting OSS (7 year history) • A relational database client library in Scala • Provide a “Scala-like” (Scala idiomatic) way • Practical: easy-to-use APIs + less dependencies • Brief history: • Started as a simple lib similar to Twitter’s querulous • Has more functionalities now • Scala 2.10: String Interpolation, Macros • SQL objects, one-to-x extraction, type-safe DSL
Basic usage of ScalikeJDBC • Only 3 steps to issue a database query: • 1) Build a SQL object • 2) Define a function to extract values from ResultSet • 3) Perform SQL execution with an implicitly passed database session (which can be transaction-wired) 1)> val q: SQL[Nothing, NoExtractor] = sql"select id, name from members limit 2" 2)> val toTuple: (WrappedResultSet) => (Int, Option[String]) = | (rs) => (rs.get[Int]("id"), rs.get[Option[String]]("name")) 3)> val rows: Seq[(Int, Option[String])] = q.map(toTuple).list.apply() [SQL Execution] select id, name from members limit 2; (0 ms) rows: Seq[(Int, Option[String])] = List((1,Some(Alice)), (2,Some(Bob)))
Other options in the same field • According to the number of stars, • 1) Slick 2,133 • 2) Quill 1,324 • 3) Doobie 1,298 • 4) ScalikeJDBC 953 • ScalikeJDBC is ranked fourth (Not bad!) • Try in-memory database example on scalikejdbc.org • Any feedback? Reach out to me today
5 tips to build long-lasting OSS • 1) Find & focus your lifework project • 2) Be deliberate on Scala dependencies • 3) Stick with binary compatibility • 4) Provide cross builds • 5) Have effective CI builds Let’s get down to the nitty-gritty of this talk!
1) Lifework project • Not technical tip, not specific to Scala OSS • It’s all about the sustainability of voluntary works • Two things to know • Find & Focus on lifework projects • Many projects terminated due to author’s social events • e.g. Changing job, having a family, school graduation • If you really want to work on it for a long time.. • Difficult to find successors • Inconvenient truth; even if you have many users… • Ideally, continue working on your project somehow
2) Be deliberate on Scala deps • It’s unfortunate that I have to say this, but it’s critical • Two things to know • "You’re in the same boat with many others” • Major upgrade: have to wait for all releases; causing significant delay of your release • Cross builds: the greatest common divisor • Suggestion: Choose Java over Scala inside • An unpleasant decision but a realistic trade-off • Pros: Can make upgrade/cross builds much easier • Cons: Requires dirty works like handling null values
2’) Live with Java in Scala • Utilize scala.collection.JavaConveters._ • Allow easily converting Java objects to Scala objects • JavaConverters._: enrich my library pattern • JavaConversions._ (implicit conversions) are deprecated • Be always aware of null values • Like a professional Java developer • Obsessively use Option everywhere • java.util.List can be null = javaList.asScala may return null value for scala.collection.Seq • Primitive wrappers can be null + auto boxing
2’) Example Code <<< Working with java.util.List >>> scala> import scala.collection.JavaConverters._ import scala.collection.JavaConverters._ scala> val j = java.util.Arrays.asList("A", "B", "C") j: java.util.List[String] = [A, B, C] scala> val s = j.asScala s: scala.collection.mutable.Buffer[String] = Buffer(A, B, C) scala> null.asInstanceOf[java.util.List[String]].asScala // use Option! res0: scala.collection.mutable.Buffer[String] = null <<< Working with primitives >>> scala> val j: java.lang.Integer = null j: Integer = null scala> val s: Int = j // auto boxing s: Int = 0 scala> val s = Option(j).map(Int.unbox) s: Option[Int] = None java.util.List can be null. null.asScala returns null. A null value can be unexpectedly converted to the type’s default value. (0 : scala.Int’s default value)
3) Stick with bin-compatibility • Bin-compatible? • Normal situation; no runtime linkage errors • If you break bin-compatibility: • Painful runtime errors (NoSuchMethodError etc) • Scala compiler cannot detect runtime issues… • Two things to do • Have clear policies about bin-compatibility guarantee • After the first stable version release • Guaranteeing among patch versions • Use MiMa to surely guarantee binary-compatibility
3’) Use MiMa to ensure that • MiMa: Migration Manager for Scala • Detect syntactic incompatibilities • Compare the byte code from current source code with jar files in past releases • Developed and has been maintained by Lightbend // project/plugins.sbt addSbtPlugin("com.typesafe" % "sbt-mima-plugin" % "0.3.0") // build.sbt project.settings(MimaPlugin.mimaDefaultSettings ++ Seq( mimaPreviousArtifacts := { Set("0.1.0", "0.1.1").map { v => organization.value % s"${name.value}_${scalaBinaryVersion.value}" % v } }, test in Test := { mimaReportBinaryIssues.value (test in Test).value } ))
3’) Java bin-compatibility? • Are Java libraries always compatible with past releases? • No, they aren’t. Not 100% safe. • Jackson minor versions are intentionally incompatible • Also, binary compatibility can be broken by mistake • I can say it’s undoubtedly less frequent than Scala • Validating all dependency upgrades is unrealistic • It’s same for both Java and Scala • Carefully choose trusted libraries • Have unit tests to detect runtime issues
4) Provide cross builds • Cross builds = publish against multiple Scala bin versions • Why? To acquire more users in different situations • Reality: We cannot always use the latest Scala • Apache Spark took 2 years to support Scala 2.12 • Two things to do • Support latest stable major & its previous • To be specific, Scala 2.12 and 2.11 at this moment • Don’t have too many files under src/main/scala-X.Y • Having multiple implementations for same API • Testing plenty of such code is hard+time-consuming
5) Have effective CI builds • Why do we need it? • To reduce review cost of pull requests • To merge pull requests with confidence • Reliable evidences needed (regressions, bin-combat, code style, etc) • Two things to do • Enable it anyway if not yet done! • You can use TravisCI/CircleCI for free • Try automating everything apart from reviews • Check bin-compatibility, code style consistency, cross builds, multiple JDKs, etc • Configure matrix builds to cover all patterns
Start with the sbt template! • sbt project template (aka g8 template) • Including all techniques in this talk • Should be a good starting point for you mkdir my-awesome-library cd my-awesome-library sbt new seratch/long-lasting-scala.g8 sbt test
Recap • You found out about ScalikeJDBC • Contact me at any time during the conf • If you like it, increment the stars (1K soon!) • 5 tips to build long-lasting Scala OSS • 1) Find & focus your lifework project • 2) Be deliberate on Scala dependencies • 3) Stick with binary compatibility • 4) Provide cross builds • 5) Have effective CI builds
seratch
akitotsukahara
k0kubun
kdubois
heronshoes
kinocoboy2
koic
iberianpig
mame
squer
totokit4
takuyay0ne
manfredsteyer
roundedbygravity
addyosmani
phodgson
productmarketing
lara
malarkey
scottboms
orderedlist
brad_frost
sachag
dotmariusz