Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Taro L. Saito, Ph.D. GitHub: @xerial Arm Treasure Data Airframe: Lightweight Building Blocks for Scala November 16th, 2018 Scale By The Bay 2018 - Unconference 1
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. About Me: Taro L. Saito (Leo) ● An Engineer with Research Background ● Ph.D., University of Tokyo ● DBMS & Genome Science ● Leading Query Engine Team ● Active OSS Developer ● airframe ● sqlite-jdbc ■ More than 1000 GitHub stars ● snappy-java ■ Compression library used in Spark, Parquet ● sbt-sonatype ■ Used in 2000+ Scala projects ● ... 2
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe ● Lightweight Building Blocks for Scala ● Essential libraries for building any applications ● Used in production for 2+ years ● Based on my code collection since 2009 ● Initially written in Java ● Gradually migrated to Scala ● Repackaged into wvlet.airframe in 2016 ● For maintainability ● 18 Modules ● Simplifying your daily programming in Scala 3
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Debugging Applications: Airframe Log ● Airframe Log: A Modern Logging Library for Scala (Medium Blog) ● ANSI color, source code location support 6
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Metrics ● Human Readable Data Format (Duration, DataSize, etc.) ● Handy Time Window String Support 7 “-1d” “-1w” “-7d”
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe JMX ● Checking the internal states of remote JVM processes ● JMX clients ● jconsole has JMX metric monitor ● Airframe JMX -> DataDog -> Dashboard 8
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Today’s Goals ● Lean How to Use Airframe DI (Dependency Injection) ● Understand what can be simplified with DI ● Learn 5 Airframe DI Design Patterns ● That improve the thought processes in programming 11
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. What is Dependency Injection (DI)? ● Many Articles… ● Inversion of Control Containers and the Dependency Injection pattern. Martin Fowler (2004) ● StackOverflow, Wikipedia, … ● Many Frameworks... ● Spring, Google Guice, Scaldi, Macwire, Grafter, Weld, etc. ● No framework approaches also exist (Pure-Scala DI) ● Recent Definition: ● Dependency Injection is the process of creating the static, stateless graph of service objects, where each service is parameterised by its dependencies. ■ What is Dependency Injection? by Adam Warski (2018) ● However, it’s still difficult to understand what is DI 12
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Simplifying DI with Airframe ● Airframe Usage ● import wvlet.airframe._ ● Simple 3 Step DI ● bind ● design ● build 13
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Design Patterns In Airframe ● Pattern 0 (basic): Building Service Objects with Auto-Wiring ● Pattern 1: Configuring Modules ● Pattern 2: Switching Bindings for Tests ● Pattern 3: Managing Lifecycle in FILO order ● Pattern 4: Bundling Service Traits ● Flower-bundle pattern ● Pattern 5: Binding Factory 14
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Constructor vs In-Trait Injections ● Constructor Injection ● In-Trait Injection 15
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Building Service Objects ● When coding A and B ● You can focus on only direct dependencies ● You can forget about indirect dependencies ● Airframe DI builds A, B, and direct/indirect dependencies on your behalf. A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 16 You can forget this part
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Configuring Modules ● How to pass configuration objects to corresponding modules? ● new A(new DBClient(new ConnectionPool(new DB(new DBConfig(...)), new ConnectionPoolConfig(...), new DBClientConfig(...))) ● Things You Need to Remember ● Argument orders (or argument names in Scala) of individual modules ● How to instantiate modules A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 19 DB Config Connection Pool Config HttpClient Config
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 2: Switching Bindings For Testing ● In Airframe Design ● You can replace DB and FluentdLogger to In-Memory Impl ● How to build A and B differs, but the same code can be used A Memory DB Connection Pool DB Client DB Monitor Fluentd Logger In-memory Logger B 21 Overriding Design for Testing
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Complex FILO Order Resource Management ● FILO := First-In Last-Out ● Airframe registers onStart and onShutdown lifecycle hooks when creating instances ● When closing sessions, onShutdown will be called in the reverse order ● Dependencies forms DAG ● Dependencies will be generated when creating new service objects A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 1 3 4 5 6 7 2 8 Shared Resource 24
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 4: Bundling Service Traits ● Flower-Bundle Pattern ● Create composable services 25
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 5: Bind Factory ● Create a factory that change partial dependencies ● e.g., creating differently configured instances of Databases 26
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. 3 Things You Can Forget With Airframe DI ● 1. How to Build Service Objects ● config, auto-wiring, flower bundle pattern ● 2. How to Manage Resource Lifecycle ● FILO order ● 3. How to Use DI Itself (!!) ● Only need to understand bind, design, and build. 27
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Summary: Reducing Code Complexity with Airframe DI ● You can effectively forget about: ● How to build service objects ● How to manage resources in FILO order ● How to use DI itself A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 1 3 4 5 6 7 2 8 28 Implementation Details
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Details ● bind[X] ● X becomes singleton ● Usually there is no need to declare bind[X].toSingleton ■ Unless you want to initialize X early with production mode ● bindInstance[X] ● When you need to call new X(d1, d2, …) every time ● bindFactory[A1 => X] ● For customizing A1 in X ● Design ● Immutable & Serializable ● design1 + design2 + …. ■ Overriding the previous design (adding order is important) ● Session ● withLazyMode/withProductionMode ● noLifeCycleLogging 30
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Surface: Object Shape Inspector ● Reading Type Signatures From ScalaSig ● Scala compiler embeds Scala Type Signatures (ScalaSig) to class files ● Surface supports ● type alias, tagged type ● higher-kinded types class A (data:List[B]) class A data: List[java.lang.Object] class A data: List[java.lang.Object] ScalaSig: data:List[B] javac scalac Surface.of[A] data: List[B] scala.reflect.runtime. universe.TypeTag Type Erasure 32 ???
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Current State of Airframe ● Version 0.73 (As of November 2018) ● We already had 40+ releases in 2018 ● Automated Release ● Cross building libraries for Scala 2.11, 2.12, 2.13, and Scala.js ● ‘sbt release’ command took 3 hours ■ Sequential steps: ○ compile -> test -> package -> upload x 18 modules x 4 Scala versions ● Now a new version can be released in 10 minutes on Travis CI ● Blog ● 3 Tips for Maintaining Scala Projects ● Future Work ● Adding child session support ● Support multiple constructor argument blocks 34
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Philosophy: Simplicity By Design ● “Simplicity” by Philippe Dufour ● A clock made by a legendary watchmaker in Switzerland ● Every part of the clock is built by himself ● Airframe ● Provides simplicity for application developers 35
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Summary ● To understand DI, think about what you can simplify ● How to build objects ● How to manage resources (FILO) ● Learning DI framework itself ● 5 Airframe Design Patterns ● Pattern 1: Configuring Modules ● Pattern 2: Overriding Bindings for Tests ● Pattern 3: Managing Lifecycle in FILO order ● Pattern 4: Bundling Service Traits ■ Flower-bundle pattern ● Pattern 5: Binding Factory Don’t Forget Adding GitHub Star! wvlet/airframe 36
xerial
replu
k1low
masasuzu
peisuke
coconala_engineer
yoshiitaka
nagix
yutoy
h1kita
masa5555
poteboy
zakiwarfel
chrislema
bryan
eileencodes
zakiyama
jakevdp
stephaniewalter
addyosmani
rasmusluckow
nonsquared
sachag
smashingmag