Share
Embed
国技と Scala (Japan's national sport and Scala)
Presentation at ScalaMatsuri '14.
Streaming here (Japanese).
http://live.nicovideo.jp/watch/lv191315006
# Start at 9:00〜
Takuya Fujimura
September 06, 2014
Transcript
113 slides
-
ࠃٕ ͱ Scala Takuya Fujimura Dwango Mobile Co., LTD. @tlync
Japan’s National Sport and Scala 2014 ScalaMatsuri -
About Me • name: • Takuya Fujimura (@tlync) • job:
• Tech. Director, Programmer etc… • Dwango Mobile Co., LTD. • like: • Wine, Beer, Cat, Programming • lang: • Java/Scala/JavaScript (Scheme/Swift) • Scala Newbie (< 1year) -
The SUMO association’s Official iOS/Android App. Launch a Service 2014/03
http://sumo.dwango.jp/ -
MySQL slave HDFS ac6ve PHP Play API
Apache Play Push HDFS standby MySQL master PHP iOS Android Memcached Play API Apache Play Push Memcached Play Apache api 02 LB LB Play Apache assets 01 assets 02 api 01 Legacy Backoffice tool Shared digital assets repository SUMO DB Torikumi Video Torikumi Video Torikumi Video Torikumi Video Torikumi Video Torikumi Video Torikumi Video SUMO data -
Middleware • Apache 2.4 • MySQL 5.5 • Memcached 1.4
• HDFS 2.0 ※ shared -
Framework/Libraries • Scala 2.10 • Framework … Play! 2.2.1 •
Template Engine… twirl (w/ play) • DBAL … ScalalikeJDBC 1.7.4 • Testing … ScalaTest 2.0 • DI … Scaldi 0.2.2 -
Technical Functionality • SUMO Content REST API • Video Progressive
Download • Install based Authentication • iOS/Android In-App Purchase/Billing Validation • Push Notification (Amazon SNS) etc… -
Why we use Scala? 1. Type-Safety 2. Less Code (than
PHP, Java) 3. Java Experience (Me) 4. Improving our programming skills -
Why we use Scala? 1. Type-Safety 2. Less Code (than
PHP, Java) 3. Java Experience (Me) 4. Improving our programming skills -
Familiar Story A Service needs to keep evolving. But It’s
hard to maintain middle, large software without static-typed. Test is not enough for error detection Tests is written by human vs Cost -
Familiar Story A Service needs to keep evolving. But It’s
hard to maintain middle, large software without static-typed. Test is not enough for error detection Tests is written by human vs Cost We need type-safety dev environment to keep evolving software -
Improving our programming skills • Scala is multi paradigm language
• Scala has many concepts • Programmers can get many new perspectives OOP FP Trait Monad Object Typed Class Actor -
Scala makes good programmer e.g: Just PHP Programmer > Scala
Programmer Skill Learning OOP! Immutable! map/fold! Trait! Monad! OOP? Immutable? map/fold? Trait? Monad? -
How we learned Scala? • Toy Project Try & Error
in irresponsible project It was terrible code…. • Reading Circle Programming Scala 2nd Edition 1 hour / day in our office thx @daneko0123 -
Initial Dev Time Operation Time < 平成二十二年 一月場所 平成二十三年 三月場所
平成二十四年 五月場所 平成二十五年 七月場所 平成二十六年 九月場所 平成二十七年 十一月場所 -
Design Decisions • Full type-safety environment • Domain Driven Design
• Hybrid App (Web/Native) • Microservices -
Full type-safety environment Scala Lang Play & twirl Web &
View ScalikeJDBC Persistence -
@rikishiProfile.kakuzuke.map { k => <a class="button" href="@banzukehyo(bashoId, k.id)">൪දΛݟΔ</a>} </div> }
</div> <h2>ϓϩϑΟʔϧৄࡉ</h2> <div class="profile-detail"> <table class="profile-table"> <tr> <th>࢛ވ໊</th> <td>@rikishiProfile.shikonaFull <br> (@rikishiProfile.shikonaYomiFull) </td> </tr> <tr> <th>൪</th> <td>@rikishiProfile.banzuke match { case Some(b) => {@b.fullName} case None => {൪֎} } </td> </tr> <tr> <th>ग़</th> <td>@rikishiProfile.birthplace</td> </tr> @rikishiProfile.heya.map { h => <tr> Complex? Really? -
@rikishiProfile.kakuzuke.map { k => <a class="button" href="@banzukehyo(bashoId, k.id)">൪දΛݟΔ</a>} </div> }
</div> <h2>ϓϩϑΟʔϧৄࡉ</h2> <div class="profile-detail"> <table class="profile-table"> <tr> <th>࢛ވ໊</th> <td>@rikishiProfile.shikonaFull <br> (@rikishiProfile.shikonaYomiFull) </td> </tr> <tr> <th>൪</th> <td>@rikishiProfile.banzuke match { case Some(b) => {@b.fullName} case None => {൪֎} } </td> </tr> <tr> <th>ग़</th> <td>@rikishiProfile.birthplace</td> </tr> @rikishiProfile.heya.map { h => <tr> Complex? Really? Hatena’s designer is doing it, isn’t it!? -
Design Decisions • Full Type-Safety environment • Domain Driven Design
• Hybrid App (Web/Native) • Microservices -
Design Decisions • Domain Driven Design • Mental Model =
Language = Communication = Codes • Make Understandability & Clarity & Extensibility for SUMO specific specifications as code • Hybrid App (Web/Native) • Change content without AppStore/Google Play • Microservices • Separate concerns like PUSH notification. -
./domain ./domain ├── auth │ ├── AccessToken.scala │ ├── Device.scala
│ ├── RequestToken.scala │ ├── User.scala │ └── UserRepository.scala ├── honbasho │ ├── FusenType.scala │ ├── JudgeType.scala │ ├── banzuke │ │ ├── Banzuke.scala │ │ └── BanzukeRepository.scala │ ├── basho │ │ ├── Basho.scala │ │ └── BashoRepository.scala │ ├── hoshitori │ │ ├── CareerRecord.scala │ │ ├── Hoshitori.scala │ │ ├── HoshitoriHyo.scala │ │ ├── HoshitoriHyoRepository.scala Domain? -
object Kakuzuke { ! ! ! ! ! !
… def all: Seq[Kakuzuke] = Seq(Makuuchi, Juryo, Makushi Sandanme, Jonidan, Jonokuchi) } sealed abstract class Kakuzuke(val id: Int, val name: S object Kakuzuke { case object Makuuchi extends Kakuzuke(1, "ນ") case object Juryo extends Kakuzuke(2, "े྆") case object Makushita extends Kakuzuke(3, "ນԼ") case object Sandanme extends Kakuzuke(4, "ࡾஈ") case object Jonidan extends Kakuzuke(5, "ংೋஈ") case object Jonokuchi extends Kakuzuke(6, "ংϊޱ") 格付 -
class RikishiProfile( id: Int, shikona: String, shikonaYomi: String, birthplace: String,
thumbnail: Option[RikishiThumbnail], val shikonaFull: String, val shikonaYomiFull: String, val heya: Option[Heya], val realName: String, val birthDay: LocalDate, val age: Int, val height: Float, val weight: Float, val banzuke: Option[Banzuke], val kakuzuke: Option[Kakuzuke], val prizeWinningStats: PrizeWinningStats) extends RikishiBaseProfile( id, shikona, shikonaYomi, 力士 -
Microservices Play SUMO REST API Play Push No&fica&on
API Old PHP Backoffice Tool Amazon SNS hEp:9000 hEp:9001 hEp:9001 -
Microservices Play SUMO REST API Play Push No&fica&on
API Old PHP Backoffice Tool Amazon SNS hEp:9000 hEp:9001 hEp:9001 Generic Push No&fca&on Service SUMO Domain SUMO Domain -
Microservices Play SUMO REST API Play Push No&fica&on
API Old PHP Backoffice Tool Amazon SNS hEp:9000 hEp:9001 hEp:9001 Generic Push No&fca&on Service SUMO Domain SUMO Domain Isolate generic functionality -
Library Impressions • ScalikeJDBC … DB Access • twirl …
Template Engine • Scaldi … DI • ScalaTest … Testing -
ScalikeJDBC • Good • Just SQL … Avoid unnecessary abstraction.
Good for existing system. • Type-Safe … QueryDSL • Bad • Development resource? → NO!!!! • ConnectionPoolSetting is not flexible (… well, we can submit pull request) • Note • Slick is looks like cool. But we don’t want to translate complex SQL queries to Scala code. -
Twirl • Excellent! • Type-Safety… Prevent many errors at compile
time. • Don’t be afraid to change method/property signature !!! • Good • Flexibility … Just Scala • Bad • Flexibility … Just Scala, Unfriendly for designers • Slow Feedback Cycle … Slow compilation… We developed initial templates without twirl using Grunt etc. • Note • I think better of this than I did before -
Twirl • Excellent! • Type-Safety… Prevent many errors at compile
time. • Don’t be afraid to change method/property signature !!! • Good • Flexibility … Just Scala • Bad • Flexibility … Just Scala, Unfriendly for designers • Slow Feedback Cycle … Slow compilation… We developed initial templates without twirl using Grunt etc. • Note • I think better of this than I did before Should be evaluated!? -
Scaldi • Good • minimum & simple • Bad •
Verbose declaration ! • Dynamic … You won’t get an error until you run your application • Note • The cake pattern seems quite verbose for me. • Next, I’ll look for other solution. (like Play! 2.4 DI) class FooService(implicit inj: Injector) extends Injectable Fixed after presentation: use constructor injection! instead of this. It’s simple! -
ScalaTest • Good • Simple … vs Specs2. Specs2 provides
several notations that seems not simple for me. • Bad • Limited structuring texts … vs RSpec. No context etc… • Note • Well… Either way -
Practices • Exception Handling • Non-Stop Deployment • Static Code
Analysis • Tuning Akka for Blocking App -
Scala provides several ways handling exceptions. Problem Exception Handling Try
Option Either try catch (Validation/scalaz) -
Scala provides several ways handling exceptions. Problem Exception Handling Try
Option Either try catch (Validation/scalaz) Which is Scala way? -
class AccessTokenService(private val secret: String) (implicit i: Injector) extends Logging
with Injectable { ! val userRepository = inject[UserRepository] val TermOfValidityMills = 60000 // 1min ! def issueToken(reqToken: RequestToken): Either[InvalidTokenException, AccessToken] = { log.debug(s"issueToken - reqToken=$reqToken") ! ... } } Either with Exception? -
class AccessTokenService(private val secret: String) (implicit i: Injector) extends Logging
with Injectable { ! val userRepository = inject[UserRepository] val TermOfValidityMills = 60000 // 1min ! def issueToken(reqToken: RequestToken): Try[AccessToken] = { log.debug(s"issueToken - reqToken=$reqToken") ! ... } } Nope. Use Try -
I don’t know Scala way… But our policy is below.
try catch Option Try Either (Validation,Either etc../scalaz) No explanation needed Do not use generally (Java world syntax) Use this for exception handling! < Scala 2.10 Use error handling except exceptions. … -
Problem Static Code Analysis [IMO] In 2013, Many unmaintained static
code analysis libraries. such as SCCT, sbt-scct… -
Others Static Code Analysis • ScalaStyle • Examines your Scala
code and indicates potential problems with it. • Scapegoat • Scala static code analyzer • WartRemover • Flexible Scala code linting tool • cpd4sbt • Integrating PMD's Copy/Paste Detector • Scalariform (not analysis) • Source code formatting -
Play! needs to stop when deployment basically. Stop service n
sec = LB’s Heartbeat Problem Non-Stop Deployment -
1. Use Load Balancer API 2. Implement Server Status API
/w LB 3. Pray to implement graceful restart in the future ;p 4. Others… Do you have any ideas? Solution? Non-Stop Deployment -
1. Use Load Balancer API 2. Implement Server Status API
/w LB 3. Continue to pray to implement graceful restart in the future 4. Others… Do you have any idea? Solution? Non-Stop Deployment -
Deployment Flow Non-Stop Deployment 1. Remove Node from LB via
LB API ↓ 2. Deploy to detached Node new app. ↓ 3. Waiting for start ↓ 4. Add Node to LB via LB API ↓ 5. Repeat… ! -
@task @roles("app") def _deploy_main_job(branch, working_tree, dist_skip, cache): ! developer_mode =
working_tree or dist_skip ! # on local prepare_remote() ! if not dist_skip: package(cache) ! compile_rc_script() transfer() ! # on remote infrate_package() if env.enable_lb_switching: disconnect_loadblancer() time.sleep(5) stop_app() switch_current_app() start_app() if env.enable_lb_switching: reconnect_loadblancer() ! -
compile_rc_script() transfer() ! # on remote infrate_package() if env.enable_lb_switching: disconnect_loadblancer()
time.sleep(5) stop_app() switch_current_app() start_app() if env.enable_lb_switching: reconnect_loadblancer() ! clean_old_releases() ! @task @roles("app") def disconnect_loadblancer(): local(“/usr/local/bin/lb {0} disable".format(env.host)) ! @task @roles("app") def reconnect_loadblancer(): local("/usr/local/bin/lb {0} enable".format(env.host)) -
Caution Tuning Thread Pool The Play! default configuration is optimized
for Non-Blocking I/O -
• Pure asynchronous • for Non-Blocking app • Highly synchronous
• for Blocking app • Many specific thread pools • Hybrid • Few specific thread pools • Hybrid Tuning Practices Tuning hread Pool -
• SUMO app is powered by Scala/Play! • Type-Safety world
makes great safety & productivity. • Scala is suited for long-maintained or middle-large applications. • I worry about binary compatibility Summary -
! Scala is already practical. From now on, let’s share
our practices, codes… more to keep evolving Scala ecosystem & community. -
! thank you • Scala developers • ScalaMatsuri staffs •
Sumo project members & supporters • everybody