Slide 1
Slide 1 text
How to launch a new product using Scala
Lessons Learned
Yoshimasa Niwa
Feb 25, 2017 in Tokyo
αʔϏεΛθϩ͔Β࡞ͬͯେͩͬͨͭͷ͜ͱ
Slide 2
Slide 2 text
@niw
Yoshimasa Niwa
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
California, USA
San Francisco
αϯϑϥϯγεί͔Β͖·ͨ͠ɻ
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
that we’ve launched.
Products
θϩ͔Βͭͬͨ͘ϓϩμΫτͷհ
Slide 8
Slide 8 text
Search GIF and instantly Tweet it.
GIF Button
(*'Ϙλϯػೳ
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Unleash your creativity by adding fun Stickers.
Stickers
εςοΧʔػೳ
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
How to launch a product using Scala
Lessons Learned
I was (am) not a Scala specialist.
Launched user facing products using Scala.
There are many lessons I’ve learned.
αʔϏεΛθϩ͔Β࡞ͬͯେͩͬͨͭͷ͜ͱ
Slide 14
Slide 14 text
Today’s talk
At glance
History, we’re learning from history.
Architecture, that built from the lessons.
Implementation, that we’re using.
Executions, form what I’ve learned lessons.
ຊͷͷ֓ཁ
Slide 15
Slide 15 text
A short history behind that whale.
History
γεςϜ͕ࠓͷΑ͏ʹͳΔ·Ͱʹྺ࢙͕͋Γ·ͨ͠ɻ
Slide 16
Slide 16 text
in 2009.
The biggest Ruby on Rails application
5XJUUFSੈքͰ͓ͦΒ͘࠷େ͖͍3VCZPO3BJMTͷΞϓϦͰͨ͠
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
So it called “Monorail.”
Many pains in the large monolithic system
• It’s really easy to break someone’s modules.
• grep(1) is the only tool we can believe.
• Deploy everything, revert everything.
ڊେͳ୯ҰͷγεςϜͰਏ͔ͬͨɻ
Slide 20
Slide 20 text
into small services in Scala
Decompose monolithic system
ڊେͳ୯ҰͷγεςϜΛղͯ͠4DBMBͰ࠶࣮͍ͯͬͨ͠
Slide 21
Slide 21 text
to decompose monolithic Ruby on Rails application
Rewrite each module in Scala
class User < ActiveRecode::Base
def self.find(...)
$user_service.find(...)
end
end
User.find(1) #=> #
ڊେͳ3VCZPO3BJMTΞϓϦέʔγϣϯΛղͯͦ͠ΕͧΕͷϞδϡʔϧΛ4DBMBͰ
ॻ͖ͨ͠ɻ
Slide 22
Slide 22 text
into a single system.
Compose each small service
ղͨ͠αʔϏεΛ߹ͯ͠ɺγεςϜͱͯ͠ػೳ͢Δ
Slide 23
Slide 23 text
Deal with complexity
“Functional” Architecture
ʮؔతͳʯΞʔΩςΫνϟͰɺෳࡶͳΈʹରԠ͢Δɻ
Slide 24
Slide 24 text
Goal is managing complexity.
System is complex
γεςϜͱͯෳࡶɻͦͷෳࡶ͞Λ͍͔ʹཧ͢Δ͔ͱ͍͏ͷ͕ΰʔϧɻ
Slide 25
Slide 25 text
Example code
System is complex
def main(args: Array[String]) {
args match {
case Seq("command1") =>
...
case Seq("command2") =>
...
case Seq("command3") =>
...
// Continue 10,000 lines
}
}
ෳࡶͳίʔυͷྫ
Slide 26
Slide 26 text
Divide-and-rule
Separate interests and responsibilities
ෳࡶͳγεςϜׂ౷࣏ɻॲཧͷରͱΛׂɻ
Slide 27
Slide 27 text
Divide-and-rule.
System is complex
def func1() {
...
}
def func2() {
...
}
def func3() {
...
}
def main(args: Array[String])
{
args match {
case Seq("command1") =>
func1()
case Seq("command2") =>
func2()
case Seq(“command3") =>
func3()
...
}
}
ෳࡶͳίʔυΛநԽׂͯͨ͠͠ྫ
Slide 28
Slide 28 text
Getting request and returning response.
Service is a function
αʔόʔɺͭ·ΓؔɻϦΫΤετΛҾʹͱͬͯɺϨεϙϯεΛฦ͢ɻ
def service(
request: Request
): Response
Slide 29
Slide 29 text
Create new functions in the system.
Create a new service
৽αʔϏεΛ࡞ΔʹγεςϜʹ৽͍ؔ͠ΛՃ͢Δ
def service(
request: Request
): Response
def newService(
request: Request
): Response
Slide 30
Slide 30 text
Response is not always available immediately.
In real world, however.
def service(request: Request): Response = ...
// How slow is this?
val reposes = service(request)
ݱ࣮ʹɺ͔͠͠ɺؔͷଈ࠲ʹखʹೖΔΘ͚Ͱͳ͍ɻ
Slide 31
Slide 31 text
We need to wait.
Everything is asynchronous
ͯ͢ඇಉظɻ͕ͪ࣌ؒ͋Δɻ
Slide 32
Slide 32 text
Example
GIF Button
(*'Ϙλϯͷ߹
Loading Successful Failed
Slide 33
Slide 33 text
A capsule of the value
Option[+A] ?
Λ͍ΕΔͱ͍͑ɺ0QUJPO<">
+A
Option
Value
Some None
Slide 34
Slide 34 text
A capsule of the value
Future[+A]
͋Δ͕औΓ͏Δঢ়ଶΛ'VUVSFͰแΜͰ͓͘
+A
Future
? Value Exception
Pending Successful Failed
Slide 35
Slide 35 text
Example
Future[+A]
def func(a: String): Future[String] = …
def main(args: Array[String]) {
// This call is not blocking.
val future = func(args.head)
// Do awesome things...
// Wait until reply is defined
println(Await.result(future))
}
ྫ͑ɺ͜͏͍͏ײ͡ʹͳΔɻ
Slide 36
Slide 36 text
Getting request and reply in Future.
Service[-Req,+Rep]
αʔϏεϦΫΤετΛड͚औͬͯɺϨεϙϯεͷ'VUVSFΛฦ͢ɺͱॻ͚Δɻ
Req => Future[Rep]
Rep
Req Waiting… Response Exception
Pending Successful Failed
Slide 37
Slide 37 text
Future works same as the other capsules.
Compose Future[+A]
• map[A,B](A => B): Future[B]
• flatMap[A,B](A => Future[B]): Future[B]
• join[A,B](Future[A], Future[B]):
Future[(A,B)]
• collect[A](Seq[Future[A]]): Future[Seq[A]]
'VUVSFଞͷ4DBMBͷίϨΫγϣϯͱಉ༷ʹ߹Մೳɻ
A
Future[A]
B
Future[B]
map
A
Future[A]
B
Future[B]
flatMap
Slide 38
Slide 38 text
Example
Compose service calls
val userRepo: Service[Long, User] = ...
val profileRepo: Service[User, Profile] = ...
val postRepo: Service[User, Seq[Post]] = ...
userRepo(userId).flatMap { user =>
Future.join(
profileRepo(user),
postRepo(user)
).map { (profile, posts) =>
renderResponse(profile, posts)
}
}
αʔϏεͷ݁Ռ'VUVSF͔ͩΒɺݺͼग़͠Λ࣍ʑ߹Ͱ͖Δɻ
Slide 39
Slide 39 text
class GifSearchHandler {
def apply(request: SearchRequest):
Future[SearchResponse] = for {
user <- userRepo(request.user_id)
results <- Future.collect(
gifRepos.map { repo =>
repo(request.query, user.lang)
}
)
} yield merge(results)
}
GIF Button
Example
ྫ͑(*'#VUUPOͷݕࡧॲཧ͜Μͳײ͡
Slide 40
Slide 40 text
Deal with complexity
“Functional” Architecture
Divide-and-rule
Function as a service
Compose response Future[+A] to make a logic
asynchronously
ʮؔతͳʯΞʔΩςΫνϟ
Slide 41
Slide 41 text
Things we need to build services
Implementation
αʔϏεΛ࡞Δͷʹඞཁͳ࣮
Slide 42
Slide 42 text
• Define functions
• Resolve functions
• Implement functions
• Instantiate functions
• Discover functions
Things we need
to call function
γεςϜ͕ඞཁͱ͍ͯ͠Δ͜ͱ
Slide 43
Slide 43 text
Thrift, an IDL to define models and services
Scrooge, a Thrift code generator in Scala
Thrift, Scrooge
Define functions
ఆٛʹ5ISJGUΛ͏ɻ4DSPPHFͰ4DBMBͷίʔυੜɻ
Slide 44
Slide 44 text
Resolve functions
Mono repo
Git, “special version”
Faster clone
Pants, a build system for mono repo.
੩తͳґଘؔϞϊϨϙͱ1BOUTϏϧυγεςϜͰղܾ
Slide 45
Slide 45 text
Finagle, a RPC core library.
Finatra, a framework using Finagle
TwitterServer (HTTP server and Thrift server)
Guice dependency injection
Finagle, Finatra
Implement functions
࣮ʹ'JOBHMFΛϕʔεʹ'JOBUSBΛ͍ͬͯΔ
Slide 46
Slide 46 text
Instantiate functions
Mesos, Aurora
Mesos, a cluster manager for data center.
Aurora framework, a job scheduler.
Define job in aurora DSL file.
.FTPTͱ"VSPSBϑϨʔϜϫʔΫͰΠϯελϯεΛཧ
Slide 47
Slide 47 text
Discover functions
Zookeeper
Zookeeper, a reliable directory for service discovery.
Announced by Aurora
Resolved by Finagle
;PPLFFQFSͰαʔϏεΛൃݟ͢Δɻ"VSPSBͰΠϯελϯεΛ໊લొͯ͠ɺ'JOBHMF
Ͱղܾ
Slide 48
Slide 48 text
.scala
.thrift
.aurora
.scala
.thrift
.aurora
Git
.scala
.thrift
.aurora
Mesos
Aurora
Zookeeper
/s/gifsearch
Instance
Instance
Slide 49
Slide 49 text
Things we need to build services
Implementations
Define in Thrift, manage every things in a single Git
repository.
Finatra to implement services.
Aurora to run services and discovery.
αʔϏεΛ࡞Δͷʹඞཁͳ࣮
Slide 50
Slide 50 text
Executions
Let’s talk about making things.
࡞ਐߦʹ͍ͭͯ
Slide 51
Slide 51 text
We’re a team
Engineers, product manager, and designers works in a team.
ΤϯδχΞɺϓϩμΫτϚωʔδϟɺσβΠφ͕ɺͻͱͭͷνʔϜͰࣄ͠·͢ɻ
Slide 52
Slide 52 text
P
Product
Manager
D
Designer
E
Engineer
Design
Plan
Implement
Slide 53
Slide 53 text
Engineers have different responsibilities than Product Managers
Product vs. Engineering
ϓϩμΫτରΤϯδχΞϦϯά
ΤϯδχΞϓϩμΫτϚωʔδϟͱҧ͏͕͋Δ
Slide 54
Slide 54 text
P
Product
Manager
D
Designer
E
Engineer
Design
Plan
Implement
Slide 55
Slide 55 text
Maximize meeting product requirements
Minimize tech-debt
ϓϩμΫτͷཁٻʹ͑ͭͭɺٕज़తͳෛͷҨ࢈ΛݮΒ͢ɻ
Slide 56
Slide 56 text
Integrate everything at first.
Build app from top to bottom
Bad assumption may increase tech-debt.
Right assumption may reduce tech-debt.
Reduces many bad assumptions.
·্͔ͣΒԼ·Ͱͱʹ͔͘ܨ͍Ͱಈ͔͢ɻఆΛͰߟ͑Δ͜ͱΛͰ͖Δ͚ͩݮΒ͢ɻ
S S
S S
S
Service Service
Service
Service Service
Slide 57
Slide 57 text
Create many instances with modification on Aurora
Run devel instances
Work-in-progress instance is okay.
Working mock is better than tech-doc to deal with
product manager, sometimes.
։ൃ༻ΠϯελϯεΛ࣮ߦ͓ͯ͘͠ɻ
Slide 58
Slide 58 text
Override services for debugging and testing
Dtab
A delegation table.
Override finagle resolver to swap services.
Dtab-local: header for HTTP request.
%UBCΛ໊ͬͯલղܾΛ্ॻ͖ͯ͠αʔϏεΛϥϯλΠϜʹೖΕସ͑Δ
Slide 59
Slide 59 text
Naming and Dtab
Naming
/s/gifsearch
/s/prod/gifsearch
/zk/zk.domain:2181/prod/gifsearch
/$/com.twitter.serverset/zk.domain:2181/prod/
gifsearch
αʔϏεͷ໊લղܾͱ%UBC
Slide 60
Slide 60 text
Naming and Dtab
Naming
/s/gifsearch
/s/prod/gifsearch
/s/devel/gifsearch
/zk/zk.domain:2181/devel/gifsearch
/$/com.twitter.serverset/zk.domain:2181/devel/
gifsearch
αʔϏεͷ໊લղܾͱ%UBC
/s/prod/gifsearch
=>/s/devel/gifsearch;
Slide 61
Slide 61 text
Deal with anomalies and exceptional cases
Design vs. Engineering
σβΠϯରΤϯδχΞϦϯά
ྫ֎ʹ͍͔ʹରԠ͢Δ͔
Slide 62
Slide 62 text
P
Product
Manager
D
Designer
E
Engineer
Design
Plan
Implement
Slide 63
Slide 63 text
by maintain the application state.
Reduce exceptional cases
ΞϓϦέʔγϣϯͷঢ়ଶཧΛͯ͠ྫ֎ΛݮΒ͢ɻ
Slide 64
Slide 64 text
are user facing and used by human beings.
iOS and Android clients
ϞόΠϧΫϥΠΞϯτਓ͕͔ؒͭ͏ͷɻ༧ଌ֎ͷૢ࡞Λ͞ΕΔ͜ͱ͋Δɻ
Slide 65
Slide 65 text
User interface is a representation of internal states.
Manage internal state.
Users changes the internal state.
React to the internal state changes.
Applying animation, if needed.
෦ঢ়ଶͱมߋͷྲྀΕΛཧ͢Δɻ6*෦ঢ়ଶΛදݱ͢Δͷɻ
S
Action State Change
UI
Update
Action
U
User
Slide 66
Slide 66 text
Communication between teams and services
Engineering vs. Engineering
ΤϯδχΞϦϯάରΤϯδχΞϦϯά
νʔϜಉ࢜ͷΓͱΓ
Slide 67
Slide 67 text
P
Product
Manager
D
Designer
E
Engineer
Design
Plan
Implement
E
Engineer
nt
Im
plem
ent
Slide 68
Slide 68 text
Truth is in front of you as a code.
Read code
৴͡ΒΕΔͷιʔε͚ͩɻ
Slide 69
Slide 69 text
Single “source” of truth.
Mono repo helps it.
Identify exact source code by deploy SHA-1.
Browse entire source code that the system is using.
୯ҰιʔεϨϙδτϦͰਅ࣮ΛݟΔ
commit 8270324cdf0d1e60a83dbefa1de29d89f6107587
commit f40648b9105f7478139694acd0821c15d3315dc2
commit fa0dc865887a02b49576bdc1de57f4ad802d4710
commit ebf31743cbe3008ec311d88aea42b987b0288cd3
Slide 70
Slide 70 text
Name tells what it is.
Define right functions
Define API in reasonable way.
Naming is really really important.
They never read tech-doc.
ਖ਼͍ؔ͠ͷఆٛυΩϡϝϯτΑΓେɻ
outputImageProviderFromBufferWithPixelFormat:pixelsW
ide:pixelsHigh:baseAddress:bytesPerRow:releaseCallba
ck:releaseContext:colorSpace:shouldColorMatch:
Slide 71
Slide 71 text
Manage dependencies.
Downstream
Estimate QPS and latency.
Right timeout and retry policy.
Right load banaler
ApertureLoadBalancer for lower QPS
214ͱϨΠςϯγΛཧɻϥΠϜΞτͱϦτϥΠͷௐɺϩʔυόϥϯαͷௐɻ
S S
S S
S
Service Service
Service
Service Service
Slide 72
Slide 72 text
Manage dependencies
Upstream
Look at CancelledRequestException.
Upstream timed out for downstream, it will cancel
request and downstream will see this exception.
$BODFMMFE3FRVFTU&YDFQUJPOʹҙɻ
6QTUSFBN͕ݫ͍͠λΠϜΞτΛ࣋ͬͯΔՄೳੑ͋Γɻ
S S
Service Service
Timeout!
Cancel
Exception
Timeout
Exception
Slide 73
Slide 73 text
Conclusion
Lessons Learned
·ͱΊ
Slide 74
Slide 74 text
How to launch a product using Scala
Lessons Learned
1. Build app from top to bottom
to deal with product managers.
2. Manage internal state
to deal with designers.
3. Read code
to deal with engineers.
αʔϏεΛθϩ͔Β࡞ͬͯେͩͬͨͭͷ͜ͱ
Slide 75
Slide 75 text
#thankyou
@niw
Questions?
͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ