Any organization that designs a
system will produce a design whose
structure is a copy of the organization's
communication structure
Conway’s Law
Slide 12
Slide 12 text
The rules and boundaries
of our systems should
stem from architecture
not organisation
Slide 13
Slide 13 text
How the system interacts
What is the systems purpose
Macro Architecture
Domain Architecture
How the system is built
Micro Architecture
Slide 14
Slide 14 text
Systems as the unit of work
Not unit of work as a system
Slide 15
Slide 15 text
Ģ
B
ģ
z
SYSTEMS
of
SYSTEMS
Slide 16
Slide 16 text
The thing about real systems is
AUTONOMY
Slide 17
Slide 17 text
CROSS LANGUAGE CODE SEARCH
AS AN EXAMPLE
Slide 18
Slide 18 text
Code search the JVM wrong way
Pick Framework
Slide 19
Slide 19 text
Scala + Play
Pick Language + Framework
Code search the JVM wrong way
Slide 20
Slide 20 text
UI API
FILESYSTEM
Code search the JVM wrong way
Scala + Play
Slide 21
Slide 21 text
UI API
DB INDEXER
Code search the JVM wrong way
Scala + Play
Slide 22
Slide 22 text
SYSTEMS, NOT SUB-SYSTEMS
Slide 23
Slide 23 text
Code search as Systems
INDEXING SEARCH
Slide 24
Slide 24 text
Code search as Systems
Domain Architecture
INDEXING SEARCH
Slide 25
Slide 25 text
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Slide 26
Slide 26 text
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Macro Architecture
Slide 27
Slide 27 text
STANDARD FORMATS HELP
GIVE US AUTONOMY
Slide 28
Slide 28 text
Code search as Systems
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
OS for services
Slide 29
Slide 29 text
Code search as Systems
INDEXING SEARCH
Micro Architecture
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
Slide 30
Slide 30 text
Code search as Systems
INDEXING SEARCH
Micro Architecture
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
MORE AUTONOMY
Slide 31
Slide 31 text
RULES
ARCHITECTURE IS ABOUT
NOT BOXES
Slide 32
Slide 32 text
Signs of failure
Systems are not autonomous
Systems share a domain model
Systems need to be evolved in step
Systems need to be built by single team
Systems share a data base
Slide 33
Slide 33 text
Systems as the unit of work
If you can’t add a person to
your project then you are
failing worse than you think
again
An Aside
Slide 34
Slide 34 text
Ģ
B
ģ
z
EVOLUTION
Slide 35
Slide 35 text
Thinking ahead is not about avoiding change
Thinking ahead is about letting us change at
different rates for different problems
Slide 36
Slide 36 text
Indexing V2
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Slide 37
Slide 37 text
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Micro Architecture
Indexing V2
Slide 38
Slide 38 text
now vs later
priorities shift over time,
can't let early traction
sacrifice long term speed
An Aside
Slide 39
Slide 39 text
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Indexing V2
DIFFERENT SYSTEMS SAME RULES
DIFFERENT SYSTEMS DIFFERENT RULES
An Aside
Macro Architecture
Search V2
INDEXING SEARCH
CODE
CTAGS
Super-Zippy Format
application/html
application/search.v1+json
Super-Zippy Format
Slide 42
Slide 42 text
Notice how easy it is to think of
evolution in the large
Slide 43
Slide 43 text
Rates of change
domain
macro
micro
Slide 44
Slide 44 text
Rates of change
domain
macro
micro
Freedom to choose tools and technology by succeeding at domain and protocols
Slide 45
Slide 45 text
HOW DO SYSTEMS THINK
HOW DO SYSTEMS COMMUNICATE
HOW DO SYSTEMS CHANGE
Slide 46
Slide 46 text
Implications at
Scale
Slide 47
Slide 47 text
The concept of micro-services are great
An Aside
The concept of small libraries are great
But they are not enough without
thinking in systems
Slide 48
Slide 48 text
Larger Systems of Systems
Slide 49
Slide 49 text
How long does it take to get 1 line
of code to production
Slide 50
Slide 50 text
Larger Systems of Systems
Slide 51
Slide 51 text
Mistaking Modules For Systems
Slide 52
Slide 52 text
Mistaking Modules For Systems
Slide 53
Slide 53 text
Mistaking Modules For Systems
Slide 54
Slide 54 text
Implication of Autonomy
USERS
Slide 55
Slide 55 text
Implication of (lack of) Autonomy
USERS
Slide 56
Slide 56 text
Ģ
B
ģ
z
SYSTEMS
in
ACTION
Slide 57
Slide 57 text
CMSs hold customer data hostage
CONTENT ANALYSIS & DATA MINING PROJECT
Free data
Analyse data
Derive data
CMSs tied to horrible platforms
Goals
Constraints
Slide 58
Slide 58 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
Slide 59
Slide 59 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
JSP Java
Slide 60
Slide 60 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
1 JSP Java
Slide 61
Slide 61 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
Java
1 JSP
Slide 62
Slide 62 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
1 JSP Java
Java
Scala
Slide 63
Slide 63 text
WHAT I IMAGINED
DATA
WEB
CORE LEECH PORTAL
CONTENT ANALYSIS & DATA MINING PROJECT
Slide 64
Slide 64 text
WEB / CORE / DATA / LEECH
WEBSHERE
REALITY SETTING IN
Slide 65
Slide 65 text
CAUGHT THINKING IN THE
SMALL
MODULES - LIBRARIES - TECH
Slide 66
Slide 66 text
We should leverage architecture changes
rather than force new technology
Slide 67
Slide 67 text
CONTENT
ANALYSIS
IDENTITY
CONTENT
STORE
QUERY
ENGINE
A Better Approach
Slide 68
Slide 68 text
Play 1.1 App
Our Starting Point
Slide 69
Slide 69 text
Play 1.1 App
Enabling Moves
IDENTITY
Slide 70
Slide 70 text
Play 1.1 App
Protobufs
Thrift
HTTP
Establishing Rules & Autonomy
sbt-assembly
OS package
Containers
IDENTITY
Protocols
Deployment
Slide 71
Slide 71 text
Play 1.1 App
Identifying Demand
CONTENT
STORE
IDENTITY
Slide 72
Slide 72 text
Play 1.1 App
Validation
CONTENT
STORE
IDENTITY
Slide 73
Slide 73 text
Play 1.1 App
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
Slide 74
Slide 74 text
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
QUERY
ENGINE
Slide 75
Slide 75 text
architecture is an everyday task
Slide 76
Slide 76 text
good architecture means never
having to rewrite
Slide 77
Slide 77 text
All editors are bad, really bad
ONLINE CONTENT EDITING
Something aaS
APIs
Ready to change
Small team, tight deadline
Goals
Constraints
Lots of ideas, not sure what would work
C Scala Scala
Spelling Identity Templates
Editor
Customer
UI
Micro Architecture
Scala/JS Scala
Postgres
Postgres Postgres
syslog
jail per service
rc.d
1 executable
Minimal stack
Slide 81
Slide 81 text
Editor was working well
ONLINE CONTENT EDITING
Incremental
Multiple Versions
Add not modify
Needed to add more services
Goals
Constraints
System need to keep running
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Scala/JS Scala JS
C Scala Scala Scala
multilog
jail per service
daemon tools
1 executable
Minimal stack
Slide 85
Slide 85 text
Demand for standalone features
ONLINE CONTENT EDITING
Quick
Minimal Overhead
Maximum Reuse
Not tied to our editor
Goals
Constraints
Run in parallel to current system
ARCHITECTURE SHOULD HAVE
RULES FOR HOW WE SHARE CODE
Slide 91
Slide 91 text
Ģ
B
ģ
z
RECAP
Slide 92
Slide 92 text
RULES
ARCHITECTURE IS ABOUT
NOT BOXES
Slide 93
Slide 93 text
good architecture means never
having to rewrite
Slide 94
Slide 94 text
Thinking ahead is not about avoiding change
Thinking ahead is about letting us change at
different rates for different problems
Slide 95
Slide 95 text
Systems as the unit of work
Not unit of work as a system
Slide 96
Slide 96 text
The thing about real systems is
AUTONOMY
Slide 97
Slide 97 text
Ģ
B
ģ
z
FIN
@markhibberd
Slide 98
Slide 98 text
References + More Info
Stefan Tilkov's Breaking the Monolith slides video
Coda Hale and Ryan Kennedy on "Streamie" notification service at yammer
Slide Deck v2
v1