Any organization that designs a
system will produce a design whose
structure is a copy of the organization's
communication structure
Conway’s Law
Slide 12
Slide 12 text
The rules and boundaries
of our systems should
stem from architecture
not organisation
Slide 13
Slide 13 text
How the system interacts
What is the systems purpose
Macro Architecture
Domain Architecture
How the system is built
Micro Architecture
Slide 14
Slide 14 text
Systems as the unit of work
Not unit of work as a system
Slide 15
Slide 15 text
Ģ
B
ģ
z
SYSTEMS
of
SYSTEMS
Slide 16
Slide 16 text
The thing about real systems is
AUTONOMY
Slide 17
Slide 17 text
CROSS LANGUAGE CODE SEARCH
AS AN EXAMPLE
Slide 18
Slide 18 text
Code search the JVM wrong way
Pick Framework
Slide 19
Slide 19 text
Scala + Play
Pick Language + Framework
Code search the JVM wrong way
Slide 20
Slide 20 text
UI API
FILESYSTEM
Code search the JVM wrong way
Scala + Play
Slide 21
Slide 21 text
UI API
DB INDEXER
Code search the JVM wrong way
Scala + Play
Slide 22
Slide 22 text
SYSTEMS, NOT SUB-SYSTEMS
Slide 23
Slide 23 text
Code search as Systems
INDEXING SEARCH
Slide 24
Slide 24 text
Code search as Systems
Domain Architecture
INDEXING SEARCH
Slide 25
Slide 25 text
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Slide 26
Slide 26 text
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Macro Architecture
Slide 27
Slide 27 text
STANDARD FORMATS HELP
GIVE US AUTONOMY
Slide 28
Slide 28 text
Code search as Systems
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
OS for services
Slide 29
Slide 29 text
Code search as Systems
INDEXING SEARCH
Micro Architecture
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
Slide 30
Slide 30 text
Code search as Systems
INDEXING SEARCH
Micro Architecture
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
MORE AUTONOMY
Slide 31
Slide 31 text
Signs of failure
Systems are not autonomous
Systems share a domain model
Systems need to be evolved in step
Systems need to be built by single team
Systems share a data base
Slide 32
Slide 32 text
The concept of micro-services are great
An Aside
The concept of small libraries are great
But they are not enough without
thinking in systems
Slide 33
Slide 33 text
Ģ
B
ģ
z
EVOLUTION
Slide 34
Slide 34 text
Thinking ahead is not about avoiding change
Thinking ahead is about letting us change at
different rates for different problems
Slide 35
Slide 35 text
Indexing V2
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Slide 36
Slide 36 text
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Micro Architecture
Indexing V2
Slide 37
Slide 37 text
now vs later
priorities shift over time,
can't let early traction
sacrifice long term speed
An Aside
Slide 38
Slide 38 text
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Indexing V2
DIFFERENT SYSTEMS SAME RULES
DIFFERENT SYSTEMS DIFFERENT RULES
An Aside
Macro Architecture
Search V2
INDEXING SEARCH
CODE
CTAGS
Super-Zippy Format
application/html
application/search.v1+json
Super-Zippy Format
Slide 41
Slide 41 text
Notice how easy it is to think of
evolution in the large
Slide 42
Slide 42 text
Rates of change
domain
macro
micro
Slide 43
Slide 43 text
Rates of change
domain
macro
micro
Freedom to choose tools and technology by nailing domain and protocols
Slide 44
Slide 44 text
Implications at
Scale
Slide 45
Slide 45 text
Larger Systems of Systems
Slide 46
Slide 46 text
How long does it take to get 1 line
of code to production
Slide 47
Slide 47 text
Larger Systems of Systems
Slide 48
Slide 48 text
Mistaking Modules For Systems
Slide 49
Slide 49 text
Systems as the unit of work
If you can’t add a person to
your project then you are
failing worse than you think
again
An Aside
Slide 50
Slide 50 text
Ģ
B
ģ
z
SYSTEMS
in
ACTION
Slide 51
Slide 51 text
CMSs hold customer data hostage
CONTENT ANALYSIS & DATA MINING PROJECT
Free data
Analyse data
Derive data
CMSs tied to horrible platforms
Goals
Constraints
Slide 52
Slide 52 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
Slide 53
Slide 53 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
JSP Java
Slide 54
Slide 54 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
1 JSP Java
Slide 55
Slide 55 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
Java
1 JSP
Slide 56
Slide 56 text
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
1 JSP Java
Java
Scala
Slide 57
Slide 57 text
WHAT I IMAGINED
DATA
WEB
CORE LEECH PORTAL
CONTENT ANALYSIS & DATA MINING PROJECT
Slide 58
Slide 58 text
WEB / CORE / DATA / LEECH
WEBSHERE
REALITY SETTING IN
Slide 59
Slide 59 text
CAUGHT THINKING IN THE
SMALL
MODULES - LIBRARIES - TECH
Slide 60
Slide 60 text
We should leverage architecture changes
rather than force new technology
Slide 61
Slide 61 text
CONTENT
ANALYSIS
IDENTITY
CONTENT
STORE
QUERY
ENGINE
A Better Approach
Slide 62
Slide 62 text
Play 1.1 App
Our Starting Point
Slide 63
Slide 63 text
Play 1.1 App
Enabling Moves
IDENTITY
Slide 64
Slide 64 text
Play 1.1 App
Protobufs
Thrift
HTTP
Establishing Rules & Autonomy
sbt-assembly
OS package
Containers
IDENTITY
Protocols
Deployment
Slide 65
Slide 65 text
Play 1.1 App
Identifying Demand
CONTENT
STORE
IDENTITY
Slide 66
Slide 66 text
Play 1.1 App
Validation
CONTENT
STORE
IDENTITY
Slide 67
Slide 67 text
Play 1.1 App
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
Slide 68
Slide 68 text
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
QUERY
ENGINE
Slide 69
Slide 69 text
architecture is an everyday task
Slide 70
Slide 70 text
All editors are bad, really bad
ONLINE CONTENT EDITING
Something aaS
APIs
Ready to change
Small team, tight deadline
Goals
Constraints
Lots of ideas, not sure what would work
C Scala Scala
Spelling Identity Templates
Editor
Customer
UI
Micro Architecture
Scala/JS Scala
Postgres
Postgres Postgres
syslog
jail per service
rc.d
1 executable
Minimal stack
Slide 74
Slide 74 text
Editor was working well
ONLINE CONTENT EDITING
Incremental
Multiple Versions
Add not modify
Needed to add more services
Goals
Constraints
System need to keep running
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Scala/JS Scala JS
C Scala Scala Scala
multilog
jail per service
daemon tools
1 executable
Minimal stack
Slide 78
Slide 78 text
Demand for standalone features
ONLINE CONTENT EDITING
Quick
Minimal Overhead
Maximum Reuse
Not tied to our editor
Goals
Constraints
Run in parallel to current system