MicroServices Challenges - Fred George - Agile SG 2016

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Fred George Copyright © 2015 by Fred George. All rights reserved. 2 • Technologist • Computer networks – 70’s • Token Ring LAN – 80’s • GUI’s – late 80’s • OO – late 80’s • Agile – late 90’s • MicroServices – mid-2000’s • Programmer • Since 1968 (Basic) • 65,000 hours experience • 70+ languages • Computer Science Degree 1973 • Manager • 17 years IBM • Business degree, MIT Sloan School 1986 • Product Owner, IBM • VP, ThoughtWorks • Co-founder, Outpace (Silicon Valley) • Senior Advisor to 3 tech companies

Slide 3

Slide 3 text

Fred George Copyright © 2015 by Fred George. All rights reserved. 3 Consulting Roles • Change Agent • Disruptor • “Hand grenade I am throwing into development” • CTO describing Fred to his Vice President

Slide 4

Slide 4 text

Industry Trends… Copyright © 2015 by Fred George. All rights reserved. 4 MicroServices Cloud Docker Dev/Ops Full-stack developer Agile Lean startup Cassandra Event bus MVP (minimum viable product Programmer anarchy No-SQL One Hacker Way

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Why: New Problem Domains: The Cynefin Framework Copyright © 2015 by Fred George. All rights reserved. 6 Simple Complicated Complex Chaotic Cause a Effect Cause aaa Effect Effect? Cause? Cause ? Effect Disorder

Slide 7

Slide 7 text

Why: Competition is coming Copyright © 2015 by Fred George. All rights reserved. 7 • Enablement through technologies • Cloud computing • Ubiquitous, high bandwidth • Languages (and supporting frameworks) • Recognition of business opportunities • Silicon Valley innovators as role models • Accelerating business needs • Few inhibitors for global competitors • Reduction of entry barriers for niche competitors

Slide 8

Slide 8 text

Hardware Lead Times Copyright © 2015 by Fred George. All rights reserved. 8 6 mo. 1990 1995 2005 30 min. 1 week 1 day 2015 1 mo. 2000 2010 5 min. 5 sec. Order lead times (log) Data Center Virtual Machines Docker Commercial Cloud Capacity planning? Dedicated ops team?

Slide 9

Slide 9 text

Copyright © 2015 by Fred George. All rights reserved. 9 10 1 1000 100 100,000 1 10 100 1000 10,000 Size (loc) Number of Services MicroServices Rails / Java Zone SOA Workshop Forward Netflix Services Impact

Slide 10

Slide 10 text

Summary Principles of MicroServices Copyright © 2015 by Fred George. All rights reserved. 10 • Very, very small • Team size of one to develop/maintain • Loosely coupled (including flow) • Multiple versions acceptable (encouraged?) • Self-monitoring of each service • Publish interesting “stuff” (w/o explicit requirements) • “Application” seems to be a poor conceptualization

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Chad Fowler vs. Fred George •Chad: Use Synchronous as default • Algorithms typically described serially • Programmer understanding / productivity •Fred: Use Asynchronous as default • Robustness should be primary goal • Supports better de-coupling • Teach the programmers! Copyright © 2015 by Fred George. All rights reserved. 13

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Asynchronous Services Example Copyright © 2015 by Fred George. All rights reserved. 17 Bus Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login ? ? ?

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

B Copyright © 2015 by Fred George. All rights reserved. 23 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A A B Sally logs in as Member

Slide 24

Slide 24 text

B Copyright © 2015 by Fred George. All rights reserved. 24 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A A B Sally logs in as Member

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Channel Interfaces Service Taxonomy Channel Interfaces Copyright © 2015 by Fred George. All rights reserved. 30 Need Processing Enrichment Services Solution Services Client Biases Solution Blockers Solution Selection Logging / Monitoring / KPI’s

Slide 31

Slide 31 text

Slide 32

Slide 32 text

•Fortune 100 view: • Entity-oriented; consistency essential • As few as possible •MicroService view: • DB per MicroService (if persistence needed) • Poly-glot (various NoSQL, SQL) + event buses • Few (10%) writable; even fewer transactional How Many Databases? Copyright © 2015 by Fred George. All rights reserved. 32

Slide 33

Slide 33 text

Copyright © 2015 by Fred George. All rights reserved. 33 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A B Rental offers Event publishing Solution collecting 300ms responder Redis

Slide 34

Slide 34 text

Copyright © 2015 by Fred George. All rights reserved. 34 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A B Membership enrichment ETL Key-value

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

70’s Design Approach Copyright © 2016 by Fred George. All rights reserved. 37 ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz

Slide 38

Slide 38 text

~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ 90’s Design Approach Copyright © 2016 by Fred George. All rights reserved. 38 Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ Operational Reporting

Slide 39

Slide 39 text

Design Approach Copyright © 2016 by Fred George. All rights reserved. 39 … but you can’t change the data structure without making all the code “suspect” Corollary ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz Zzzzzzzz Zzzz Zzzzzz Zzz zzzzzz ~~~~~~~~ ~~~~~~~~ ~~~~~~~~

Slide 40

Slide 40 text

Slide 41

Slide 41 text

•Conceptualization (Job) • Every service has one job • If two jobs, make two services •Communication • Minimize messages (whether RESTful or Events) •Encapsulate • Service has its own persistence • If sharing persistence, just one logical service MicroServices Like OO Copyright © 2015 by Fred George. All rights reserved. 41

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

3 Companies – 3 Variants Copyright © 2015 by Fred George. All rights reserved. 46 300+ 3 + dozens 25 Ruby, Node.js, Clojure, R, … Clojure,Node.js Clojure DB w cron, RESTful, Kafka bus RabbitMQ Orchestration, RabbitMQ

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Copyright © 2015 by Fred George. All rights reserved. 54 MicroServices Cloud Docker Dev/Ops Full-stack developer Agile Lean startup Cassandra Event bus MVP (minimum viable product Programmer anarchy No-SQL

Slide 55

Slide 55 text

Slide 56

Slide 56 text

•Synchronicity degree • Primary API access to services •Ratio of number of services and average size • Zones for clarity •DB / service ratio • Expose potential DB hindrances to rapid deployment Taxonomy May be Useful (before it’s too late) Copyright © 2015 by Fred George. All rights reserved. 56

Slide 57

Slide 57 text

Count / Size Ratio Copyright © 2015 by Fred George. All rights reserved. 57 10 1 1000 100 100,000 1 10 100 1000 10,000 Size (loc) Number of Services MicroServices Rails / Java Zone SOA Animation Forward Netflix

Slide 58

Slide 58 text

More Information… •Google: • MicroService Architecture for videos • MicroXchg 2015 conference •Complementary topic: • Google for Programmer Anarchy Copyright © 2015 by Fred George. All rights reserved. 58

Slide 59

Slide 59 text