Microservices architecture pitfalls

Slide 1

Slide 1 text

Microservices architecture pitfalls WJUG meeting ◦ march 2015 Mateusz Gajewski  Solutions Architect @ Allegro Twitter: @wendigo

Slide 2

Slide 2 text

About me given: I started working in Allegro in 2009 (5 mln AO, 50 devs) when: Allegro reached 40 mln AO, 400 devs then: I am Solutions Architect 2

Slide 3

Slide 3 text

Agenda • Microservices, microservices, microservices! ;) • Some challenges & pitfalls: • Architectural, • Operational, • Organisational 3

Slide 4

Slide 4 text

Let’s go back in time to year 2012 4

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Back then we wanted • agile development, • scalability, • resilience, • lower costs, • hybrid cloud. 6

Slide 7

Slide 7 text

Basically SOA + JVM was an answer! 7

Slide 8

Slide 8 text

But our system was too BIG & too complex to do it with existing enterprise solutions 8

Slide 9

Slide 9 text

s/Enterprise/OSS/g Solutions ;) 9

Slide 10

Slide 10 text

we’ve started to do *buzzword* 10

Slide 11

Slide 11 text

And now, literally everyone is doing microservices!!?? 11

Slide 12

Slide 12 text

Microservices by Fowler 12 Lots of *buzzwords* http://martinfowler.com/articles/microservices.html

Slide 13

Slide 13 text

SOA ≈ microservices? 13

Slide 14

Slide 14 text

microservices architecture ≈ ﬁne-grained SOA − enterprise (commercial) sh*t ≈ highly scalable, distributed system 14

Slide 15

Slide 15 text

Distributed systems • concurrency of components, • independent failure of components, • lack of a global clock. 15

Slide 16

Slide 16 text

The Eight Fallacies of Distributed Computing 16 by Peter Deutsch 1991

Slide 17

Slide 17 text

#1: Network is reliable 17

Slide 18

Slide 18 text

#2: Latency is zero 18

Slide 19

Slide 19 text

#3: Bandwidth is inﬁnite 19

Slide 20

Slide 20 text

#4: Network is secure 20

Slide 21

Slide 21 text

#5: Topology doesn’t change 21

Slide 22

Slide 22 text

#6: There is one administrator 22

Slide 23

Slide 23 text

#7: Transport cost is zero 23

Slide 24

Slide 24 text

#8: Network is homogeneous 24

Slide 25

Slide 25 text

distributed systems are hard →  microservices are much harder ;) 25

Slide 26

Slide 26 text

What have we learnt? 26

Slide 27

Slide 27 text

Act I: architectural constraints 27

Slide 28

Slide 28 text

CAP is not just theorem it’s reality against us 28

Slide 29

Slide 29 text

bye, bye ACID semantics 29

Slide 30

Slide 30 text

Long live BASE guarantees! Basically Available, Soft state, Eventually consistent 30

Slide 31

Slide 31 text

distributed transactions add complexity 31

Slide 32

Slide 32 text

it’s far cheaper to do compensation 32

Slide 33

Slide 33 text

33 http://bravenewgeek.com/you-cannot-have-exactly-once-delivery/

Slide 34

Slide 34 text

you need idempotent APIs and events sinks 34

Slide 35

Slide 35 text

35 choreography > orchestration

Slide 36

Slide 36 text

So we’ve built Hermes a.k.a circulatory system 36

Slide 37

Slide 37 text

network can be congested! 37

Slide 38

Slide 38 text

REST+JSON on top of HTTP/1.1  is ﬁne 38

Slide 39

Slide 39 text

REST+JSON on top of HTTP/2.0  with TLS is ﬁner 39

Slide 40

Slide 40 text

we don’t rely on network anymore net splits in public clouds happens everytime! 40

Slide 41

Slide 41 text

we adopted antifragile organization 41

Slide 42

Slide 42 text

Slide 43

Slide 43 text

powerful tandem 43 + Reactive programming Circuit breaker pattern

Slide 44

Slide 44 text

you need to support non- native old services, clients and systems 44

Slide 45

Slide 45 text

Slide 46

Slide 46 text

conclusion: constant architecture improvement 46

Slide 47

Slide 47 text

47 Act II: operational troubles

Slide 48

Slide 48 text

creating new service should be instant! 48

Slide 49

Slide 49 text

Slide 50

Slide 50 text

automation  with gradle & axions 50

Slide 51

Slide 51 text

Slide 52

Slide 52 text

so now we’ve got over 1800 repositories grouped under 250 projects 52

Slide 53

Slide 53 text

all with CI, code quality checks, security checks, integrated with sonar & artefact repository 53

Slide 54

Slide 54 text

but what with  services upgrades? 54

Slide 55

Slide 55 text

we’ve initially built our own service stack  … and it was ok - for a while 55

Slide 56

Slide 56 text

now we are extending spring-boot  with so called andamio project 56

Slide 57

Slide 57 text

rapid deployments integrated with CI/CD environment and canary tests are must-have 57

Slide 58

Slide 58 text

war ﬁles  ▾ scp + puppet ▾ golden images ▾ docker (immutable images) ▾ 58

Slide 59

Slide 59 text

frequency of changes →   automated  monitoring, logging   & operational insights 59

Slide 60

Slide 60 text

graphite statsd cabot tessera kibana logstash zabbix newrelic selena pingdom … 60

Slide 61

Slide 61 text

Monitoring As A Service + SLA Monitoring + 61

Slide 62

Slide 62 text

we need to build real-time anomaly detection soon 62

Slide 63

Slide 63 text

63 Act III: organizational shift

Slide 64

Slide 64 text

strategic DDD is good for splitting up monolith 64

Slide 65

Slide 65 text

but leave tactical DDD up to teams 65

Slide 66

Slide 66 text

huge polyglot hangover 66

Slide 67

Slide 67 text

acquiring distributed skills 67

Slide 68

Slide 68 text

you build it - you run it 68

Slide 69

Slide 69 text

coupling avoidance 69

Slide 70

Slide 70 text

please don’t audit me 70

Slide 71

Slide 71 text

distributed (micro) data curation 71

Slide 72

Slide 72 text

So after two years… 72

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Final thoughts 74

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Thanks! Any questions? Visit our blog: allegrotech.io Follow us on twitter: @allegrotechblog Check our OSS projects: github.com/allegro And meetup group: meetup.com/allegrotech 78