5 ways we screwed up Microservices

Lessons learned while adopting Microservices at Yammer

Transcript

1. Hi!  My  name  is  Gonzalo  and  we’re  going  to  go  over  some  challenges  we  had  while  
   adop:ng  an  SOA  architecture.  
   1  
   

   View Slide

2. View Slide

3. View Slide

4. View Slide

5. This  is  our  main  page  
   5  
   

   View Slide

6. We  use  a  lot  of  services,  they  are  micro.  And  that’s  it.  
   6  
   

   View Slide

7. The  end.  
   7  
   

   View Slide

8. Well,  not  exactly,  we  had  some  early  mistakes  
   8  
   

   View Slide

9. *  We  didn’t  use  Circuit  breakers  at  the  start,  so  we  didn't  have  Back  Pressure.  
   *  We  had  unbounded  queues  (Which  can  cause  RabbitMQ  to  run  out  of  memory)  
   9  
   

   View Slide

10. *  We  Thought  all  services  had  the  same  storage  requirements  and  created  a  shared  
    AP  storage  that  could  get  saturated  by  a  single  service...  
    *  We  are  not  a  db  company,  so  our  custom  sharing/rou:ng  logic  wasn’t  the  best.  
    *  Our  consistent  hashing  didn’t  spread  par::ons  enough  and  created  hotspots.  
    *  Sharing  storage  also  complicates  tracking  state  changes  and  makes  database  
    migra:ons  a  nightmare.  
    10  
    

    View Slide

11. *  JSON  is  awesome  and  lightweight…  But  it's  schema-­‐less,  so  you  can't  deprecate  old  
    endpoints  easily  and  you  will  need  to  create  new  endpoints  for  adding  new  required  
    fields.  
    •  Integra:on  tes:ng  is  vital  
    11  
    

    View Slide

12. *  Created  some  Mini-­‐Monoliths  
    *  Model  dependencies  are  bad,  Services  dependencies  are  ten  :mes  worse  (SLA  
    degrades  exponen:ally  as  your  dependencies  increase)  If  you  have  two  services  at  
    99.99%  availability  now  you  are  down  to  99.95%  (Note:  These  numbers  are  made  up,  
    but  some:mes  it  is  that  bad.)    
    *  Graceful  degrada:on  should  not  be  an  aferthought.  
    *  We  didn’t  enforce  strict  boundaries  which  created  dependencies  on  their  internals.  
    This  creates  fragile  services.  
    12  
    

    View Slide

13. *  We  didn’t  have  consistent  metrics  
    *  Our  dashboards  were  slow  
    *  We  didn’t  have  aggregated  logs  
    *  Tracing  issues  across  services  is  hard.  
    13  
    

    View Slide

14. View Slide

15. View Slide

16. Microservices  help  you  deploy  fixes  faster  and  scale  beger.  
    16  
    

    View Slide

17. View Slide

18. Building  Microservices  by  Sam  Newman  
    18  
    

    View Slide

19. View Slide

20. And  watch  this  Big  Ruby  talk  by  Brian  Morton.  
    hgps://www.youtube.com/watch?v=GuJ49PNBsn8  
     
    20  
    

    View Slide

21. View Slide