Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud Chaos and Microservices Mayhem

Cloud Chaos and Microservices Mayhem

The way we use Java has changed. Applets - or any browser-side Java - are a long, long, time ago. Java moved to the back end, and stayed there, but the back-end has gotten bigger, more distributed, and more complicated. Some things that were a good idea ten years ago turn out to be a terrible idea in the cloud; and what used to be ‘good enough’ for testing really isn’t anymore. Managing microservices architecture demands a lot of us, to ensure observability, operational resiliency, and organisational agility. This talk will introduce some of the new tools, patterns, and best practices for Java development in 2022. It also gives a tour of some of the most painful anti-patterns Holly has seen as a consultant with the IBM Garage.

Holly Cummins

October 03, 2022
Tweet

More Decks by Holly Cummins

Other Decks in Programming

Transcript

  1. None
  2. cloud chaos and microservices mayhem Holly Cummins Red Hat @holly_cummins

    GOTO Copenhagen
  3. None
  4. None
  5. #RedHat @holly_cummins things you need to do well in 2022:

  6. #RedHat @holly_cummins handwashing things you need to do well in

    2022:
  7. #RedHat @holly_cummins handwashing apps things you need to do well

    in 2022:
  8. #RedHat @holly_cummins handwashing apps ops things you need to do

    well in 2022:
  9. #RedHat @holly_cummins handwashing apps ops devops things you need to

    do well in 2022:
  10. #RedHat @holly_cummins handwashing apps ops devops devsecops things you need

    to do well in 2022:
  11. #RedHat @holly_cummins handwashing apps ops devops devsecops finops things you

    need to do well in 2022:
  12. None
  13. None
  14. #RedHat @holly_cummins a few

  15. #RedHat @holly_cummins

  16. #RedHat @holly_cummins technology changes fast (duh)

  17. #RedHat @holly_cummins technology changes fast (duh) … but it’s taking

    us a while to catch up to cloud
  18. #RedHat @holly_cummins technology changes fast (duh) … but it’s taking

    us a while to catch up to cloud
  19. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins tracing

  20. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins observability

  21. #RedHat @holly_cummins hardware old way logs

  22. #RedHat @holly_cummins hardware old way cloud way logs container logs

  23. #RedHat @holly_cummins hardware old way cloud way logs logs

  24. #RedHat @holly_cummins hardware old way cloud way logs oops

  25. #RedHat @holly_cummins hardware old way cloud way logs container logs

  26. #RedHat @holly_cummins hardware old way cloud way logs container logs

  27. #RedHat @holly_cummins hardware old way cloud way logs logs

  28. #RedHat @holly_cummins hardware old way cloud way logs logs container

    logs container logs container logs container logs
  29. #RedHat @holly_cummins hardware old way cloud way logs logs container

    logs container logs container logs container logs
  30. #RedHat @holly_cummins hardware old way cloud way logs logs container

    logs container logs container logs container logs aggregation + correlation
  31. #RedHat @holly_cummins hardware old way cloud way logs logs container

    logs container logs container logs container logs aggregation + correlation open telemetry (open tracing)
  32. #RedHat @holly_cummins

  33. #RedHat @holly_cummins good news: observability is ‘fixed’

  34. #RedHat @holly_cummins good news: observability is ‘fixed’ bad news: not

    all of us are using it
  35. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins performance

    requirements
  36. #RedHat @holly_cummins memory footprint is money startup time is money

  37. #RedHat @holly_cummins all that dynamism we worked so hard is

    (now) a bad idea
  38. #RedHat @holly_cummins ahead of time optimisation is (now) a good

    idea
  39. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins performance

    optimisation
  40. #RedHat @holly_cummins

  41. #RedHat @holly_cummins garbage collection and microservices

  42. #RedHat @holly_cummins garbage collection and microservices

  43. #RedHat @holly_cummins garbage collection and microservices

  44. #RedHat @holly_cummins garbage collection and microservices

  45. #RedHat @holly_cummins garbage collection and microservices if you chain 100

    microservices together, you are almost guaranteed a GC pause
  46. #RedHat @holly_cummins

  47. #RedHat @holly_cummins

  48. #RedHat @holly_cummins the JVM tunes itself differently on servers

  49. #RedHat @holly_cummins the JVM tunes itself differently on servers …

    but what’s a server?
  50. #RedHat @holly_cummins answer(ish):

  51. #RedHat @holly_cummins // This is the working definition of a

    server class machine: // >= 2 physical CPU's and >=2GB of memory answer(ish): actual hotspot source code
  52. #RedHat @holly_cummins

  53. #RedHat @holly_cummins takeaway: don’t shrink your containers too much or

    your JVM will make bad decisions advice accurate as of June 2022. all performance advice should be independently verified. do not try this at home without testing first. your mileage may vary.
  54. #RedHat @holly_cummins wait, isn’t that the opposite of the advice

    in the previous section? should my containers be big or small?
  55. #RedHat @holly_cummins this is the chaos part :) wait, isn’t

    that the opposite of the advice in the previous section? should my containers be big or small?
  56. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins management

  57. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins keeping

    track of hardware
  58. #RedHat @holly_cummins The cloud makes it so easy to provision

    hardware.
  59. #RedHat @holly_cummins That doesn’t mean the hardware is free.

  60. #RedHat @holly_cummins That doesn’t mean the hardware is free.

  61. #RedHat @holly_cummins That doesn’t mean the hardware is free. Or

    useful.
  62. #RedHat @holly_cummins That doesn’t mean the hardware is free. Or

    useful. Or shuts itself off.
  63. #RedHat @holly_cummins Hey boss, I created a Kubernetes cluster.

  64. #RedHat @holly_cummins Hey boss, I created a Kubernetes cluster. I

    forgot it for 2 months.
  65. #RedHat @holly_cummins Hey boss, I created a Kubernetes cluster. I

    forgot it for 2 months. … and it’s €1000 a month.
  66. Photo by daveynin, flickr

  67. #RedHat @holly_cummins 2017 survey 25% of 16,000 servers doing no

    useful work
  68. #RedHat @holly_cummins 2017 survey 25% of 16,000 servers doing no

    useful work “perhaps someone forgot to turn them off”
  69. #RedHat @holly_cummins finops figuring out who in your company forgot

    to turn off their cloud
  70. #RedHat @holly_cummins companies that went out of business for cloud

    cost accidents
  71. @holly_cummins #RedHat

  72. #RedHat @holly_cummins (See Troy tomorrow at GOTO !)

  73. #RedHat @holly_cummins (See Troy tomorrow at GOTO !)

  74. #RedHat @holly_cummins $11,448.30 (See Troy tomorrow at GOTO !)

  75. #RedHat @holly_cummins “I did find a setting on your plan

    that set the max cacheable file size to 15GB and it looks like your zipfile is 18GB big” $11,448.30 (See Troy tomorrow at GOTO !)
  76. #RedHat @holly_cummins

  77. #RedHat @holly_cummins maybe this is the chaos part

  78. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins releasing

  79. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins releasing

  80. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins microservices

  81. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins microservices

  82. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins

  83. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “we’re

    moving too slowly.
  84. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “we’re

    moving too slowly. we should modernise our COBOL application into microservices.
  85. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “we’re

    moving too slowly. we should modernise our COBOL application into microservices. … but our release board only meets twice a year.”
  86. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins modularity

  87. #RedHat @holly_cummins microservices are not the goal

  88. #RedHat @holly_cummins microservices are not the goal they are the

    means
  89. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins wishful

    mimicry
  90. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins cloud

    != microservices
  91. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins cloud

    native != microservices
  92. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “every

    time we touch one microservice, all the others break.”
  93. #RedHat @holly_cummins distributed monolith

  94. #RedHat @holly_cummins distributed monolith but without compile-time checking … or

    guaranteed function execution
  95. #RedHat @holly_cummins distributed monolith but without compile-time checking … or

    guaranteed function execution
  96. #RedHat @holly_cummins distributed monolith but without compile-time checking … or

    guaranteed function execution
  97. #RedHat @holly_cummins distributed monolith but without compile-time checking … or

    guaranteed function execution
  98. #RedHat @holly_cummins “each of our microservices has duplicated the same

    object model … with twenty classes and seventy fields”
  99. #RedHat @holly_cummins Microservice Domain

  100. #RedHat @holly_cummins Microservice Domain

  101. #RedHat @holly_cummins Microservice Domain (this is bad)

  102. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins distributed

    != decoupled
  103. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “ohhhh,

    we weren’t expecting your service to do that … “
  104. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins “uh,

    what do you mean you corrected the typo in your json?”
  105. #IBMGarage + IBM Cloud © 2020 IBM Corporation @holly_cummins If

    you’re going to do microservices, you need to get good at automation. And testing.
  106. IBM Cloud © 2020 IBM Corporation

  107. IBM Cloud © 2020 IBM Corporation the test pyramid

  108. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end

    tests
  109. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end

    tests unit tests
  110. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end

    tests unit tests contract tests
  111. IBM Cloud © 2020 IBM Corporation the test pyramid (you

    can TDD at every level!) end-to-end tests unit tests contract tests
  112. #RedHat @holly_cummins How to test a fire alarm?

  113. IBM Cloud © 2020 IBM Corporation how not to test

    a fire alarm
  114. IBM Cloud © 2020 IBM Corporation how not to test

    a fire alarm
  115. #RedHat @holly_cummins

  116. #RedHat @holly_cummins unit testing a fire alarm

  117. #RedHat @holly_cummins uh … is that enough?

  118. #RedHat @holly_cummins contract testing a fire alarm

  119. #RedHat @holly_cummins contract testing a fire alarm

  120. @holly_cummins to the code!

  121. #RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

  122. #RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

  123. #RedHat @holly_cummins demo recap: • consumer-driven contract testing can save

    your bacon • pact is a mock for the consumer • pact is a functional test for the producer • shared json contracts aligns expectations across services
  124. #RedHat @holly_cummins so … don’t assume cloud is a solved

    problem don’t assume microservices will solve all your problems contract testing may solve some of your problems
  125. thank you @holly_cummins slides

  126. slides