π A holistic approach
Project profile
Architectural challenges
Infrastructure challenges
Software challenges
Lessons learned
Foto von Agnieszka Kowalczyk auf Unsplash
Slide 4
Slide 4 text
(1) Architectural challenges
Slide 5
Slide 5 text
π Project
setting
Foto von Mike Petrucci auf Unsplash
Slide 6
Slide 6 text
π‘ Platform architecture
Slide 7
Slide 7 text
π³ Team system architecture
Slide 8
Slide 8 text
π³ Solr cluster topology
Split API & shop
traffic
API cluster
absorbes request
peaks
Medium cluster
utilization
π° 1k
per instance/month
Slide 9
Slide 9 text
π³ Solr Cluster topology
Goal: get rid of
redundant
infrastructure
How to enable Solr
to handle request
spikes?
Slide 10
Slide 10 text
(2) Operational challenges
Foto von Chris Lawton auf Unsplash
Slide 11
Slide 11 text
π¦Ύ ARM our lord and savior
Docker buildx to build
β¨
multi-arch builds
Custom arm64 AMI
Multi-Arch ASGs via
MixedInstances in LaunchTemplate
Graviton
AMD
Intel
Slide 12
Slide 12 text
π° ARM vs AMD
Graviton2
AMD
Linux load1
(less is better)
AMD power
management β‘
Slide 13
Slide 13 text
π Graviton2 vs Graviton3
Linux load1
(less is better)
Graviton3
Graviton2
Slide 14
Slide 14 text
π Our instance type journey
Slide 15
Slide 15 text
π± monitor
ecosystem
innovation
Slide 16
Slide 16 text
(3) Software Challenges
Foto von Alexander Hafemann auf Unsplash
Slide 17
Slide 17 text
π Why software challenges?
Response time is tied to cpu utilization π±
Foto von Jeremy Lapak auf Unsplash
Slide 18
Slide 18 text
π¨β Java Flight Recorder
Event based tracing framework built into the JVM
Very low overhead < 1%
Designed for production use
Free to use
Foto von Richard Cartmell auf Unsplash
π΄ G1 Garbage
Collector
Minimum pause
collector
200ms max
collection time
Uses 90-105ms for
a 32g heap
Slide 35
Slide 35 text
β» Z Garbage
Collector
Reduces
collection time
to 600Β΅s for a
32g heap π³
Uses Linux huge
pages
Slide 36
Slide 36 text
πΎ Lessons learned
β’ Have strong observability in
place
β’ Involve yourself in the OSS
software you use β yes maybe
you are the first one having
this exact problem
β’ Test in prod or live a lie
Slide 37
Slide 37 text
Dennis Berger (@debe)
Torsten KΓΆster (@tboeghk)
Questions?