Load Balancing: A strategy for scalable internet applications

Transcript

1. Load Balancing Pierre Goudjo A strategy for scalable internet applications
3. Scalability
4. Wikipedia Scalability is the property of a system to handle a growing amount of work by adding resources to the system
6. High availability
7. Wikipedia High availability is a characteristic of a system which aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.
8. Availability is usually expressed as a percentage of uptime in a given year
9. Load balancing
10. Wikipedia Load balancing refers to the process of distributing a set of tasks over a set of resources (computing units), with the aim of making their overall processing more efficient.
12. Software/hardware load balancers
13. Hardware load balancer
14. Hardware load balancer Pros • E ffi cient • Rich and deep feature set • Lower salary costs
15. Hardware load balancer Cons • Proprietary hardware appliances • Over-provisioned • Expensive
17. Software load balancer
18. Software load balancer Pros • Run on commodity servers • Cheaper than hardware solutions • Easier to maintain • Easier to scale up or down
19. Software load balancer Cons • Can be trickier to setup • User responsible of underlying hardware
22. Loading balancing can occur at various levels of the OSI model
23. TCP - UDP
24. https://www.nginx.com/resources/glossary/layer-4-load-balancing/ A Layer 4 load balancer bases the load-balancing decision on the source and destination IP addresses and ports recorded in the packet header, without considering the contents of the packet.
25. HTTP - RTMP - …
26. https://www.nginx.com/resources/glossary/layer-7-load-balancing/ Layer 7 load balancing operates at the high ‑ level application layer, which deals with the actual content of each message
27. HTTP is the most used Layer 7 protocol for website traffic on the internet
28. Layer 4 vs Layer 7 load balancing • Layer 4 load balancing is faster than Layer 7 • Layer 4 has visibility on the network information such as port and protocol • All protocols based on TCP or UDP are supported by Layer 4 load balancer • Layer 7 load balancer has application awareness and can make complex load balancing decisions based on messages content • Layer 7 load balancer are usually protocol-centric
29. There are two primary approches to load balancing
30. Static load balancing
31. Cloud fl are docs A load balancing algorithm is "static" when it does not take into account the state of the system for the distribution of tasks.
32. Round-Robin algorithm
33. Weighted Round-Robin algorithm A=3,B=2,C=1
34. Consistent Hash based algorithm A=3,B=2,C=1
35. Hashing Methods • URL Hash Method • Source IP Hash Method • Header Hash Method
36. Random algorithm 5 2 3 4 6 1
37. Dynamic load balancing
38. Cloud fl are docs Dynamic load balancing uses algorithms that take into account the current state of each server and distribute traffic accordingly.
39. Least connections algorithm
42. Amazon Elastic Load Balancer
43. Elastic Load Balancing automatically distributes incoming application traffic across multiple targets
44. Load balancing EC2
45. Load balancing Lambda
46. Load balancing containers
47. Load balancing IPs
48. ELB choose targets from a target group
49. Target can register themselves into a target group and be able to receive tra ff i c
50. Target registers the port they can be reached on
51. A load balancer can have multiple listener routing to di ff erent target groups
52. ELB nodes are inside an availability zone
53. A target group can span multiple Availability zones
54. Default behaviour of ELB on load repartition
55. Default ELB behaviour regarding targets in di ff erent availability zone
56. Load balancing behaviour when cross-zone load balancing is activated
57. Tra ff i c still get routed to the other AZ even if there is no ELB node up in that zone
58. Why not activate cross-zone load balancing all the time?
59. Cross-zone load balancing can cause regional data transfer
60. ELB Load balancer types
61. Application Load Balancer
62. • Layer 7 load balancer • Slow start • Protocol listeners for • HTTP/1 • HTTP/2 • gRPC • WebSockets • Sticky sessions • TLS o ff l oading • Server Name Indication (SNI) with multiple certi fi cates • Lambda functions as target Application Load Balancer
63. • Listener rules actions • Fixed responses • Redirect • Forward • Listener rules conditions (5 / rules) • host headers • http request method • Source IP • Path pattern • Query string Application Load Balancer Layer 7 Load Balancing
64. Network Load Balancer
65. • Layer 4 load balancer • Protocol listeners for • TCP • TLS • UDP • Sticky sessions • TLS o ff l oading • Static IP support • Elastic IP support • Connection draining Network Load Balancer
66. Gateway Load Balancer
67. • Layer 3 load balancer • Used to load balance tra ffi c through virtual appliances • Egress and Ingress fl ow through the virtual appliances and can be dropped • Is a simpler way to use virtual appliances available on AWS Marketplace Gateway Load Balancing
69. • fi rewalls • intrusion detection systems • prevention systems • deep packet inspection systems Third-party virtual appliances