Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Load Balancing: A strategy for scalable internet applications

Load Balancing: A strategy for scalable internet applications



January 20, 2022

More Decks by Pierre GOUDJO

Other Decks in Technology


  1. Load Balancing Pierre Goudjo A strategy for scalable internet applications

  2. None
  3. Scalability

  4. Wikipedia Scalability is the property of a system to handle

    a growing amount of work by adding resources to the system
  5. None
  6. High availability

  7. Wikipedia High availability is a characteristic of a system which

    aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.
  8. Availability is usually expressed as a percentage of uptime in

    a given year
  9. Load balancing

  10. Wikipedia Load balancing refers to the process of distributing a

    set of tasks over a set of resources (computing units), with the aim of making their overall processing more efficient.
  11. None
  12. Software/hardware load balancers

  13. Hardware load balancer

  14. Hardware load balancer Pros • E ffi cient • Rich

    and deep feature set • Lower salary costs
  15. Hardware load balancer Cons • Proprietary hardware appliances • Over-provisioned

    • Expensive
  16. None
  17. Software load balancer

  18. Software load balancer Pros • Run on commodity servers •

    Cheaper than hardware solutions • Easier to maintain • Easier to scale up or down
  19. Software load balancer Cons • Can be trickier to setup

    • User responsible of underlying hardware
  20. None
  21. None
  22. Loading balancing can occur at various levels of the OSI

  23. TCP - UDP

  24. https://www.nginx.com/resources/glossary/layer-4-load-balancing/ A Layer 4 load balancer bases the load-balancing decision

    on the source and destination IP addresses and ports recorded in the packet header, without considering the contents of the packet.
  25. HTTP - RTMP - …

  26. https://www.nginx.com/resources/glossary/layer-7-load-balancing/ Layer 7 load balancing operates at the high ‑

    level application layer, which deals with the actual content of each message
  27. HTTP is the most used Layer 7 protocol for website

    traffic on the internet
  28. Layer 4 vs Layer 7 load balancing • Layer 4

    load balancing is faster than Layer 7 • Layer 4 has visibility on the network information such as port and protocol • All protocols based on TCP or UDP are supported by Layer 4 load balancer • Layer 7 load balancer has application awareness and can make complex load balancing decisions based on messages content • Layer 7 load balancer are usually protocol-centric
  29. There are two primary approches to load balancing

  30. Static load balancing

  31. Cloud fl are docs A load balancing algorithm is "static"

    when it does not take into account the state of the system for the distribution of tasks.
  32. Round-Robin algorithm

  33. Weighted Round-Robin algorithm A=3,B=2,C=1

  34. Consistent Hash based algorithm A=3,B=2,C=1

  35. Hashing Methods • URL Hash Method • Source IP Hash

    Method • Header Hash Method
  36. Random algorithm 5 2 3 4 6 1

  37. Dynamic load balancing

  38. Cloud fl are docs Dynamic load balancing uses algorithms that

    take into account the current state of each server and distribute traffic accordingly.
  39. Least connections algorithm

  40. None
  41. None
  42. Amazon Elastic Load Balancer

  43. Elastic Load Balancing automatically distributes incoming application traffic across multiple

  44. Load balancing EC2

  45. Load balancing Lambda

  46. Load balancing containers

  47. Load balancing IPs

  48. ELB choose targets from a target group

  49. Target can register themselves into a target group and be

    able to receive tra ff i c
  50. Target registers the port they can be reached on

  51. A load balancer can have multiple listener routing to di

    ff erent target groups
  52. ELB nodes are inside an availability zone

  53. A target group can span multiple Availability zones

  54. Default behaviour of ELB on load repartition

  55. Default ELB behaviour regarding targets in di ff erent availability

  56. Load balancing behaviour when cross-zone load balancing is activated

  57. Tra ff i c still get routed to the other

    AZ even if there is no ELB node up in that zone
  58. Why not activate cross-zone load balancing all the time?

  59. Cross-zone load balancing can cause regional data transfer

  60. ELB Load balancer types

  61. Application Load Balancer

  62. • Layer 7 load balancer • Slow start • Protocol

    listeners for • HTTP/1 • HTTP/2 • gRPC • WebSockets • Sticky sessions • TLS o ff l oading • Server Name Indication (SNI) with multiple certi fi cates • Lambda functions as target Application Load Balancer
  63. • Listener rules actions • Fixed responses • Redirect •

    Forward • Listener rules conditions (5 / rules) • host headers • http request method • Source IP • Path pattern • Query string Application Load Balancer Layer 7 Load Balancing
  64. Network Load Balancer

  65. • Layer 4 load balancer • Protocol listeners for •

    TCP • TLS • UDP • Sticky sessions • TLS o ff l oading • Static IP support • Elastic IP support • Connection draining Network Load Balancer
  66. Gateway Load Balancer

  67. • Layer 3 load balancer • Used to load balance

    tra ffi c through virtual appliances • Egress and Ingress fl ow through the virtual appliances and can be dropped • Is a simpler way to use virtual appliances available on AWS Marketplace Gateway Load Balancing
  68. None
  69. • fi rewalls • intrusion detection systems • prevention systems

    • deep packet inspection systems Third-party virtual appliances