How Cloudflare deals with largest DDoS attacks?

Transcript

1. How Cloudflare deals with
   largest DDoS attacks
   Marek Majkowski @majek04
   

   View Slide

2. View Slide

3. Reverse proxy
   3
   Eyeball Reverse proxy Origin server
   • Optimizations
   • Caching
   • DDoS protection
   • Security
   

   View Slide

4. View Slide

5. View Slide

6. How to perform an attack?
   6
   

   View Slide

7. 7
   X
   Attacker Visitor
   Target
   Bots
   

   View Slide

8. 1. Get a cool costume
   8
   

   View Slide

9. 9
   2. Fix your keyboard
   

   View Slide

10. 3. Get a cool name
    10
    • Anonymous
    • Lizard Squad
    • WireX
    • Syrian Electronic Army
    • vDOS
    

    View Slide

11. 4. Choose attack type
    • L3 Amplification (requires IP Spoofing)
    • L3 Direct
    • L7 - repetitive
    • L7 - smart
    11
    

    View Slide

12. Attack types
    • Amplification
    • SYN
    • DNS
    • HTTP
    • HTTPS
    12
    ⟯
    ⟯
    OSI L3 (source IP irrelevant)
    OSI L7 (established TCP/IP)
    

    View Slide

13. Attack types
    13
    IP Spoofing Large Easy to block
    Easy to
    implement
    L3 Amplification required YES yes yes
    L3 Direct desired no depends depends
    L7 repetitive no yes yes
    L7 smart no no no
    

    View Slide

14. 14
    more bots == better
    

    View Slide

15. 5. Choose infection vector
    • Ask users - hacktivism
    • Windows XP zombies
    • VPS Servers - Shellshock
    • CPE routers - Mirai
    • Android phones - WireX
    15
    

    View Slide

16. Infection vectors
    16
    IP Spoofing Plenty? Infection software
    Windows XP
    maybe, hard
    to maintain
    acquire? Zeus
    VPS Servers sometimes internet scan ?
    CPE Routers sometimes
    new
    vulnerability
    custom
    Android phones yes phishing custom
    

    View Slide

17. 17
    https://www.fbi.gov/wanted/cyber
    

    View Slide

18. View Slide

19. Five case studies
    19
    

    View Slide

20. 1. Asia direct
    20
    

    View Slide

21. View Slide

22. 22
    Target
    Server
    Attacker
    510 Gbps
    Direct SYN flood
    

    View Slide

23. View Slide

24. 24
    1 in 10000 packets
    

    View Slide

25. Internet economics
    25
    Tier 1
    provider
    Attacking
    network
    Target
    network
    $
    $
    

    View Slide

26. 26
    Spoofed?
    (source: DaPuglet)
    

    View Slide

27. 27
    5.6.7.8
    8.8.8.8
    IP Spoofing
    

    View Slide

28. 28
    Enables impersonation
    Real
    8.8.8.8 Destination
    5.6.7.8
    Spoofed
    8.8.8.8
    

    View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. Asia direct: Profile
    • Direct SYN floods
    • VERY big - 510Gbps, 300Mpps
    • Hitting small number of pops
    • Capable of IP Spoofing
    • Attacker: ????
    • Infection: Servers
    37
    

    View Slide

38. Is it a day job?
    38
    

    View Slide

39. Asia: mitigation
    39
    iptables -A INPUT \
    --dst 1.2.3.4 \
    -p udp --dport 53 \
    -m bpf --bytecode "14,0 0 0 20,177 0 0 0,12 0 0 0,7
    0 0 0,64 0 0 0,21 0 7 124090465,64 0 0 4,21 0 5
    1836084325,64 0 0 8,21 0 3 56848237,80 0 0 12,21 0 1
    0,6 0 0 1,6 0 0 0" \
    -j DROP
    

    View Slide

40. BPF Bytecode
    40
    ldx 4*([14]&0xf)
    ld #34
    add x
    tax
    lb_0:
    ldb [x + 0]
    add x
    add #1
    tax
    ld [x + 0]
    jneq #0x07657861, lb_1
    ld [x + 4]
    jneq #0x6d706c65, lb_1
    ld [x + 8]
    jneq #0x03636f6d, lb_1
    ldb [x + 12]
    jneq #0x00, lb_1
    ret #1
    lb_1:
    ret #0
    

    View Slide

41. View Slide

42. 42
    Internet Router
    NIC Kernel App
    XDP Iptables
    

    View Slide

43. Application integration
    43
    1.2.3.4
    1.2.3.5
    1.2.3.6
    dig A example.com
    1.2.3.7
    X
    

    View Slide

44. 2. Asia amplification
    44
    

    View Slide

45. View Slide

46. 46
    UDP Server
    UDP Client
    request response
    

    View Slide

47. 47
    Attacker
    Target
    UDP Server
    request
    response
    

    View Slide

48. 48
    Attacker
    Target
    UDP Server
    request
    response
    10 bytes
    100 bytes
    

    View Slide

49. 49
    Attacker
    Target
    UDP Servers
    requests
    responses
    

    View Slide

50. View Slide

51. 51
    192.0.2.0/24
    Internet
    Los Angeles
    192.0.2.0/24
    London
    192.0.2.0/24
    Amsterdam
    192.0.2.0/24
    Moscow
    192.0.2.0/24
    San Jose
    192.0.2.0/24
    New York
    

    View Slide

52. View Slide

53. March 2013: Spamhaus
    53
    300 Gbps of traffic
    27 Gbps of spoofing
    Exposed
    DNS Resolvers
    

    View Slide

54. View Slide

55. 55
    Mbps - AMP sport=1900 min:406 avg:22608 med=4663 max:186124
    dev:35565 count:1319
    Mbps - AMP sport=1900:
    value |-------------------------------------------------- count
    134 | 0
    268 | *************************************** 145
    536 |************************************************** 182
    1073 | ******************************************** 163
    2147 | ******************************************* 157
    4294 | *********************************** 131
    8589 | ************************ 88
    17179 | ************************************************ 176
    34359 | ************************************ 133
    68719 | ******************************** 117
    137438 | ******* 27
    

    View Slide

56. Asia SSDP: Profile
    • Amplification SSDP attakcs
    • Pretty large - 186Gbps
    • Hitting LARGE number of pops
    • (we know) source is in Asia, Interesting targets
    • Requires IP Spoofing
    • Attacker: ????
    • Infection: Servers
    56
    

    View Slide

57. View Slide

58. View Slide

59. 59
    https://badupnp.benjojo.co.uk/
    

    View Slide

60. 3. IoT - connected cameras
    60
    

    View Slide

61. View Slide

62. View Slide

63. View Slide

64. 64
    IP reputation
    (source: the internet)
    

    View Slide

65. View Slide

66. View Slide

67. 67
    GET /en HTTP/1.1
    User-Agent:
    Cookie:
    Host: example.com
    Connection: close
    Content-Length: 800000
    a[]=&b[]=&a[]=&b[]=&a[]=&b[]=&a[]=&b[]=&a[]=&b[]=&a[]=&b[]=...
    

    View Slide

68. IoT: Profile
    • Repetitive L7 attacks
    • Pretty large - 700k rps, 400Gbps,
    • 128k source IP's
    • Ukraine, Vietnam
    • Attacker: ????
    • Infection: CCTV Botnet
    68
    

    View Slide

69. 69
    Internet Router
    NIC Kernel App
    conntrack
    iptables fingerprints
    

    View Slide

70. 4. Porcupine
    70
    

    View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. Porcupine: Profile
    • Junk payload L7 attacks
    • Pretty large - 1M rps, 200k IP's/h
    • Brasil, Algeria, Tunisia, Ukraine
    • Attacker: .
    • Infection: .
    76
    

    View Slide

77. View Slide

78. 5. WireX
    78
    

    View Slide

79. View Slide

80. 80
    User-Agent: jigpuzbcomkenhvladtwysqfxr
    User-Agent: yudjmikcvzoqwsbflghtxpanre
    User-Agent: mckvhaflwzbderiysoguxnqtpj
    User-Agent: deogjvtynmcxzwfsbahirukqpl
    User-Agent: fdmjczoeyarnuqkbgtlivsxhwp
    User-Agent: yczfxlrenuqtwmavhojpigkdsb
    User-Agent: dnlseufokcgvmajqzpbtrwyxih
    

    View Slide

81. View Slide

82. View Slide

83. View Slide

84. 84
    function attack(String target, String userAgent, String referer) {
    HashMap WebViewHeaders = new HashMap();
    WebViewHeaders->put(“Referer”,referer);
    WebViewHeaders->put(“X-Requested-With”,””);
    WebView[] AttackerViews = new WebView[100];
    for (int i=0; iAttackerViews[i] = new WebView();
    AttackerViews[i]->clearHistory();
    AttackerViews[i]->clearFormData();
    AttackerViews[i]->clearCache(true);
    WebViewSettings AWVS = AttackerViews[i]->getSettings()
    AttackWebViewSettings->setJavaScriptEnabled(true);
    AttackWebViewSettings->setUserAgentString(userAgent);
    AttackWebViewSettings->setCacheMode(LOAD_NO_CACHE);
    this->deleteDatabase(“webview.db”);
    this->deleteDatabase(“webviewCache.db”);
    AttackerViews[i]->loadUrl(target,WebViewHeaders);
    }
    }
    }
    

    View Slide

85. WireX: Profile
    • Real-looking L7 requests
    • Mid-size - 80k rps, 120k IP's/h
    • Location: Everywhere
    • Targets: interesting
    • Attacker: .
    • Infection: 300+ Android apps
    85
    

    View Slide

86. 6. Elections
    86
    

    View Slide

87. View Slide

88. Attack data
    • trump.com: ~54,000 attacks per day
    • donaldjtrump.com: ~501,000 attacks per day
    • trump.com: Attacked on 100% of days
    • donaldjtrump.com: Attacked on 94.4% of days
    88
    

    View Slide

89. Attack data
    • trump.com:
    • 25 Apr 2016 ~3,4M requests (39 rps)
    • donaldjtrump.com:
    • 10 Dec 2015 ~15M requests (173 rps)
    89
    

    View Slide

90. View Slide

91. View Slide

92. 7. The great cannon
    92
    

    View Slide

93. 93
    Eyeball
    Target
    Infected
    website
    

    View Slide

94. View Slide

95. Infected website?
    • Compromised JS
    • Competition case
    • China great firewall
    • L3 injection to scripts
    • Rogue AD
    95
    

    View Slide

96. Cloudflare
    96
    

    View Slide

97. 97
    1. Architecture
    

    View Slide

98. 98
    2. Mitigation in software
    

    View Slide

99. 99
    Attack
    Detection
    Mitigation
    Automation
    

    View Slide

100. Thanks!
     • Architected for DDoS
     • Iptables + BPF great for L3
     • Kernel bypass
     • XDP
     • Iptables + Conntrack great for L7
     • Also: connlimit, hashlimits, ipset
     100
     marek@cloudflare.com @majek04
     

     View Slide