URL to HTML

URL to HTML

What happens in between the time you type a URL in your browser and the time you see the fully rendered page.

0110e86fdb31486c22dd381326d99de9?s=128

Francois Marier

May 24, 2014
Tweet

Transcript

  1. URL to HTML a minute in the life of a

    webpage François Marier @fmarier mozilla
  2. mozilla newmarket

  3. 8 engineers

  4. 8 engineers 1 designer

  5. 8 engineers 1 designer 1 manager

  6. video & media marketplace cloud services crash investigation

  7. video & media marketplace cloud services crash investigation

  8. video & media marketplace cloud services crash investigation

  9. video & media marketplace cloud services crash investigation

  10. > 1,000 employees world-wide

  11. </ >

  12. None
  13. 1. Learn HTML 2. ? 3. Profit !

  14. None
  15. None
  16. abstractions

  17. asbtraction construct used to understand a complicated topic at a

    high level
  18. asbtraction extra layer added to avoid writing the same code

    over and over
  19. drawSquare()

  20. drawLine()

  21. drawLine()

  22. drawLine()

  23. drawLine()

  24. None
  25. drawSquare()

  26. None
  27. if you don't understand the layers below, you won't know

    what to do when the abstraction breaks
  28. mastery requires a high-level understanding of the rest of the

    stack
  29. web performance

  30. web performance how bytes make it to the user

  31. web performance how bytes make it to the user how

    the browser renders the page
  32. URL DNS IP TCP HTTP / TLS HTML

  33. URL DNS IP TCP HTTP / TLS HTML

  34. URL uniform ressource locator

  35. http://www.example.com

  36. http://www.example.com

  37. http://www.example.com /articles/

  38. http://www.example.com /articles/tutorial.cgi

  39. http://www.example.com /articles/tutorial.cgi ?showsolutions=0&topic=web

  40. http://www.example.com /articles/tutorial.cgi ?showsolutions=0&topic=web #part5

  41. http://www.example.com:80 /articles/tutorial.cgi ?showsolutions=0&topic=web #part5

  42. http://username:password@ www.example.com:80 /articles/tutorial.cgi ?showsolutions=0&topic=web #part5

  43. DNS domain name system

  44. $ cat /etc/resolv.conf nameserver 208.67.222.222 nameserver 208.67.220.220

  45. None
  46. www.cs.auckland.ac.nz

  47. www.cs.auckland.ac.nz 130.216.158.22

  48. None
  49. $ dig nz NS @199.7.83.42

  50. $ dig nz NS @199.7.83.42 ; <<>> DiG 9.8.1-P1 <<>>

    nz NS @199.7.83.42 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 412 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 7, ADD ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;nz. IN NS ;; AUTHORITY SECTION: nz. 172800 IN NS ns1.dns.net.nz. nz. 172800 IN NS ns2.dns.net.nz. nz. 172800 IN NS ns3.dns.net.nz. nz. 172800 IN NS ns4.dns.net.nz. nz. 172800 IN NS ns5.dns.net.nz. nz. 172800 IN NS ns6.dns.net.nz. nz. 172800 IN NS ns7.dns.net.nz.
  51. $ dig ac.nz NS @ns1.dns.net.nz

  52. $ dig ac.nz NS @ns1.dns.net.nz ; <<>> DiG 9.8.1-P1 <<>>

    ac.nz NS @ns1.dns.net.nz ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 391 ;; flags: qr aa rd; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;ac.nz. IN NS ;; ANSWER SECTION: ac.nz. 86400IN NS ns7.dns.net.nz. ac.nz. 86400IN NS ns4.dns.net.nz. ac.nz. 86400IN NS ns2.dns.net.nz. ac.nz. 86400IN NS ns1.dns.net.nz. ac.nz. 86400IN NS ns6.dns.net.nz. ac.nz. 86400IN NS ns3.dns.net.nz. ac.nz. 86400IN NS ns5.dns.net.nz.
  53. $ dig auckland.ac.nz NS @ns1.dns.net.nz

  54. $ dig auckland.ac.nz NS @ns1.dns.net.nz ; <<>> DiG 9.8.1-P1 <<>>

    auckland.ac.nz NS @ns1.dns.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 598 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADD ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;auckland.ac.nz. IN NS ;; AUTHORITY SECTION: auckland.ac.nz. 86400IN NS pubsec.domainz.net.nz. auckland.ac.nz. 86400IN NS dns1.auckland.ac.nz. auckland.ac.nz. 86400IN NS dns2.auckland.ac.nz. ;; ADDITIONAL SECTION: dns1.auckland.ac.nz.86400IN A 130.216.1.2 dns2.auckland.ac.nz.86400IN A 130.216.1.1
  55. $ dig cs.auckland.ac.nz NS @dns1.auckland.ac.nz

  56. $ dig cs.auckland.ac.nz NS @dns1.auckland.ac.nz ; <<>> DiG 9.8.1-P1 <<>>

    cs.auckland.ac.nz NS @dns1.auc ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 485 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;cs.auckland.ac.nz. IN NS ;; ANSWER SECTION: cs.auckland.ac.nz. 10800 IN NS dns2.auckland.ac.nz. cs.auckland.ac.nz. 10800 IN NS kronos2.cs.auckland.ac.n cs.auckland.ac.nz. 10800 IN NS dns1.auckland.ac.nz. cs.auckland.ac.nz. 10800 IN NS kronos1.cs.auckland.ac.n ;; ADDITIONAL SECTION: dns1.auckland.ac.nz.1800 IN A 130.216.1.2
  57. $ dig www.cs.auckland.ac.nz @kronos1.cs.auckland.ac.nz

  58. $ dig www.cs.auckland.ac.nz @kronos1.cs.auckland.ac.nz ; <<>> DiG 9.8.1-P1 <<>> www.cs.auckland.ac.nz

    A @krono ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 175 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;www.cs.auckland.ac.nz. IN A ;; ANSWER SECTION: www.cs.auckland.ac.nz. 10800 IN A 130.216.158.22 ;; AUTHORITY SECTION: cs.auckland.ac.nz. 10800IN NS kronos2.cs.auckland.ac.nz cs.auckland.ac.nz. 10800IN NS dns2.auckland.ac.nz. cs.auckland.ac.nz. 10800IN NS dns1.auckland.ac.nz. cs.auckland.ac.nz. 10800IN NS kronos1.cs.auckland.ac.nz
  59. l.root-servers.net ns1.dns.net.nz ns1.dns.net.nz dns1.auckland.ac.nz kronos1.cs.auckland.ac.nz

  60. l.root-servers.net ns1.dns.net.nz ns1.dns.net.nz dns1.auckland.ac.nz kronos1.cs.auckland.ac.nz

  61. l.root-servers.net ns1.dns.net.nz ns1.dns.net.nz dns1.auckland.ac.nz kronos1.cs.auckland.ac.nz

  62. l.root-servers.net ns1.dns.net.nz ns1.dns.net.nz dns1.auckland.ac.nz kronos1.cs.auckland.ac.nz

  63. l.root-servers.net ns1.dns.net.nz ns1.dns.net.nz dns1.auckland.ac.nz kronos1.cs.auckland.ac.nz

  64. IP internet protocol

  65. www.bbc.co.uk 212.58.246.94

  66. None
  67. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  68. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  69. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  70. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  71. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  72. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  73. $ mtr 212.58.246.94 1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4.

    121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net 10. ae-62-62.csw1.SanJose1.Level3.net 11. ae-61-61.ebr1.SanJose1.Level3.net 12. ae-2-2.ebr2.NewYork1.Level3.net 13. ae-62-62.csw1.NewYork1.Level3.net 14. ae-61-61.ebr1.NewYork1.Level3.net 15. ae-43-43.ebr2.London1.Level3.net 16. ae-57-222.csw2.London1.Level3.net 17. ae-229-3605.edge4.London1.Level3.net 18. BBC-TECHNOL.edge4.London1.Level3.net 19. ??? 20. ??? 21. ae0.er01.cwwtf.bbc.co.uk 22. 132.185.255.165 23. bbc-vip015.cwwtf.bbc.co.uk
  74. 130.216.158.22 212.58.246.94

  75. 130.216.158.22 212.58.246.94 router drops packets packets arrive in wrong order

  76. 130.216.158.22 212.58.246.94 router drops packets cable is cut packets arrive

    in wrong order
  77. 130.216.158.22 212.58.246.94 router drops packets cable is cut packets arrive

    in wrong order
  78. ideal network actual network

  79. TCP transmission control protocol

  80. guarantees in-order delivery of packets

  81. abstraction of a reliable point-to-point connection with built-in re-try logic

  82. applications have a lot less errors to deal with

  83. UDP user datagram protocol

  84. None
  85. TCP UDP

  86. reminder: abstractions are leaky

  87. 3-way handshake establishing a new connection

  88. hi

  89. how are you?

  90. good, you?

  91. client server

  92. client SYN x=42 server

  93. client SYN x=42 SYN+ACK y=10,x=43 server

  94. client SYN x=42 SYN+ACK y=10,x=43 ACK y=11 server

  95. HTTP hypertext transfer protocol

  96. http://www.example.com

  97. http://www.example.com

  98. clear text protocol

  99. client request server

  100. client request response server

  101. Host: www.example.com User-Agent: Mozilla/5.0 (rv:29.0) Firefox/29.0 DNT: 1

  102. Host: www.example.com User-Agent: Mozilla/5.0 (rv:29.0) Firefox/29.0 DNT: 1 <request body

    goes in here>
  103. Content-Type: text/html Date: Thu, 22 May 2014 05:34:47 GMT Last-Modified:

    Fri, 09 Aug 2013 23:54:35 GMT Content-Length: 1270 <!doctype html> <html> <head> <title>Example Domain</title> </head> <body> <div> <h1>Example Domain</h1> <p>This domain is established to be used for domain in examples without prior coordination </div> </body> </html>
  104. 200 OK

  105. 404 Not Found

  106. None
  107. $ curl http://www.example.com <!doctype html> <html> <head> <title>Example Domain</title> </head>

    <body> <div> <h1>Example Domain</h1> <p>This domain is established to be used for domain in examples without prior coordinatio </div> </body> </html>
  108. $ curl --head http://www.example.com HTTP/1.1 200 OK Accept-Ranges: bytes Cache-Control:

    max-age=604800 Content-Type: text/html Date: Thu, 22 May 2014 05:42:26 GMT Etag: "359670651" Expires: Thu, 29 May 2014 05:42:26 GMT Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT Server: ECS (cpm/F858) X-Cache: HIT x-ec-custom-error: 1 Content-Length: 1270
  109. verbs (fancy word for commands)

  110. GET

  111. POST

  112. GET /article/43228

  113. GET /article/43228 GET /article/43228

  114. GET /article/43228 POST /article/delete/last

  115. GET /article/43228 POST /article/delete/last POST /article/delete/last

  116. GET /article/43228 POST /item/20/buy POST /item/20/buy $$$ $$$

  117. None
  118. TLS transport layer security

  119. SSL secure sockets layer

  120. HTTPS hypertext transfer protocol secure

  121. secure (sometimes)

  122. client server (pk, sk)

  123. client hello! server (pk, sk)

  124. client hello! hello! pubkey server (pk, sk)

  125. client hello! hello! pubkey server (pk, sk) session key

  126. client hello! hello! pubkey i'm done! encrypt pk (session key)

    server (pk, sk) session key
  127. client hello! hello! pubkey i'm done! encrypt pk (session key)

    server (pk, sk) session key session key
  128. man-in-the-middle

  129. client server (pk, sk)

  130. client server (pk, sk) NSA (pk, sk)

  131. client hello! server (pk, sk) NSA (pk, sk)

  132. client hello! server (pk, sk) NSA (pk, sk) hello!

  133. client hello! hello! pubkey server (pk, sk) NSA (pk, sk)

    hello!
  134. client hello! hello! pubkey server (pk, sk) NSA (pk, sk)

    hello! hello! pubkey
  135. client hello! hello! pubkey server (pk, sk) key NSA (pk,

    sk) hello! hello! pubkey
  136. client hello! hello! pubkey i'm done! encrypt(key) server (pk, sk)

    key NSA (pk, sk) hello! hello! pubkey
  137. client hello! hello! pubkey i'm done! encrypt(key) server (pk, sk)

    key NSA (pk, sk) hello! hello! pubkey key
  138. client hello! hello! pubkey i'm done! encrypt(key) server (pk, sk)

    key NSA (pk, sk) hello! hello! pubkey i'm done! encrypt(key) key
  139. client hello! hello! pubkey i'm done! encrypt(key) server (pk, sk)

    key NSA (pk, sk) hello! hello! pubkey i'm done! encrypt(key) key key
  140. client hello! hello! pubkey i'm done! encrypt(key) server (pk, sk)

    key NSA (pk, sk) hello! hello! pubkey i'm done! encrypt(key) key key
  141. authentication (of the server)

  142. client hello! hello! pubkey server (pk, sk) session key

  143. client hello! hello! signed pubkey server (pk, sk) session key

    verify signature
  144. client hello! hello! signed pubkey server (pk, sk) session key

    verify signature i'm done! encrypt pk (session key) session key
  145. client hello! hello! signed pubkey server (pk, sk) NSA (pk,

    sk) hello! hello! signed pubkey key
  146. client hello! hello! signed pubkey server (pk, sk) NSA (pk,

    sk) hello! hello! signed pubkey key abort!
  147. how can you tell you're talking to the right person?

    (and not to the NSA)
  148. trusted third-party certificate authority

  149. trusted third-party certificate authority

  150. None
  151. EFF has found more than 650 certificate authorities in the

    wild
  152. None
  153. *.google.com

  154. *.google.com

  155. *.google.com

  156. *.google.com 7 different domains

  157. *.google.com 7 different domains

  158. $100

  159. $1,000 $100

  160. $1,000 $1,000 $100

  161. HTML hypertext markup language

  162. parsing

  163. .png .jpg .js .css

  164. None
  165. None
  166. resolve all hostnames establish TCP connections negotiate TLS session URL

    DNS IP TCP HTTP / TLS HTML
  167. @fmarier francois@mozilla.com questions?

  168. Copyright © 2014 Francois Marier <francois@mozilla.com> This work is licensed

    under a Creative Commons Attribution-ShareAlike 4.0 International License. leaky pipe: https://www.flickr.com/photos/ifl/3920636654 leaky pipe with elephant: https://www.flickr.com/photos/rcrhee/10785374875 sky tower: https://www.flickr.com/photos/elisfanclub/6120863439 golden gate: https://www.flickr.com/photos/jeffgunn/6663212147 san jose: https://www.flickr.com/photos/the_tahoe_guy/3183673224 statue of liberty: https://www.flickr.com/photos/suewaters/7574642942 big ben: https://www.flickr.com/photos/timmorris/3103896345 bbc house: https://www.flickr.com/photos/redvers/532073098 fingers crossed: https://www.flickr.com/photos/bearpark/6861722073 prince charles : http://en.wikipedia.org/wiki/File:Prince_Charles_2012.jpg southern cross cable: https://en.wikipedia.org/wiki/File:Southern_Cross_Cable_route.svg image credits