Advanced JavaScript

Transcript

1. Advanced JavaScript Creating Modern Web Applications @BastianHofmann
2. Fancy title, isn't it? but what is it about
3. it's about JS, how you can build rich, single page web applications with it that integrate 3rd party data safely and are using real-time communication between server and client or between two clients to create a real responsive application
4. Let's go back in time but first .. about 15 years ago
5. the web looked like this, and you can count yourself happy that I spared you the animations and background music
6. or more seriously like, this. lot's of pages linked together, some forms, but far way from the user experience a desktop app had
7. here ebay from back then. overall what you could do with a web app was very limited, images, text, forms, animated gifs, background music, marquee text, js was considered security risk, turn it off, due to features not available in browser, low bandwith
8. php 3 was out, who was around back then doing php? and hey y2k compliant
9. Fast forward to today
10. web sites, no web apps look more like this, rich application, desktop like feeling, offline capabilities, responsive
11. other example: maps
12. or whole ides in the browser
13. So what will we cover today? what will this talk be about?
14. •JS Web Application development •WebSockets •OEmbed and Caja •APIs and OAuth2 •Real-Time updates PubsubHubbub •WebRTC we'll talk about several techniques to create such rich, responsive web applications, that integrate 3rd party data, some so brand new that they are only available in dev channel or nightly builds and some that you can use now in your existing apps
15. A few words about me before that ...
16. i work at researchgate, the social network for scientists and researchers
17. ResearchGate gives science back to the people who make it happen. We help researchers build reputation and accelerate scientific progress. On their terms. ‟ the goal is to give...
18. over 2.5 million users
19. here some impressions of the page
20. we are hiring, talk to me if you want
21. have this, and also work on some cool stuff
22. in addition to this i also speak frequently on conferences throughout the world
23. work and live in berlin
24. Questions? Ask by the way, if you have any questions throughout this talk, if you don't understand something, just raise your hand and ask. probably my fault anyways since i spoke to quickly or my accent was too bad
25. http://speakerdeck.com/u/bastianhofmann the slides with speaking notes will be available on speakerdeck after the talk
26. ? how about you? languages you use regularly? have your own product? agency work? long lived, short lived projects? large codebases? frameworks you use? js experience?
27. Before we start with lot's of code...
28. ... let's talk about Application architecture ... with that I mean
29. Structure of your code
30. Code and component re-use both across backend (server) and frontend (browser) code
31. Rapid Development what this means for the speed of development
32. Large code-bases and how this scales in ...
33. Frameworks to the rescue usually if you have a larger project, you tend to use frameworks to help you in architectural decisions, they give you guidance in how to develop your app, you can concentrate more on your actual business logic
34. Most Web Frameworks are incomplete but .. .. at least in my opinion
35. if you look at popular frameworks for the server side
36. or popular frameworks for the client side (i know jquery is more a dom abstraction library than a framework)
37. webserver HTML browser JS they are all designed to fit a more traditional model of a web application, server side renders out html (or json), browser displays it, some javascript to handle browser events, do ajax request and dom manipulations
38. de duplication ode duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplicatio code duplicat code duplic code dup code du code cod co co but you don't share any code between client and server resulting in lot's of code duplication. but the possibilities to share code are there: models, templates, validation rules e.g.
39. so with web applications getting more and more complex, and more and more stuff happening in the client, in js, to create these rich and responsive web apps, something has to change in order to create large, maintainable apps rapidly. lets look a bit into the future
40. http://www.meteor.com the app is going to be based on meteor js, a js framework that combines server side and client side programming which reduces the amount of boilerplate code you have to write for server to client communication tremendously, still very early, in some areas limited, but really fun to develop with
41. https://github.com/bashofmann/adv_js_demo/ the code is available on github, so you can check it out later and revisit what i did, run it, debug it, play around with it
42. Let‘s start as I said, I want to create a JavaScript application that is very responsive, with real-time updates and integrates 3rd party data from various sources in a secure way. so let's do some groundwork first
43. demo: basic setup, authorization, displaying the feed, input form, instant updated, websocket communication
44. Fancy, but what's in it for me? now you may ask ...
45. Legacy Project because you most likely have a ...
46. Lot's of things! actually there are lot's of things you can use today in your old, existing project (if time war story refactoring the researchgate codebase)
47. CSS Bastian first of all this is my relationship with css
48. http://twitter.github.com/bootstrap so to make it easier for you, you can look at a very good css framework and see how they are doing stuff, e.g. at researchgate we are not using bootstrap, but modeled some of our css after it
49. Good starting point when refactoring your CSS it's a ...
50. Templates next ...
51. var html="<ul>"; for (var i=0; i < viewers.length; i++) { html += "<li>" + viewers[i].displayName + "</li>"; } html += "<ul>"; document.getElementById("#div").innerHTML = html; Where is the error? oftentimes you see something like this in js applications, you iterate over some data, concatenate an html string, and put that into some dom node where is the error? problems: 1st xss, 2nd structure not visible, no code highlights, no ide support, easy to make errors 3rd if your app get's bit, hard to find where the markup is coming from, 4th we are forgetting everything we learned with server side development of web apps, mvc, separation of concerns
52. Templates so, use ....
53. Mustache.JS https://github.com/janl/mustache.js } there are lot's of js templating languages, some examples
54. Hogan.js } http://twitter.github.com/hogan.js/ mustache but with precompiled templates -> faster
55. http://handlebarsjs.com/ also a mustache dialect, compatible but with more features, we are using that one in meteor
56. Two-way communication the last thing you saw in the demo was ... ... which means the app updates when something happens automatically, no reloads necessary
57. WebSockets http://dev.w3.org/html5/websockets/ one way to do this is websockets, if this is not available, you can fall back to other techniques, long polls, short polls, flash, ...
58. socket.io http://socket.io/ libraries helping you with that, e.g. ...
59. SockJS http://sockjs.org/ .. this one is used by meteor
60. http://www.html5rocks.com/en/ tutorials/websockets/basics/ if you want to know how websockets work, read the spec, and/or this blog article
61. Now that we did the groundwork, let's have some fun!
62. MashUps lets mash or app up with some external data
63. cool video: http:// www.youtube.com/ watch?v=OFzkTxiwziQ as you saw in the stream you often times have status updates like this one, with a link, wouldn't it be nice to not only have the link but also a preview of what's behind the link under it. of course youtube offers embedding videos in other pages through an iframe, so you could parse the url, see that it's youtube, parse out the video id, put it into the iframe url, but what if youtube changes the embed tag? what if you want to support more than youtube? lots of work to build and maintain...
64. OEmbed http://oembed.com/ fortunately there is a protocol helping us with that
65. http://www.youtube.com/watch? v=OyJd2qsRkNk in oembed if you have url like
66. you make a get request to it and get back the html page, in the head there are some links pointing to an oembed endpoint ('application/json|xml+oembed)
67. http://www.youtube.com/oembed?url=http%3A %2F%2Fwww.youtube.com%2Fwatch%3Fv %3DOyJd2qsRkNk&maxwidth=500&format=json if you query this endpoint
68. { "provider_url":"http:\/\/www.youtube.com\/", "title":"Jupiter Jones - Das Jahr in dem ich schlief (Musik Video)", "html":"\u003cobject width=\"500\" height=\"306\"\u003e \u003cparam name=\"movie\" value=\"http:\/\/www.youtube.com\/v\/ OyJd2qsRkNk?version=3\"\u003e\u003c\/param\u003e\u003cparam name= \"allowFullScreen\" value=\"true\"\u003e\u003c\/param\u003e \u003cparam name=\"allowscriptaccess\" value=\"always\"\u003e \u003c\/param\u003e\u003cembed src=\"http:\/\/www.youtube.com\/v\/ OyJd2qsRkNk?version=3\" type=\"application\/x-shockwave-flash \" width=\"500\" height=\"306\" allowscriptaccess=\"always \" allowfullscreen=\"true\"\u003e\u003c\/embed\u003e\u003c\/object \u003e", "author_name":"St182", "height":306, "thumbnail_width":480, "width":500, "version":"1.0", "author_url":"http:\/\/www.youtube.com\/user\/Stinkfist182", "provider_name":"YouTube", "thumbnail_url":"http:\/\/i4.ytimg.com\/vi\/OyJd2qsRkNk\/ hqdefault.jpg", "type":"video", "thumbnail_height":360 } you get back a json object with meta information about the url, title, thumbnail and for videos some html which is the embed tag from youtube in this example
69. cool video: the result may then look like this
70. http://embed.ly/ if we don't want to do all this oembed endpoint lookup and have something more comfortable there is also a web service called embedly, you can just post the url to it and get back the oembed json object
71. OpenGraph http://ogp.me/ other ways to get meta information about a url are
72. <html prefix="og: http://ogp.me/ns#"> <head> <title>The Rock (1996)</title> <meta property="og:title" content="The Rock"/> <meta property="og:type" content="video.movie" /> <meta property="og:url" content="http:// www.imdb.com/title/tt0117500/" /> <meta property="og:image" content="http:// ia.media-imdb.com/images/rock.jpg" /> ... </head> ... </html> with opengraph there are og meta tags in the pages head
73. Twitter Cards
74. <meta name="twitter:card" content="summary"> <meta name="twitter:site" content="@nytimes"> <meta name="twitter:url" content="http:// www.nytimes.com/2012/02/19/arts/music/amid- police-presence-fans-congregate-for-whitney- houstons-funeral-in-newark.html"> <meta name="twitter:title" content="Parade of Fans for Houston’s Funeral"> <meta name="twitter:description" content="..."> <meta name="twitter:image" content="http:// graphics8.nytimes.com/images/2012/02/19/us/ 19whitney-span/19whitney-span- articleLarge.jpg"> also special twitter meta tags in the page's head
75. Microdata http://www.w3.org/html/wg/drafts/ microdata/master/ http://schema.org/ ... and ..
76. Microformats http://microformats.org/ with both these specs you annotate some dom nodes with special classes, used for example in the google search
77. Meta Tags or you can just parse some common meta tags like title or description as a fallback
78. Demo OEmbed
79. So embedding external content
80. Is this safe?
81. External HTML could lead to xss if there are script tags in the html
82. http://embed.ly/ if you use embed.ly you're somehow fine, because they are making sure nothing harmful is in there
83. Trust nobody! but ... (probably not even yourself)
84. Caja http://code.google.com/p/google-caja/ sanitize external html, caja strips out all harmful stuff, like script tags, javascript: urls
85. demo: xss, caja
86. API Access next ...
87. to demo that i'm using status.net, an opensource microblogging platform, just because on conferences i can't rely on wifi. one problem though, we want to make ajax request from js to an external domain
88. Same origin policiy problem: same origin policy, which says you are only allowed to access windows or make ajax request to resources that have the same host name as the page that you are on. very important browser security feature, otherwise you could steal the sessions of all other applications that are currently open or have a running session or permanent login cookie. but for api requests this would be fine, we can trust the api provider not to leak sessions
89. JSONP on way jsonp, a bit hacky, google it if you want details
90. Cross-Origin Resource Sharing much nicer, and good browser support
91. Backend api.twitter.com Client client.net AJAX Access-Control-Allow-Origin: * http://www.w3.org/TR/cors/ with cors, before the browser actually does the ajax request it performs an options request and the api needs to return the access-control-allow-origin header, whose value should match the requesting host. if this is the case the ajax request will be executed
92. Authorization of course most api's need authorization to access user specific resources like the user's home timeline
93. http://oauth.net/ one way to do this in javascript applications is oauth2
94. http://tools.ietf.org/html/rfc6749 it's final :)
95. User-Agent Profile i'll not explain the whole of oauth2, but just what is necessary for js application
96. http://twitter.com/authorize? &clientId=... Open Popup lanyrd.com it works like this, on the client app, lanyrd.com, you click a button, the client opens a new browser window pointing to the authorization endpoint of the api, with the clientId (you need to be preregistered at the api) of the client app (and some other parameters)
97. http://twitter.com/authorize? &clientId=... Open Popup HTTPS GET twitter.com/ authorize lanyrd.com in this popups the api provider asks the user to log in,
98. http://twitter.com/authorize? &clientId=... Open Popup Login twitter.com/ authorize lanyrd.com if he is not
99. http://twitter.com/authorize? &clientId=... Open Popup Grant Permission twitter.com/ authorize lanyrd.com and then to grant the client permission to access the api on the user's behalf
100. HTTPS Redirect RedirectURI#ac cessToken RedirectURI# accessToken lanyrd.com twitter.com/ authorize lanyrd.com if the user said yes, the user gets redirected, still in the popup, to a preregistered redirect callback url, the access token for the api is in the fragment part of this url, this means it will not be sent to the backend and not to any proxies but stays in the browser
101. RedirectURI# accessToken Parse Access Token from Fragment Send it to opening window Close popup lanyrd.com lanyrd.com the callback page can only return some static html and js. the js reads out the fragment, and since callback page and client app in the parent window are on the same hostname, it can call a function at the parent window to pass the access token to the client app and then close itself
102. HTTPS Ajax Request to API Access Token twitter.com lanyrd.com now the client app has the access token and can access the api
103. Storing the access token of course you have to store the access token somewhere. cookie would be one example, but that would send the access token over the wire and we may not want this
104. Local Storage http://www.w3.org/TR/webstorage/ better local storage
105. Get new entries pushed by the API now that we have access to api it would be nice to get new entries of the user who connected to it pushed automatically so that we can include it into our stream
106. PubSubHubbub http://code.google.com/p/pubsubhubbub/ many apis have their own implementations, status.net uses pubsubhubbub, but the principles are often the same
107. retrieves Atom feed with Hub URL Hub posts sth pings every subscriber subscribes for feed acks subscription with pubsubhubbub you subscribe your application (backend) for the users feed at a central hub that is able to provide this users feed. after some subscription pingpong, each time when the user adds a new entry to his feed, it is send to the hub and the hub will forward it to every subscriber (of course only possible backend to backend since the hub can't send it directly to a browser)
108. demo: oauth2, pubsubhubbub, sara, chat
109. http://www.webrtc.org/ next p2p realtime communication between clients, this is possible with webrtc, very new, currently only possible in chrome dev channel and canary and firefox nightly builds, you can exchange any binary data, not only text, audio, video
110. PEERJS http://peerjs.com/ one easy abstraction layer for webrtc just was published a few days ago, called peerjs, only works reliably for text at the moment, due to browser bugs (if even that)
111. http://www.html5rocks.com/en/ tutorials/webrtc/basics/ you can read up details in the spec or here
112. Try it out yourself if you want to ...
113. http://vagrantup.com/ i built a vagrant vm for it
114. https://github.com/bashofmann/vm_js_mashup
115. $ vagrant up $ vagrant ssh $ cd app $ mrt Follow the readme! basically that's what you have to do, but there are some small additional steps (adapting your host file for example, follow the readme)
116. http://twitter.com/BastianHofmann http://profiles.google.com/bashofmann http://lanyrd.com/people/BastianHofmann http://speakerdeck.com/u/bastianhofmann https://github.com/bashofmann https://www.researchgate.net/profile/Bastian_Hofmann/ mail@bastianhofmann.de Did you like this talk? https://joind.in/7977 thanks, you can contact me on any of these platforms or via mail. if you liked this talk or didn't like it, please rate it on joind.in. this is very important for me for improving my talk, for the organizers for selecting the best talks and speakers and for you to get the best content on the conferences you visit.