Tweet
Tweet

Slide 1

Slide 1 text

Advanced JavaScript Creating Modern Web Applications @BastianHofmann

Slide 2

Slide 2 text

Fancy title, isn't it? but what is it about

Slide 3

Slide 3 text

it's about JS, how you can build rich, single page web applications with it that integrate 3rd party data safely and are using real-time communication between server and client or between two clients to create a real responsive application

Slide 4

Slide 4 text

Let's go back in time but first .. about 15 years ago

Slide 5

Slide 5 text

the web looked like this, and you can count yourself happy that I spared you the animations and background music

Slide 6

Slide 6 text

or more seriously like, this. lot's of pages linked together, some forms, but far way from the user experience a desktop app had

Slide 7

Slide 7 text

here ebay from back then. overall what you could do with a web app was very limited, images, text, forms, animated gifs, background music, marquee text, js was considered security risk, turn it off, due to features not available in browser, low bandwith

Slide 8

Slide 8 text

php 3 was out, who was around back then doing php? and hey y2k compliant

Slide 9

Slide 9 text

Fast forward to today

Slide 10

Slide 10 text

web sites, no web apps look more like this, rich application, desktop like feeling, offline capabilities, responsive

Slide 11

Slide 11 text

other example: maps

Slide 12

Slide 12 text

or whole ides in the browser

Slide 13

Slide 13 text

So what will we cover today? what will this talk be about?

Slide 14

Slide 14 text

•JS Web Application development •WebSockets •OEmbed and Caja •APIs and OAuth2 •Real-Time updates PubsubHubbub •WebRTC we'll talk about several techniques to create such rich, responsive web applications, that integrate 3rd party data, some so brand new that they are only available in dev channel or nightly builds and some that you can use now in your existing apps

Slide 15

Slide 15 text

A few words about me before that ...

Slide 16

Slide 16 text

i work at researchgate, the social network for scientists and researchers

Slide 17

Slide 17 text

ResearchGate gives science back to the people who make it happen. We help researchers build reputation and accelerate scientific progress. On their terms. ‟ the goal is to give...

Slide 18

Slide 18 text

over 2.5 million users

Slide 19

Slide 19 text

here some impressions of the page

Slide 20

Slide 20 text

we are hiring, talk to me if you want

Slide 21

Slide 21 text

have this, and also work on some cool stuff

Slide 22

Slide 22 text

in addition to this i also speak frequently on conferences throughout the world

Slide 23

Slide 23 text

work and live in berlin

Slide 24

Slide 24 text

Questions? Ask by the way, if you have any questions throughout this talk, if you don't understand something, just raise your hand and ask. probably my fault anyways since i spoke to quickly or my accent was too bad

Slide 25

Slide 25 text

http://speakerdeck.com/u/bastianhofmann the slides with speaking notes will be available on speakerdeck after the talk

Slide 26

Slide 26 text

? how about you? languages you use regularly? have your own product? agency work? long lived, short lived projects? large codebases? frameworks you use? js experience?

Slide 27

Slide 27 text

Before we start with lot's of code...

Slide 28

Slide 28 text

... let's talk about Application architecture ... with that I mean

Slide 29

Slide 29 text

Structure of your code

Slide 30

Slide 30 text

Code and component re-use both across backend (server) and frontend (browser) code

Slide 31

Slide 31 text

Rapid Development what this means for the speed of development

Slide 32

Slide 32 text

Large code-bases and how this scales in ...

Slide 33

Slide 33 text

Frameworks to the rescue usually if you have a larger project, you tend to use frameworks to help you in architectural decisions, they give you guidance in how to develop your app, you can concentrate more on your actual business logic

Slide 34

Slide 34 text

Most Web Frameworks are incomplete but .. .. at least in my opinion

Slide 35

Slide 35 text

if you look at popular frameworks for the server side

Slide 36

Slide 36 text

or popular frameworks for the client side (i know jquery is more a dom abstraction library than a framework)

Slide 37

Slide 37 text

webserver HTML browser JS they are all designed to fit a more traditional model of a web application, server side renders out html (or json), browser displays it, some javascript to handle browser events, do ajax request and dom manipulations

Slide 38

Slide 38 text

de duplication ode duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplication code duplicatio code duplicat code duplic code dup code du code cod co co but you don't share any code between client and server resulting in lot's of code duplication. but the possibilities to share code are there: models, templates, validation rules e.g.

Slide 39

Slide 39 text

so with web applications getting more and more complex, and more and more stuff happening in the client, in js, to create these rich and responsive web apps, something has to change in order to create large, maintainable apps rapidly. lets look a bit into the future

Slide 40

Slide 40 text

http://www.meteor.com the app is going to be based on meteor js, a js framework that combines server side and client side programming which reduces the amount of boilerplate code you have to write for server to client communication tremendously, still very early, in some areas limited, but really fun to develop with

Slide 41

Slide 41 text

https://github.com/bashofmann/adv_js_demo/ the code is available on github, so you can check it out later and revisit what i did, run it, debug it, play around with it

Slide 42

Slide 42 text

Let‘s start as I said, I want to create a JavaScript application that is very responsive, with real-time updates and integrates 3rd party data from various sources in a secure way. so let's do some groundwork first

Slide 43

Slide 43 text

demo: basic setup, authorization, displaying the feed, input form, instant updated, websocket communication

Slide 44

Slide 44 text

Fancy, but what's in it for me? now you may ask ...

Slide 45

Slide 45 text

Legacy Project because you most likely have a ...

Slide 46

Slide 46 text

Lot's of things! actually there are lot's of things you can use today in your old, existing project (if time war story refactoring the researchgate codebase)

Slide 47

Slide 47 text

CSS Bastian first of all this is my relationship with css

Slide 48

Slide 48 text

http://twitter.github.com/bootstrap so to make it easier for you, you can look at a very good css framework and see how they are doing stuff, e.g. at researchgate we are not using bootstrap, but modeled some of our css after it

Slide 49

Slide 49 text

Good starting point when refactoring your CSS it's a ...

Slide 50

Slide 50 text

Templates next ...

Slide 51

Slide 51 text

var html="
    "; for (var i=0; i < viewers.length; i++) { html += "
  • " + viewers[i].displayName + "
    • "; } html += "
      "; document.getElementById("#div").innerHTML = html; Where is the error? oftentimes you see something like this in js applications, you iterate over some data, concatenate an html string, and put that into some dom node where is the error? problems: 1st xss, 2nd structure not visible, no code highlights, no ide support, easy to make errors 3rd if your app get's bit, hard to find where the markup is coming from, 4th we are forgetting everything we learned with server side development of web apps, mvc, separation of concerns

Slide 52

Slide 52 text

Templates so, use ....

Slide 53

Slide 53 text

Mustache.JS https://github.com/janl/mustache.js } there are lot's of js templating languages, some examples

Slide 54

Slide 54 text

Hogan.js } http://twitter.github.com/hogan.js/ mustache but with precompiled templates -> faster

Slide 55

Slide 55 text

http://handlebarsjs.com/ also a mustache dialect, compatible but with more features, we are using that one in meteor

Slide 56

Slide 56 text

Two-way communication the last thing you saw in the demo was ... ... which means the app updates when something happens automatically, no reloads necessary

Slide 57

Slide 57 text

WebSockets http://dev.w3.org/html5/websockets/ one way to do this is websockets, if this is not available, you can fall back to other techniques, long polls, short polls, flash, ...

Slide 58

Slide 58 text

socket.io http://socket.io/ libraries helping you with that, e.g. ...

Slide 59

Slide 59 text

SockJS http://sockjs.org/ .. this one is used by meteor

Slide 60

Slide 60 text

http://www.html5rocks.com/en/ tutorials/websockets/basics/ if you want to know how websockets work, read the spec, and/or this blog article

Slide 61

Slide 61 text

Now that we did the groundwork, let's have some fun!

Slide 62

Slide 62 text

MashUps lets mash or app up with some external data

Slide 63

Slide 63 text

cool video: http:// www.youtube.com/ watch?v=OFzkTxiwziQ as you saw in the stream you often times have status updates like this one, with a link, wouldn't it be nice to not only have the link but also a preview of what's behind the link under it. of course youtube offers embedding videos in other pages through an iframe, so you could parse the url, see that it's youtube, parse out the video id, put it into the iframe url, but what if youtube changes the embed tag? what if you want to support more than youtube? lots of work to build and maintain...

Slide 64

Slide 64 text

OEmbed http://oembed.com/ fortunately there is a protocol helping us with that

Slide 65

Slide 65 text

http://www.youtube.com/watch? v=OyJd2qsRkNk in oembed if you have url like

Slide 66

Slide 66 text

you make a get request to it and get back the html page, in the head there are some links pointing to an oembed endpoint ('application/json|xml+oembed)

Slide 67

Slide 67 text

http://www.youtube.com/oembed?url=http%3A %2F%2Fwww.youtube.com%2Fwatch%3Fv %3DOyJd2qsRkNk&maxwidth=500&format=json if you query this endpoint

Slide 68

Slide 68 text

{ "provider_url":"http:\/\/www.youtube.com\/", "title":"Jupiter Jones - Das Jahr in dem ich schlief (Musik Video)", "html":"\u003cobject width=\"500\" height=\"306\"\u003e \u003cparam name=\"movie\" value=\"http:\/\/www.youtube.com\/v\/ OyJd2qsRkNk?version=3\"\u003e\u003c\/param\u003e\u003cparam name= \"allowFullScreen\" value=\"true\"\u003e\u003c\/param\u003e \u003cparam name=\"allowscriptaccess\" value=\"always\"\u003e \u003c\/param\u003e\u003cembed src=\"http:\/\/www.youtube.com\/v\/ OyJd2qsRkNk?version=3\" type=\"application\/x-shockwave-flash \" width=\"500\" height=\"306\" allowscriptaccess=\"always \" allowfullscreen=\"true\"\u003e\u003c\/embed\u003e\u003c\/object \u003e", "author_name":"St182", "height":306, "thumbnail_width":480, "width":500, "version":"1.0", "author_url":"http:\/\/www.youtube.com\/user\/Stinkfist182", "provider_name":"YouTube", "thumbnail_url":"http:\/\/i4.ytimg.com\/vi\/OyJd2qsRkNk\/ hqdefault.jpg", "type":"video", "thumbnail_height":360 } you get back a json object with meta information about the url, title, thumbnail and for videos some html which is the embed tag from youtube in this example

Slide 69

Slide 69 text

cool video: the result may then look like this

Slide 70

Slide 70 text

http://embed.ly/ if we don't want to do all this oembed endpoint lookup and have something more comfortable there is also a web service called embedly, you can just post the url to it and get back the oembed json object

Slide 71

Slide 71 text

OpenGraph http://ogp.me/ other ways to get meta information about a url are

Slide 72

Slide 72 text

The Rock (1996) ... ... with opengraph there are og meta tags in the pages head

Slide 73

Slide 73 text

Twitter Cards

Slide 74

Slide 74 text

also special twitter meta tags in the page's head

Slide 75

Slide 75 text

Microdata http://www.w3.org/html/wg/drafts/ microdata/master/ http://schema.org/ ... and ..

Slide 76

Slide 76 text

Microformats http://microformats.org/ with both these specs you annotate some dom nodes with special classes, used for example in the google search

Slide 77

Slide 77 text

Meta Tags or you can just parse some common meta tags like title or description as a fallback

Slide 78

Slide 78 text

Demo OEmbed

Slide 79

Slide 79 text

So embedding external content

Slide 80

Slide 80 text

Is this safe?

Slide 81

Slide 81 text

External HTML could lead to xss if there are script tags in the html

Slide 82

Slide 82 text

http://embed.ly/ if you use embed.ly you're somehow fine, because they are making sure nothing harmful is in there

Slide 83

Slide 83 text

Trust nobody! but ... (probably not even yourself)

Slide 84

Slide 84 text

Caja http://code.google.com/p/google-caja/ sanitize external html, caja strips out all harmful stuff, like script tags, javascript: urls

Slide 85

Slide 85 text

demo: xss, caja

Slide 86

Slide 86 text

API Access next ...

Slide 87

Slide 87 text

to demo that i'm using status.net, an opensource microblogging platform, just because on conferences i can't rely on wifi. one problem though, we want to make ajax request from js to an external domain

Slide 88

Slide 88 text

Same origin policiy problem: same origin policy, which says you are only allowed to access windows or make ajax request to resources that have the same host name as the page that you are on. very important browser security feature, otherwise you could steal the sessions of all other applications that are currently open or have a running session or permanent login cookie. but for api requests this would be fine, we can trust the api provider not to leak sessions

Slide 89

Slide 89 text

JSONP on way jsonp, a bit hacky, google it if you want details

Slide 90

Slide 90 text

Cross-Origin Resource Sharing much nicer, and good browser support

Slide 91

Slide 91 text

Backend api.twitter.com Client client.net AJAX Access-Control-Allow-Origin: * http://www.w3.org/TR/cors/ with cors, before the browser actually does the ajax request it performs an options request and the api needs to return the access-control-allow-origin header, whose value should match the requesting host. if this is the case the ajax request will be executed

Slide 92

Slide 92 text

Authorization of course most api's need authorization to access user specific resources like the user's home timeline

Slide 93

Slide 93 text

http://oauth.net/ one way to do this in javascript applications is oauth2

Slide 94

Slide 94 text

http://tools.ietf.org/html/rfc6749 it's final :)

Slide 95

Slide 95 text

User-Agent Profile i'll not explain the whole of oauth2, but just what is necessary for js application

Slide 96

Slide 96 text

http://twitter.com/authorize? &clientId=... Open Popup lanyrd.com it works like this, on the client app, lanyrd.com, you click a button, the client opens a new browser window pointing to the authorization endpoint of the api, with the clientId (you need to be preregistered at the api) of the client app (and some other parameters)

Slide 97

Slide 97 text

http://twitter.com/authorize? &clientId=... Open Popup HTTPS GET twitter.com/ authorize lanyrd.com in this popups the api provider asks the user to log in,

Slide 98

Slide 98 text

http://twitter.com/authorize? &clientId=... Open Popup Login twitter.com/ authorize lanyrd.com if he is not

Slide 99

Slide 99 text

http://twitter.com/authorize? &clientId=... Open Popup Grant Permission twitter.com/ authorize lanyrd.com and then to grant the client permission to access the api on the user's behalf

Slide 100

Slide 100 text

HTTPS Redirect RedirectURI#ac cessToken RedirectURI# accessToken lanyrd.com twitter.com/ authorize lanyrd.com if the user said yes, the user gets redirected, still in the popup, to a preregistered redirect callback url, the access token for the api is in the fragment part of this url, this means it will not be sent to the backend and not to any proxies but stays in the browser

Slide 101

Slide 101 text

RedirectURI# accessToken Parse Access Token from Fragment Send it to opening window Close popup lanyrd.com lanyrd.com the callback page can only return some static html and js. the js reads out the fragment, and since callback page and client app in the parent window are on the same hostname, it can call a function at the parent window to pass the access token to the client app and then close itself

Slide 102

Slide 102 text

HTTPS Ajax Request to API Access Token twitter.com lanyrd.com now the client app has the access token and can access the api

Slide 103

Slide 103 text

Storing the access token of course you have to store the access token somewhere. cookie would be one example, but that would send the access token over the wire and we may not want this

Slide 104

Slide 104 text

Local Storage http://www.w3.org/TR/webstorage/ better local storage

Slide 105

Slide 105 text

Get new entries pushed by the API now that we have access to api it would be nice to get new entries of the user who connected to it pushed automatically so that we can include it into our stream

Slide 106

Slide 106 text

PubSubHubbub http://code.google.com/p/pubsubhubbub/ many apis have their own implementations, status.net uses pubsubhubbub, but the principles are often the same

Slide 107

Slide 107 text

retrieves Atom feed with Hub URL Hub posts sth pings every subscriber subscribes for feed acks subscription with pubsubhubbub you subscribe your application (backend) for the users feed at a central hub that is able to provide this users feed. after some subscription pingpong, each time when the user adds a new entry to his feed, it is send to the hub and the hub will forward it to every subscriber (of course only possible backend to backend since the hub can't send it directly to a browser)

Slide 108

Slide 108 text

demo: oauth2, pubsubhubbub, sara, chat

Slide 109

Slide 109 text

http://www.webrtc.org/ next p2p realtime communication between clients, this is possible with webrtc, very new, currently only possible in chrome dev channel and canary and firefox nightly builds, you can exchange any binary data, not only text, audio, video

Slide 110

Slide 110 text

PEERJS http://peerjs.com/ one easy abstraction layer for webrtc just was published a few days ago, called peerjs, only works reliably for text at the moment, due to browser bugs (if even that)

Slide 111

Slide 111 text

http://www.html5rocks.com/en/ tutorials/webrtc/basics/ you can read up details in the spec or here

Slide 112

Slide 112 text

Try it out yourself if you want to ...

Slide 113

Slide 113 text

http://vagrantup.com/ i built a vagrant vm for it

Slide 114

Slide 114 text

https://github.com/bashofmann/vm_js_mashup

Slide 115

Slide 115 text

$ vagrant up $ vagrant ssh $ cd app $ mrt Follow the readme! basically that's what you have to do, but there are some small additional steps (adapting your host file for example, follow the readme)

Slide 116

Slide 116 text

http://twitter.com/BastianHofmann http://profiles.google.com/bashofmann http://lanyrd.com/people/BastianHofmann http://speakerdeck.com/u/bastianhofmann https://github.com/bashofmann https://www.researchgate.net/profile/Bastian_Hofmann/ [email protected] Did you like this talk? https://joind.in/7977 thanks, you can contact me on any of these platforms or via mail. if you liked this talk or didn't like it, please rate it on joind.in. this is very important for me for improving my talk, for the organizers for selecting the best talks and speakers and for you to get the best content on the conferences you visit.