Slide 1

Slide 1 text

Realtime Web Applications Osvaldo Matos Júnior

Slide 2

Slide 2 text

Traditional web page

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Dynamic web page

Slide 5

Slide 5 text

Dynamic web page

Slide 6

Slide 6 text

Realtime web page

Slide 7

Slide 7 text

Realtime web page

Slide 8

Slide 8 text

Realtime web page Chat News Feed Notifications Comments Vote up/down Text edition … many things to update

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

JusBrasil Challenge

Slide 11

Slide 11 text

Push vs Pull

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

JusBrasil Live Architecture

Slide 15

Slide 15 text

JusLive 1.0

Slide 16

Slide 16 text

Inspired on Quora Webnode2 and LiveNode http://www.quora.com/Shreyes-Seshasai/Posts/Tech-Talk-webnode2-and-LiveNode

Slide 17

Slide 17 text

Component-Based www www www

Slide 18

Slide 18 text

Component-Based

Slide 19

Slide 19 text

Component-Based

Slide 20

Slide 20 text

Component Types Complex Simple

Slide 21

Slide 21 text

Topic Follow Button Before Click After Click

Slide 22

Slide 22 text

Python Java Script HTML

Slide 23

Slide 23 text

Python class TopicFollowButton(Component): __alias__ = 'topic.followbutton' def __init__(self, tid, topic=None): self.tid = tid self.following = False self.topic = topic or load_from_database(self.tid) def follow(self, tid): self.following = True def unfollow(self, tid): self.following = False def __html__(self): return render('components/topic_followbutton.html', topic=self.topic, following=self.following, id=self.id)

Slide 24

Slide 24 text

Python class Component(object): def __init__(self, **kwargs): self.__dict__.update(kwargs) @property def name(self): return self.__class__.__name__ @property def id(self): return '__%s' % rand_id(ID_SIZE) def __html__(self): return NotImplementedError() def __js__(self): return 'new %s("%s", "", %s, "%s")' % (self.name, self.id, self.js_init, self.__alias__)

Slide 25

Slide 25 text

Controller @app.route('/topic//') def topic_view(tid, slug): topic = get_topic(tid) components = { 'topic_follow_button' = TopicFollowButton(topic) } return render('topic/view.html', components=components)

Slide 26

Slide 26 text

Page rendering # templates/topic/view.html ... ... loadComponents([new TopicFollowButton("__6YekPR9", "", {"tid": 10000036}, "topic.followbutton")]);

Slide 27

Slide 27 text

HTML # templates/componentes/topic_followbutton.html {% if following %} {% else %} Seguir Tópico {% endif %}

Slide 28

Slide 28 text

Load Components # templates/topic/view.html ... ... loadComponents([new TopicFollowButton("__6YekPR9", "", {"tid": 10000036}, "topic.followbutton")]);

Slide 29

Slide 29 text

Java Script TopicFollowButton = Component.extend({ onload: function() { this.$('#@follow').click( bind( this, function(){ this.serverCall('follow') .kwargs({tid : this.tid }) .target( this.$('#@follow')) .send(); return false; })); this.$('#@unfollow').click( bind( this, function(){ this.serverCall('unfollow') .kwargs({tid : this.tid}) .target( this.$('#@unfollow')) .send(); return false; })); } });

Slide 30

Slide 30 text

The @ prefix Each component has a unique ID. The @ sign is replaced by the component's ID: $('#@follow') -> $('#__xyz_follow')

Slide 31

Slide 31 text

serverCall() /ajax? callback=jQuery18206321731214411557_1399129 850063&json=%7B%22args%22%3A%5B%5D% 2C%22kwargs%22%3A%7B%7D% 7D&__vcon_json=%22topic.followbutton% 22&__vcon_method=follow&js_init=%7B% 22tid%22%3A542484%2C%22

Slide 32

Slide 32 text

@app.route('/ajax') def ajax(): module_alias = params.get('__vcon_json', None) method = params.get('__vcon_method') js_init = json.loads(params.get('js_init', "{}")) _json = json.loads(params.get('json', '{"args":[],"kwargs":{}}')) args = _json['args'] kwargs = _json['kwargs'] module = COMPONENTS[module_alias](**js_init) func = getattr(module, method, None) result = func(*args, **kwargs) data = {'js': result.__js__(), 'html': result.__html__()} return json.dumps(data)

Slide 33

Slide 33 text

Server-side Rendering

Slide 34

Slide 34 text

Register and Update register: register the events that the client is interested to. update: wait for updates (long polling).

Slide 35

Slide 35 text

Register and Update /register www www /updates

Slide 36

Slide 36 text

Register and Update

Slide 37

Slide 37 text

Ajax Cross-Domain JusBrasil uses subdomains for profile pages. http://stf.jusbrasil.com.br/ http://www.jusbrasil.com.br/ajax http://www.jusbrasil.com.br/live/update JSONP with subdomain: http://stf.jusbrasil.com.br/ajax http://live.jusbrasil.com.br/update Limit of browser connections per host http://xyz.live.jusbrasil.com.br/update

Slide 38

Slide 38 text

Redis

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Event Table event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ...

Slide 42

Slide 42 text

Event Table event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ...

Slide 43

Slide 43 text

Window Table window_ID events window_1 event1_update window_2 event1_update window_3 event1_update window_n ...

Slide 44

Slide 44 text

Action Update pop(window) push(event) gunicorn tornado

Slide 45

Slide 45 text

HAProxy Load Balancer webapp Redis Broker tornado tornado tornado tornado Webnode Machines JusBrasil Live Architecture

Slide 46

Slide 46 text

checks for window updates tornado.ioloop.PeriodicCallback() 200ms

Slide 47

Slide 47 text

Heavy Network Traffic

Slide 48

Slide 48 text

JusLive 2.0

Slide 49

Slide 49 text

Goal: Remove Redis Polling 200ms polling sucks!

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

context = zmq.Context() subscribe = context.socket(zmq.SUB) message = subscribe.recv() # block ZeroMQ

Slide 52

Slide 52 text

ZMQStream sub = zmq_context.socket(zmq.SUB) ... stream = ZMQStream(sub) stream.on_recv(handle_message)

Slide 53

Slide 53 text

socket.bind(1001) socket.connect(1001) socket.connect(1001) listen:1001 socket.connect(1001)

Slide 54

Slide 54 text

new event Publisher Subscribers

Slide 55

Slide 55 text

MANY TO MANY

Slide 56

Slide 56 text

ZMQ Forwarder OUTGOING

Slide 57

Slide 57 text

HAProxy Load Balancer webapp ZMQ Broker tornado tornado tornado tornado Webnode Machines JusBrasil Live Architecture

Slide 58

Slide 58 text

Live Broker (Ring?) Live Broker Live Broker Live Broker Live Broker Live Broker

Slide 59

Slide 59 text

Live Broker (Ring?) Live Broker Live Broker Live Broker Live Broker Live Broker

Slide 60

Slide 60 text

Live Broker (Fully Connected) Live Broker Live Broker Live Broker Live Broker Live Broker

Slide 61

Slide 61 text

Live Broker: Broadcast Live Broker Live Broker Live Broker Live Broker Live Broker webapp tornado tornado tornado tornado tornado tornado

Slide 62

Slide 62 text

Tornado notifies all connected clients

Slide 63

Slide 63 text

event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ... window_ID events window_1 event1_update window_2 event1_update window_3 event1_update window_n ...

Slide 64

Slide 64 text

STATELESS!

Slide 65

Slide 65 text

WEBSOCKET!

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

Disadvantages

Slide 68

Slide 68 text

var _components = [new Feedback("__p7qstXI", "", {}, "feedback"),new MessagesAlerts("__dPIyzM0", "", {"render_type": "new", "pid": 3700432, "unseen" "last_sender": 3702921}, "profile.messages_alerts"),new CasesAlerts("__1ieMhRN", "", {"pid": 3700432, "unseen": 0}, "profile.cases_alerts"),new NotificationsAlerts("__UkjlYLy", "", {"unseen_notifications": 5, "pid": 3700432}, "profile.notifications_alerts"),new Newsletter("__f29y3Jo", "", {"called": false "newsletter"),new CompleteProfileAlert("__KeCvcGR", "", {"pid": 3700432, "unseen": true, "recent_updated": false}, "profile.complete_profile"),new AutocompleteMainSearch("__mrU2M5S", "", {"search": null}, "autocomplete_mainsearch"),new DocumentVote("__M32ozic", "", {"artefato": 3, "render_type "document_feed", "document_type": "noticias", "docid": 118270826}, "document.vote"),new WhoVotedDocument("__lSvXcYJ", "", {"doc_id": 118270826, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IpblN2W", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__RXJnGiA", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118242504}, "document.vote"),new WhoVotedDocument("__yLikWX5 "", {"doc_id": 118242504, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__3awaVGX", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__cKsif3u", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118245933}, "document.vote"),new WhoVotedDocument("__vJHcxEm", "", {"doc_id": 118245933, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__eNY6HJu", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__VXXnzuh", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118228731}, "document.vote"),new WhoVotedDocument("__YYlNK7s", "", {"doc_id": 118228731, "artifact": 4}, "who_voted"),new FeedProfileInteractions ("__6WEt2w1", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__eN2mIQ6", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118221888}, "document.vote"),new WhoVotedDocument("__w8CMmob", "", {"doc_id": 118221888, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__9mXMTdF", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__zruF4Fm", "", {"artefato": "render_type": "document_feed", "document_type": "noticias", "docid": 118253351}, "document.vote"),new WhoVotedDocument("__yNK4nd6", "", {"doc_id 118253351, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__qv2mGGs", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__pHlx9IP", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118268126}, "document.vote"),new WhoVotedDocument("__2tkeaWe", "", {"doc_id": 118268126, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__SjT24P9", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__cM7nlim", "", {"artefato": 2, "render_type": "document_feed", "document_type": "artigo", "docid": 118241033}, "document.vote"),new WhoVotedDocument("__Zo99beH", "", {"doc_id": 118241033, "artifact": 2}, "who_voted"),new FeedProfileInteractions ("__LxqNB1a", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__8DaBbSU", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118236134}, "document.vote"),new WhoVotedDocument("__DAUsbUr", "", {"doc_id": 118236134, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__Vx4g7Rh", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__vszEzko", "", {"artefato": 3 "render_type": "document_feed", "document_type": "noticias", "docid": 118262774}, "document.vote"),new WhoVotedDocument("__3AkNtHt", "", {"doc_id" 118262774, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__q6TJcUM", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__HTIJpBY", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118210064}, "document.vote"),new WhoVotedDocument("__tQnAK0d", "", {"doc_id": 118210064, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IizEP09", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__Gex2Y7E", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118252543}, "document.vote"),new WhoVotedDocument("__m9mbXOY", "", {"doc_id": 118252543, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__IbWOXEw", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__hPl0Bu9", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118262349}, "document.vote"),new WhoVotedDocument("__7Zz8F2C", "", {"doc_id": 118262349, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__2owbSIO", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__ciMtOE9", "", {"artefato": 4 "render_type": "document_feed", "document_type": "politica", "docid": 118260885}, "document.vote"),new WhoVotedDocument("__71M5cYi", "", {"doc_id" 118260885, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__mY0xuV5", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__G5OjPnO", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118265182}, "document.vote"),new WhoVotedDocument("__Bm4e5G0", "", {"doc_id": 118265182, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IPUaGGl", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__as8WaHF", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118262490}, "document.vote"),new WhoVotedDocument("__AFRYceq", "", {"doc_id": 118262490, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__RKLbRqE", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__hvPj4sb", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118255151}, "document.vote"),new WhoVotedDocument("__ZyJDky1", "", {"doc_id": 118255151, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__q0waQQB", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__G6osoSF", "", {"artefato": "render_type": "document_feed", "document_type": "noticias", "docid": 118224685}, "document.vote"),new WhoVotedDocument("__kOCEs7f", "", {"doc_id" 118224685, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__Jg9VAEn", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__sOYJzl8", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118257726}, "document.vote"),new WhoVotedDocument("__0D5A0F7", "", {"doc_id": 118257726, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__w9omLuQ", "", {"profile_ids": []} "perfil.feed_interactions"),new DocumentVote("__ekLFRyy", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118257838}, "document.vote"),new WhoVotedDocument("__dDXbWLF", "", {"doc_id": 118257838, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__zKvCf77", "", {"profile_ids": []}, "perfil.feed_interactions"),new MoreQuentes("__aKOylBg", "", {"last_doc": {"artefato": "NOTICIAS", "idDocumento":

Slide 69

Slide 69 text

Not recommended for single user applications

Slide 70

Slide 70 text

Authorization github.com/jusbrasil/pycan

Slide 71

Slide 71 text

Thanks! That's all for today @tupydabahia github.com/tupy