Realtime Web Applications Osvaldo Matos Júnior 

Traditional web page 

No content

Dynamic web page 

Dynamic web page 

Realtime web page 

Realtime web page 

Realtime web page Chat News Feed Notifications Comments Vote up/down Text edition … many things to update 

No content

JusBrasil Challenge 

Push vs Pull 

No content

No content

JusBrasil Live Architecture 

JusLive 1.0 

Inspired on Quora Webnode2 and LiveNode http://www.quora.com/Shreyes-Seshasai/Posts/Tech-Talk-webnode2-and-LiveNode 

Component-Based www www www 

Component-Based 

Component-Based 

Component Types Complex Simple 

Topic Follow Button Before Click After Click 

Python Java Script HTML 

Python class TopicFollowButton(Component): __alias__ = 'topic.followbutton' def __init__(self, tid, topic=None): self.tid = tid self.following = False self.topic = topic or load_from_database(self.tid) def follow(self, tid): self.following = True def unfollow(self, tid): self.following = False def __html__(self): return render('components/topic_followbutton.html', topic=self.topic, following=self.following, id=self.id) 

Python class Component(object): def __init__(self, **kwargs): self.__dict__.update(kwargs) @property def name(self): return self.__class__.__name__ @property def id(self): return '__%s' % rand_id(ID_SIZE) def __html__(self): return NotImplementedError() def __js__(self): return 'new %s("%s", "", %s, "%s")' % (self.name, self.id, self.js_init, self.__alias__) 

Controller @app.route('/topic//') def topic_view(tid, slug): topic = get_topic(tid) components = { 'topic_follow_button' = TopicFollowButton(topic) } return render('topic/view.html', components=components) 

Page rendering # templates/topic/view.html ... ... loadComponents([new TopicFollowButton("__6YekPR9", "", {"tid": 10000036}, "topic.followbutton")]); 

HTML # templates/componentes/topic_followbutton.html {% if following %} {% else %} Seguir Tópico {% endif %} 

Load Components # templates/topic/view.html ... ... loadComponents([new TopicFollowButton("__6YekPR9", "", {"tid": 10000036}, "topic.followbutton")]); 

Java Script TopicFollowButton = Component.extend({ onload: function() { this.$('#@follow').click( bind( this, function(){ this.serverCall('follow') .kwargs({tid : this.tid }) .target( this.$('#@follow')) .send(); return false; })); this.$('#@unfollow').click( bind( this, function(){ this.serverCall('unfollow') .kwargs({tid : this.tid}) .target( this.$('#@unfollow')) .send(); return false; })); } }); 

The @ prefix Each component has a unique ID. The @ sign is replaced by the component's ID: $('#@follow') -> $('#__xyz_follow') 

serverCall() /ajax? callback=jQuery18206321731214411557_1399129 850063&json=%7B%22args%22%3A%5B%5D% 2C%22kwargs%22%3A%7B%7D% 7D&__vcon_json=%22topic.followbutton% 22&__vcon_method=follow&js_init=%7B% 22tid%22%3A542484%2C%22 

@app.route('/ajax') def ajax(): module_alias = params.get('__vcon_json', None) method = params.get('__vcon_method') js_init = json.loads(params.get('js_init', "{}")) _json = json.loads(params.get('json', '{"args":[],"kwargs":{}}')) args = _json['args'] kwargs = _json['kwargs'] module = COMPONENTS[module_alias](**js_init) func = getattr(module, method, None) result = func(*args, **kwargs) data = {'js': result.__js__(), 'html': result.__html__()} return json.dumps(data) 

Server-side Rendering 

Register and Update register: register the events that the client is interested to. update: wait for updates (long polling). 

Register and Update /register www www /updates 

Register and Update 

Ajax Cross-Domain JusBrasil uses subdomains for profile pages. http://stf.jusbrasil.com.br/ http://www.jusbrasil.com.br/ajax http://www.jusbrasil.com.br/live/update JSONP with subdomain: http://stf.jusbrasil.com.br/ajax http://live.jusbrasil.com.br/update Limit of browser connections per host http://xyz.live.jusbrasil.com.br/update 

Redis 

No content

No content

Event Table event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ... 

Event Table event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ... 

Window Table window_ID events window_1 event1_update window_2 event1_update window_3 event1_update window_n ... 

Action Update pop(window) push(event) gunicorn tornado 

HAProxy Load Balancer webapp Redis Broker tornado tornado tornado tornado Webnode Machines JusBrasil Live Architecture 

checks for window updates tornado.ioloop.PeriodicCallback() 200ms 

Heavy Network Traffic 

JusLive 2.0 

Goal: Remove Redis Polling 200ms polling sucks! 

No content

context = zmq.Context() subscribe = context.socket(zmq.SUB) message = subscribe.recv() # block ZeroMQ 

ZMQStream sub = zmq_context.socket(zmq.SUB) ... stream = ZMQStream(sub) stream.on_recv(handle_message) 

socket.bind(1001) socket.connect(1001) socket.connect(1001) listen:1001 socket.connect(1001) 

new event Publisher Subscribers 

MANY TO MANY 

ZMQ Forwarder OUTGOING 

HAProxy Load Balancer webapp ZMQ Broker tornado tornado tornado tornado Webnode Machines JusBrasil Live Architecture 

Live Broker (Ring?) Live Broker Live Broker Live Broker Live Broker Live Broker 

Live Broker (Ring?) Live Broker Live Broker Live Broker Live Broker Live Broker 

Live Broker (Fully Connected) Live Broker Live Broker Live Broker Live Broker Live Broker 

Live Broker: Broadcast Live Broker Live Broker Live Broker Live Broker Live Broker webapp tornado tornado tornado tornado tornado tornado 

Tornado notifies all connected clients 

event_ID window_IDs event_1 window_1, window_2, window_3 event_2 window_2, window_5 event_3 window_1, window_5 event_n ... window_ID events window_1 event1_update window_2 event1_update window_3 event1_update window_n ... 

STATELESS! 

WEBSOCKET! 

No content

Disadvantages 

var _components = [new Feedback("__p7qstXI", "", {}, "feedback"),new MessagesAlerts("__dPIyzM0", "", {"render_type": "new", "pid": 3700432, "unseen" "last_sender": 3702921}, "profile.messages_alerts"),new CasesAlerts("__1ieMhRN", "", {"pid": 3700432, "unseen": 0}, "profile.cases_alerts"),new NotificationsAlerts("__UkjlYLy", "", {"unseen_notifications": 5, "pid": 3700432}, "profile.notifications_alerts"),new Newsletter("__f29y3Jo", "", {"called": false "newsletter"),new CompleteProfileAlert("__KeCvcGR", "", {"pid": 3700432, "unseen": true, "recent_updated": false}, "profile.complete_profile"),new AutocompleteMainSearch("__mrU2M5S", "", {"search": null}, "autocomplete_mainsearch"),new DocumentVote("__M32ozic", "", {"artefato": 3, "render_type "document_feed", "document_type": "noticias", "docid": 118270826}, "document.vote"),new WhoVotedDocument("__lSvXcYJ", "", {"doc_id": 118270826, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IpblN2W", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__RXJnGiA", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118242504}, "document.vote"),new WhoVotedDocument("__yLikWX5 "", {"doc_id": 118242504, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__3awaVGX", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__cKsif3u", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118245933}, "document.vote"),new WhoVotedDocument("__vJHcxEm", "", {"doc_id": 118245933, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__eNY6HJu", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__VXXnzuh", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118228731}, "document.vote"),new WhoVotedDocument("__YYlNK7s", "", {"doc_id": 118228731, "artifact": 4}, "who_voted"),new FeedProfileInteractions ("__6WEt2w1", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__eN2mIQ6", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118221888}, "document.vote"),new WhoVotedDocument("__w8CMmob", "", {"doc_id": 118221888, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__9mXMTdF", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__zruF4Fm", "", {"artefato": "render_type": "document_feed", "document_type": "noticias", "docid": 118253351}, "document.vote"),new WhoVotedDocument("__yNK4nd6", "", {"doc_id 118253351, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__qv2mGGs", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__pHlx9IP", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118268126}, "document.vote"),new WhoVotedDocument("__2tkeaWe", "", {"doc_id": 118268126, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__SjT24P9", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__cM7nlim", "", {"artefato": 2, "render_type": "document_feed", "document_type": "artigo", "docid": 118241033}, "document.vote"),new WhoVotedDocument("__Zo99beH", "", {"doc_id": 118241033, "artifact": 2}, "who_voted"),new FeedProfileInteractions ("__LxqNB1a", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__8DaBbSU", "", {"artefato": 4, "render_type": "document_feed", "document_type": "politica", "docid": 118236134}, "document.vote"),new WhoVotedDocument("__DAUsbUr", "", {"doc_id": 118236134, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__Vx4g7Rh", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__vszEzko", "", {"artefato": 3 "render_type": "document_feed", "document_type": "noticias", "docid": 118262774}, "document.vote"),new WhoVotedDocument("__3AkNtHt", "", {"doc_id" 118262774, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__q6TJcUM", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__HTIJpBY", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118210064}, "document.vote"),new WhoVotedDocument("__tQnAK0d", "", {"doc_id": 118210064, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IizEP09", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__Gex2Y7E", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118252543}, "document.vote"),new WhoVotedDocument("__m9mbXOY", "", {"doc_id": 118252543, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__IbWOXEw", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__hPl0Bu9", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118262349}, "document.vote"),new WhoVotedDocument("__7Zz8F2C", "", {"doc_id": 118262349, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__2owbSIO", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__ciMtOE9", "", {"artefato": 4 "render_type": "document_feed", "document_type": "politica", "docid": 118260885}, "document.vote"),new WhoVotedDocument("__71M5cYi", "", {"doc_id" 118260885, "artifact": 4}, "who_voted"),new FeedProfileInteractions("__mY0xuV5", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__G5OjPnO", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118265182}, "document.vote"),new WhoVotedDocument("__Bm4e5G0", "", {"doc_id": 118265182, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__IPUaGGl", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__as8WaHF", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118262490}, "document.vote"),new WhoVotedDocument("__AFRYceq", "", {"doc_id": 118262490, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__RKLbRqE", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__hvPj4sb", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118255151}, "document.vote"),new WhoVotedDocument("__ZyJDky1", "", {"doc_id": 118255151, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__q0waQQB", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote("__G6osoSF", "", {"artefato": "render_type": "document_feed", "document_type": "noticias", "docid": 118224685}, "document.vote"),new WhoVotedDocument("__kOCEs7f", "", {"doc_id" 118224685, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__Jg9VAEn", "", {"profile_ids": []}, "perfil.feed_interactions"),new DocumentVote ("__sOYJzl8", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118257726}, "document.vote"),new WhoVotedDocument("__0D5A0F7", "", {"doc_id": 118257726, "artifact": 3}, "who_voted"),new FeedProfileInteractions("__w9omLuQ", "", {"profile_ids": []} "perfil.feed_interactions"),new DocumentVote("__ekLFRyy", "", {"artefato": 3, "render_type": "document_feed", "document_type": "noticias", "docid": 118257838}, "document.vote"),new WhoVotedDocument("__dDXbWLF", "", {"doc_id": 118257838, "artifact": 3}, "who_voted"),new FeedProfileInteractions ("__zKvCf77", "", {"profile_ids": []}, "perfil.feed_interactions"),new MoreQuentes("__aKOylBg", "", {"last_doc": {"artefato": "NOTICIAS", "idDocumento": 

Not recommended for single user applications 

Authorization github.com/jusbrasil/pycan 

Thanks! That's all for today @tupydabahia github.com/tupy