JavaScript as Data Processing Language & HTML5 Integration

These last years, data management has become a trending and very dynamic subject. Subjects and tech trends like NoSQL, localcache, offline synchronization or message brokers are now at the core of projects infrastructures. This is linked to the new trend around functional programing, for data manipulation.
The talk goal is to show connections between NoSQL, data processing, message broking and JavaScript. How to quickly prototype with these tools! Are these tools only for mass storage or raw performance? Does it allow you to code faster?

http://qconlondon.com/london-2013/presentation/JavaScript%20as%20Data%20Processing%20Language%20&%20HTML5%20Integration

Transcript

1. + Javascript as a data processing language & htlm5 integration Qcon London 2013 – Quentin ADAM
2. + Who I am ? n  Quentin ADAM n  @waxzce on twitter n  CEO of Clever Cloud, the most powerful PaaS in Europe ;-) clever c l o u d
3. + Why we have to talk about data processing ?
4. + Big change in software industry n  One instance n  One organization n  One data repository n  One instance (distributed) n  Multiple organization + a lot of users n  One data repository Yesteday Now Multi-tenant
5. + Instance for one Organization ACID • Atomicity • Consistency • Isolation • Durability Powerful data management •  Transaction •  User management •  One above one Take advantage of ACID database
6. + But it designs a bottleneck
7. + Example : e-shop on classic mode
8. + Example : e-shop on classic mode User A buy a hdd Database Transaction : •  Stock management •  Order management •  Invoice generation •  Customer Account reward •  … Transaction user A is processed Stock & Order are just perfectly synchronize
9. + Example : e-shop on classic mode User A buy a hdd Database Transaction : •  Stock management •  Order management •  Invoice generation •  Customer Account reward •  … Transaction user A is processed Stock & Order are just perfectly synchronize User B buy a hdd Transaction user B is processed then
10. + Example : e-shop on multi-tenant mode n  i.e. : Multiple shop of various sellers on the same instance
11. + Example : e-shop on multi-tenant mode User A buy a hdd on seller A Database Transaction : •  Stock management •  Order management •  Invoice generation •  Customer Account reward •  … Transaction user A is processed Stock & Order are just perfectly synchronize User B buy a book on seller B Transaction user B is processed then
12. + Who care the synchronization of vendor A & B ?
13. + Destroy the bottle neck and switch to another storage system This is why the noSQL movement start.
14. + noSQL = not only SQL
15. + Now talking about how to get data from storage
16. + SQL don’t solve all the data processing expression.
17. + Some javascripts facts n  Run for server needs at the beginning n  Netscape Server in 1996, and then in IIS n  Clearly a major language n  Run on n  Browser n  Mobile n  Servers n  Easy to integrate n  Most peoples know it n  Most used language on github n  Event driven
18. + So, can we use javascript as a data processing language ?
19. + Data processing is focus on n  Find n  Filter n  Manage collection n  CRUD order
20. + Map / reduce => the big bang of new data processing n  Atomize the problem n  Easy to parallelize n  From the functional programing model Get the result Reduce : filter data & aggregate Map : transform data to what you need (apply function) Get a k/v data
21. + Do you wanna some functional programing ? Javascript is so flexible than we can apply several patern on top of the language
22. + And map/reduce ?
23. + NoSQL we can considerate for example n  Object Store n  Key/Value with documents capacities n  Pure Document n  Column based
24. + NoSQL we can considerate for example n  Object Store n  Key/Value with documents capacities n  Pure Document n  Column based
25. + And one more ! With localStorage features
26. + JSON domination n  Store n  Processing n  Access
27. + Example with riak n  Key/Value n  http REST & protobuff data API n  Store JSON data on a URL n  Very good distribution n  Good performances
28. + Example with riak map/reduce
29. + Put some real time n  Lots lots of noSQL DB handle a _change_ pipe : n  All datas and updates stream n  Bind it on a websocket : put some real time on your application
30. + Data as a Service API clients •  Full web client : html5 + local storage •  Mobile : phonegapp or native •  Other servers app API RESTFull Various backen DB
31. + Data as a Service n  Data as others service n  With the same language the application n  Build sync algorithm to start offline / online software n  Some protocol : atom, gdata, oData n  Some popular API : S3, twitter, EC2… n  Security and access layer : oauth…
32. + DataBase as a Service n  Xeround n  Amazon dynamo n  Lots of heroku addon n  Kinvey PaaS
33. + Transparent data consumption n  Webkit JavaScriptCore n  REST / JSON n  Data Classes n  auto-updatable n  accessors n  events n  Methods n  Cool html5 IDE n  Best use case : work oriented app Atomatic data access framework
34. + Wakanda example http://play.wakanda.org/
35. + How to start ?
36. + Find a toy project
37. + Use hosted services n  Less ops footprint n  No more instalation n  Quick learn curve n  Focus on your needs & values
38. + Innovate & write kick ass apps
39. + Thx for Listening & Q/A time clever c l o u d http://twitter.com/waxzce Quentin ADAM on google ;-) quentin.adam@clever-cloud.com n  http://www.clever-cloud.com n  http://engineering.clever-cloud.com/ Thx to the http://qconlondon.com