Analyzing performance data with Julia and D3

http://conferences.oreilly.com/fluent/javascript-html-us/public/schedule/detail/46398

Transcript

1. A N A LY Z I N G W E B P E R F O R M A N C E D ATA W I T H J U L I A & D 3 Fluent 2016 2016-03-08
2. Philip Tellis @bluesmoon https://github.com/lognormal/boomerang http://www.soasta.com/mpulse/
3. G E T S E T U P 1. Visit bit.ly/fluent-julia-d3 and either make a copy or just download the files. 2. Sign in to juliabox.org (This might fail once or twice) 3. In the Sync tab, enter either your copy of the Google Drive folder listed above, or the bit.ly link. 4. This should add a Fluent folder in your JuliaBox. 5. Also available at bit.ly/gh-julia-d3
4. J U L I A I S … • High-level, high-performance dynamic programming language • Borrows from R, Python, MatLab, etc. • Performance comparable to C • Designed for Parallelism & Cloud Computing • Operates well on Vectors and uses SIMD where possible • MIT licensed — http://julialang.org/
5. D 3 I S … • A JavaScript library that maps Data to DOM Nodes • Extended via layouts & plugins for rich data visualizations • You still need to write code to draw things • Fast on its own, but you can easily make it sluggish • BSD Licensed — http://d3js.org/
6. J U P Y T E R + D 3 Jupyter
7. H T T P S : / / W W W. J U L I A B O X . O R G / G E T S TA R T E D W I T H J U P Y T E R
8. B A S I C J U L I A T U T O R I A L • JSON & JavaScript • Matrix Operations • Stats & DataFrames • JavaScript to update a DOM Node • Notebooks 01-04 at http://bit.ly/fluent-julia-d3
9. H T T P S : / / G I T H U B . C O M / M B O S T O C K / D 3 / W I K I / G A L L E RY G E T S TA R T E D W I T H D 3
10. B A S I C D 3 T U T O R I A L • Adding nodes • Mapping data to nodes • Data Driven Documents • Examples 01-06 at https://soasta.github.io/julia-d3- tutorial/d3/
11. I T W O U L D B E C O O L I F J U L I A C O U L D C A L L O U T T O D 3
12. T H I S I S A S I M P L E E X T E N S I O N O F W H AT W E A L R E A D Y K N O W • Create an IFRAME instead of a P • Assign to its src instead of innerText • Example notebook 05 - Include an IFrame • Example JavaScript 06-data-driven-bars
13. B U T R E A L LY, J U L I A N E E D S T O PA S S D ATA T O D 3 F O R T H I S T O B E U S E F U L
14. W E C A N U S E w i n d o w . p o s t M e s s a g e • JS in IFrame looks for data in query string or listens for “message” event • Julia code sets query string or sends message using postMessage • Example notebook 06 - Pass Data to IFrame • Example JavaScript 07-d3-external-data
15. A N D O N E M O R E T H I N G …
16. T H E W E B S O C K E T I T W O U L D B E R E A L LY C O O L T O TA P I N T O T H I S W E B S O C K E T Jupyter
17. I N T E R A C T. J L • Interact.jl taps into the WebSocket to let you build dynamic widgets • You can interact with your D3 charts and have those events fed back to the Julia Kernel • https://github.com/JuliaLang/Interact.jl • It’s also available in the tutorial section of your JuliaBox
18. J U P Y T E R ’ S R E S T A P I • Jupyter/IPython also have a REST API • Any JavaScript widget can use XHR to fetch Notebook contents and kernel details, and then set up the WebSocket • Then use the WebSocket API to execute Julia code and get results back to JavaScript
19. H T T P : / / B I T. LY / I P Y T H O N - A P I T H E I P Y T H O N A P I I S I D E N T I C A L B U T H A S B E T T E R D O C S
20. T RY A F E W C A L L S I N T H E B R O W S E R • https://juliabox.org/api/sessions • https://juliabox.org/api/kernels • https://juliabox.org/api/contents/<notebook-path> • wss://juliabox.org/api/kernels/<kernel-id>/channels? session_id=<session_id>
21. T O I N T E R A C T W I T H T H E S E R V E R • Create a new Kernel & Session • Connect to WebSocket • Execute Julia Code • Pass results back to D3 • Use Chrome Web Inspector to study web socket protocol
22. G E T A K E R N E L / S E S S I O N F O R A N O T E B O O K POST https://juliabox.org/api/sessions HTTP/1.1 Cookie: <auth-cookies> Content-type: application/json { "kernel": {"name": "julia-0.4"}, "notebook": {"path": "<notebook-path>"} } { "kernel": { "id": "688f9d0a-3291-4342-829d-55faf9d81a49", "name": "julia-0.4" }, "notebook": {"path": “<notebook-path>"}, "id": "9f5d7b39-8c32-43e7-b912-1e5b14953524" }
23. C O N N E C T T O W E B S O C K E T new WebSocket("wss://juliabox.org/api/kernels/" + encodeURIComponent(session.kernel.id) + "/channels?session_id=" + encodeURIComponent(session.id) );
24. T H E M E S S A G E F O R M AT { "header": { "msg_id":msgId, "username":"username", "session":sessionId, "msg_type":"execute_request", "version":"5.0" }, "metadata":{}, "content":{ "code":code, "silent":false, "store_history":false, "user_expressions":{}, "allow_stdin":false }, "buffers":[], "parent_header":{}, "channel":"shell" }
25. PA R S E T H E R E S P O N S E 1. JSON.parse(event.data) in sock.onmessage(event)
 This parses the Jupyter payload, not the Julia one 2. Inspect data.msg_type
 This could be "execute_result", "error", "execute_reply", "stream", among others 3. execute_result is the output of display() calls in Julia. JSON.parse this twice. 4. execute_reply and error are the status of Julia calls, inspect data.content and data.content.status 5. stream is the output of print statements or non-suppressed default output, inspect data.content.text
26. U N F O R T U N AT E LY T H E R E ’ S A B U G O N J U L I A B O X • Access-Control-Allow-Origin header responds with a * • But this violates CORS for XHR that has withCredentials=true • https://github.com/JuliaLang/JuliaBox/issues/367
27. S U M M A RY • Write Julia code to analyze and summarize data • Convert data to a Dict() if necessary, and JSON encode it • Load your D3 visualization in an iframe • Use postMessage to pass data as JSON between the Julia frame and the D3 frame • Or use the REST API to execute code via the WebSocket
28. Thank You