Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Analyzing performance data with Julia and D3

Analyzing performance data with Julia and D3

Philip Tellis

March 08, 2016

More Decks by Philip Tellis

Other Decks in Technology


  1. A N A LY Z I N G W E

    B P E R F O R M A N C E D ATA W I T H J U L I A & D 3 Fluent 2016 2016-03-08
  2. G E T S E T U P 1. Visit

    bit.ly/fluent-julia-d3 and either make a copy or just download the files. 2. Sign in to juliabox.org (This might fail once or twice) 3. In the Sync tab, enter either your copy of the Google Drive folder listed above, or the bit.ly link. 4. This should add a Fluent folder in your JuliaBox. 5. Also available at bit.ly/gh-julia-d3
  3. J U L I A I S … • High-level,

    high-performance dynamic programming language • Borrows from R, Python, MatLab, etc. • Performance comparable to C • Designed for Parallelism & Cloud Computing • Operates well on Vectors and uses SIMD where possible • MIT licensed — http://julialang.org/
  4. D 3 I S … • A JavaScript library that

    maps Data to DOM Nodes • Extended via layouts & plugins for rich data visualizations • You still need to write code to draw things • Fast on its own, but you can easily make it sluggish • BSD Licensed — http://d3js.org/
  5. H T T P S : / / W W

    W. J U L I A B O X . O R G / G E T S TA R T E D W I T H J U P Y T E R
  6. B A S I C J U L I A

    T U T O R I A L • JSON & JavaScript • Matrix Operations • Stats & DataFrames • JavaScript to update a DOM Node • Notebooks 01-04 at http://bit.ly/fluent-julia-d3
  7. H T T P S : / / G I

    T H U B . C O M / M B O S T O C K / D 3 / W I K I / G A L L E RY G E T S TA R T E D W I T H D 3
  8. B A S I C D 3 T U T

    O R I A L • Adding nodes • Mapping data to nodes • Data Driven Documents • Examples 01-06 at https://soasta.github.io/julia-d3- tutorial/d3/
  9. I T W O U L D B E C

    O O L I F J U L I A C O U L D C A L L O U T T O D 3
  10. T H I S I S A S I M

    P L E E X T E N S I O N O F W H AT W E A L R E A D Y K N O W • Create an IFRAME instead of a P • Assign to its src instead of innerText • Example notebook 05 - Include an IFrame • Example JavaScript 06-data-driven-bars
  11. B U T R E A L LY, J U

    L I A N E E D S T O PA S S D ATA T O D 3 F O R T H I S T O B E U S E F U L
  12. W E C A N U S E w i

    n d o w . p o s t M e s s a g e • JS in IFrame looks for data in query string or listens for “message” event • Julia code sets query string or sends message using postMessage • Example notebook 06 - Pass Data to IFrame • Example JavaScript 07-d3-external-data
  13. A N D O N E M O R E

    T H I N G …
  14. T H E W E B S O C K

    E T I T W O U L D B E R E A L LY C O O L T O TA P I N T O T H I S W E B S O C K E T Jupyter
  15. I N T E R A C T. J L

    • Interact.jl taps into the WebSocket to let you build dynamic widgets • You can interact with your D3 charts and have those events fed back to the Julia Kernel • https://github.com/JuliaLang/Interact.jl • It’s also available in the tutorial section of your JuliaBox
  16. J U P Y T E R ’ S R

    E S T A P I • Jupyter/IPython also have a REST API • Any JavaScript widget can use XHR to fetch Notebook contents and kernel details, and then set up the WebSocket • Then use the WebSocket API to execute Julia code and get results back to JavaScript
  17. H T T P : / / B I T.

    LY / I P Y T H O N - A P I T H E I P Y T H O N A P I I S I D E N T I C A L B U T H A S B E T T E R D O C S
  18. T RY A F E W C A L L

    S I N T H E B R O W S E R • https://juliabox.org/api/sessions • https://juliabox.org/api/kernels • https://juliabox.org/api/contents/<notebook-path> • wss://juliabox.org/api/kernels/<kernel-id>/channels? session_id=<session_id>
  19. T O I N T E R A C T

    W I T H T H E S E R V E R • Create a new Kernel & Session • Connect to WebSocket • Execute Julia Code • Pass results back to D3 • Use Chrome Web Inspector to study web socket protocol
  20. G E T A K E R N E L

    / S E S S I O N F O R A N O T E B O O K POST https://juliabox.org/api/sessions HTTP/1.1 Cookie: <auth-cookies> Content-type: application/json { "kernel": {"name": "julia-0.4"}, "notebook": {"path": "<notebook-path>"} } { "kernel": { "id": "688f9d0a-3291-4342-829d-55faf9d81a49", "name": "julia-0.4" }, "notebook": {"path": “<notebook-path>"}, "id": "9f5d7b39-8c32-43e7-b912-1e5b14953524" }
  21. C O N N E C T T O W

    E B S O C K E T new WebSocket("wss://juliabox.org/api/kernels/" + encodeURIComponent(session.kernel.id) + "/channels?session_id=" + encodeURIComponent(session.id) );
  22. T H E M E S S A G E

    F O R M AT { "header": { "msg_id":msgId, "username":"username", "session":sessionId, "msg_type":"execute_request", "version":"5.0" }, "metadata":{}, "content":{ "code":code, "silent":false, "store_history":false, "user_expressions":{}, "allow_stdin":false }, "buffers":[], "parent_header":{}, "channel":"shell" }
  23. PA R S E T H E R E S

    P O N S E 1. JSON.parse(event.data) in sock.onmessage(event)
 This parses the Jupyter payload, not the Julia one 2. Inspect data.msg_type
 This could be "execute_result", "error", "execute_reply", "stream", among others 3. execute_result is the output of display() calls in Julia. JSON.parse this twice. 4. execute_reply and error are the status of Julia calls, inspect data.content and data.content.status 5. stream is the output of print statements or non-suppressed default output, inspect data.content.text
  24. U N F O R T U N AT E

    LY T H E R E ’ S A B U G O N J U L I A B O X • Access-Control-Allow-Origin header responds with a * • But this violates CORS for XHR that has withCredentials=true • https://github.com/JuliaLang/JuliaBox/issues/367
  25. S U M M A RY • Write Julia code

    to analyze and summarize data • Convert data to a Dict() if necessary, and JSON encode it • Load your D3 visualization in an iframe • Use postMessage to pass data as JSON between the Julia frame and the D3 frame • Or use the REST API to execute code via the WebSocket