Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python and Julia:: The Best of Friends?

Python and Julia:: The Best of Friends?

Malcolm Sherrington

May 06, 2016
Tweet

More Decks by Malcolm Sherrington

Other Decks in Programming

Transcript

  1. Part 1: Introduction 1 Python and Julia
 The Best of

    Friends ? Dr Malcolm Sherrington PyData London 6th May 2016 https://github.com/sherrinm/notebooks
  2. Part 1: Introduction 2 Warning !!! I have prepared 524

    slides for today’s presentation. So if you want to survive this workshop, interrupt me, ask questions and challenge my options.
  3. Part 1: Introduction 3 Steven G. Johnston Professor of Applied

    Mathematics and Physics Massachusetts Institute of Technology, Cambridge, MA.
  4. Part 1: Introduction 4 Synopsis of the Workshop Is Julia

    really fast ? • If so, then why … • Can this be retro-fitted to existing languages?
 How Julia gains from the interaction with Python.
 Can Python do the same (in reverse)?
  5. Part 1: Introduction 5 The “Two-Language” Problem We would like

    to have a scripting language that you can work with interactively: • ease of development, prototyping • data exploration • dynamic typing, visualisation Can use from Python, Matlab/Octave, R, Lisp …
 Critical parts of the coding need to be written in a lower language such as C, Fortran or Java.
 This approach has its drawbacks: • the hard parts are where we need an easier language. • it forces vectorization where it may be inappropriate. • it creates a division between the analyst and programmer.
  6. Part 1: Introduction 6 Julia apparently solves this problem by

    compiling down to native machine code. This uses the LLVM compiler technology and Julia is not alone: • Google V8 • Clang • Swift • LuaJIT Some Python developers are trying to do much the same with: • Numba • Parakeet • Pyston
 Is this going to work?
  7. Part 1: Introduction 7 Is Julia really all that fast

    ? Seemingly so with benchmarks …
  8. Part 1: Introduction 8 … but how about for Enterprise

    computing Pure Julia FFT performance (from the SGJ’s EuroSciPy presentation) 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768 65536 131072 262144 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 speed (mflops) intel-mkl-dfti in-place intel-mkl-dfti out-of-place fftw3 out-of-place fftw3 in-place fftw3-no-simd out-of-place fftw3-no-simd in-place dfftpack emayer julia bloodworth cross cwplib esrfft double-precision complex, 1d transforms powers of two
  9. Part 1: Introduction 9 Why compilation is not enough. LLVM

    is a good optimizing compiler but it needs to be given sufficient information to do its job well.
 • Type inference
 • Type stability
 • Delegation O-O paradigm 
 • Multiple dispatch • Staged code generation
  10. Part 1: Introduction 11 Julia Connectivity Julia can call C/Fortran

    libraries
 • ccall((:foo, “lib”), RetType, (xType, yType, …), x, y, …) • If “lib” is omitted default is libjulia • e.g. call((:time, “libc”), Int32, () );
 
 Julia calls other languages via various modules • Python : PyCall • C++ : Cxx • R : Rcall, RIF • Java : JavaCall The most utilised, at present, is PyCall
  11. Part 1: Introduction 12 PyCall Developed by Steven Johnston.
 Can

    call Python functions directly from Julia.
 Used in a variety of Julia modules (including) :
 • PyPlot • SymPy • Pandas • PyLCM
 IJulia and Jupyter was result of Steven Johnson spending a week at Berkeley with Fernando Perez.
  12. Part 1: Introduction 14 pyjulia : Python to Julia interface

    More experimental than the Julia to Python interface Install using a python setup.py routine Needs PyCall to be installed on the Julia side Use from python as • from julia import Julia • jl = Julia() • print “Bessel fn of 2nd kind Y 0 (1.5) is “, jl.bessely0(1.5)
 Limitations: • Not all valid Julia identifiers are valid Python identifiers.
 => Only ASCII names can be used. • Julia implements ‘bang’ functions such as push!()
 => Conventionally these are passed as jl.push_b()
  13. Part 1: Introduction 15 5 reasons to use Julia Access

    to Julia base functions.
 Implement non-vectorised critical code.
 Use specialist Julia modules. • Bayesian, MCMC and machine learning • Signal processing, optimisation methods
 Interface to C-functions and object libraries.
 Run highly parallelised tasks.

  14. Part 1: Introduction 18 Summing up It may be that

    speeding up Python using 
 LLVM-style techniques will not work
 Python has established a niche domain and probably is better to remain there 
 From Julia viewpoint: Python has already proved to be a very valuable companion.
 As Julia matures Python may come to be reliant on her in certain areas, as it is now on C.