Python and Julia:: The Best of Friends?

Transcript

1. Part 1: Introduction 1 Python and Julia
 The Best of Friends ? Dr Malcolm Sherrington PyData London 6th May 2016 https://github.com/sherrinm/notebooks
2. Part 1: Introduction 2 Warning !!! I have prepared 524 slides for today’s presentation. So if you want to survive this workshop, interrupt me, ask questions and challenge my options.
3. Part 1: Introduction 3 Steven G. Johnston Professor of Applied Mathematics and Physics Massachusetts Institute of Technology, Cambridge, MA.
4. Part 1: Introduction 4 Synopsis of the Workshop Is Julia really fast ? • If so, then why … • Can this be retro-fitted to existing languages?
 How Julia gains from the interaction with Python.
 Can Python do the same (in reverse)?
5. Part 1: Introduction 5 The “Two-Language” Problem We would like to have a scripting language that you can work with interactively: • ease of development, prototyping • data exploration • dynamic typing, visualisation Can use from Python, Matlab/Octave, R, Lisp …
 Critical parts of the coding need to be written in a lower language such as C, Fortran or Java.
 This approach has its drawbacks: • the hard parts are where we need an easier language. • it forces vectorization where it may be inappropriate. • it creates a division between the analyst and programmer.
6. Part 1: Introduction 6 Julia apparently solves this problem by compiling down to native machine code. This uses the LLVM compiler technology and Julia is not alone: • Google V8 • Clang • Swift • LuaJIT Some Python developers are trying to do much the same with: • Numba • Parakeet • Pyston
 Is this going to work?
7. Part 1: Introduction 7 Is Julia really all that fast ? Seemingly so with benchmarks …
8. Part 1: Introduction 8 … but how about for Enterprise computing Pure Julia FFT performance (from the SGJ’s EuroSciPy presentation) 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768 65536 131072 262144 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 speed (mflops) intel-mkl-dfti in-place intel-mkl-dfti out-of-place fftw3 out-of-place fftw3 in-place fftw3-no-simd out-of-place fftw3-no-simd in-place dfftpack emayer julia bloodworth cross cwplib esrfft double-precision complex, 1d transforms powers of two
9. Part 1: Introduction 9 Why compilation is not enough. LLVM is a good optimizing compiler but it needs to be given sufficient information to do its job well.
 • Type inference
 • Type stability
 • Delegation O-O paradigm 
 • Multiple dispatch • Staged code generation
10. Part 1: Introduction 10
11. Part 1: Introduction 11 Julia Connectivity Julia can call C/Fortran libraries
 • ccall((:foo, “lib”), RetType, (xType, yType, …), x, y, …) • If “lib” is omitted default is libjulia • e.g. call((:time, “libc”), Int32, () );
 
 Julia calls other languages via various modules • Python : PyCall • C++ : Cxx • R : Rcall, RIF • Java : JavaCall The most utilised, at present, is PyCall
12. Part 1: Introduction 12 PyCall Developed by Steven Johnston.
 Can call Python functions directly from Julia.
 Used in a variety of Julia modules (including) :
 • PyPlot • SymPy • Pandas • PyLCM
 IJulia and Jupyter was result of Steven Johnson spending a week at Berkeley with Fernando Perez.
13. Part 1: Introduction 13
14. Part 1: Introduction 14 pyjulia : Python to Julia interface More experimental than the Julia to Python interface Install using a python setup.py routine Needs PyCall to be installed on the Julia side Use from python as • from julia import Julia • jl = Julia() • print “Bessel fn of 2nd kind Y 0 (1.5) is “, jl.bessely0(1.5)
 Limitations: • Not all valid Julia identifiers are valid Python identifiers.
 => Only ASCII names can be used. • Julia implements ‘bang’ functions such as push!()
 => Conventionally these are passed as jl.push_b()
15. Part 1: Introduction 15 5 reasons to use Julia Access to Julia base functions.
 Implement non-vectorised critical code.
 Use specialist Julia modules. • Bayesian, MCMC and machine learning • Signal processing, optimisation methods
 Interface to C-functions and object libraries.
 Run highly parallelised tasks.

16. Part 1: Introduction 16
17. Part 1: Introduction 17
18. Part 1: Introduction 18 Summing up It may be that speeding up Python using 
 LLVM-style techniques will not work
 Python has established a niche domain and probably is better to remain there 
 From Julia viewpoint: Python has already proved to be a very valuable companion.
 As Julia matures Python may come to be reliant on her in certain areas, as it is now on C.
19. Part 1: Introduction 19 References http://julialang.org http://docs.julialang.org/en/latest/manual http://juliastats.github.io https://github.com/JuliaParallel http://llvm.org/ProjectsWithLLVM https://github.com/josephmisiti/awesome-machine-learning http://quant-econ.net https://github.com/FRBNY-DSGE/DSGE.jl
20. Part 1: Introduction 20
21. Part 1: Introduction 21