Exploit multicores in Python applications

Presented at Scipy, IIT Bombay, India

Transcript

1. Exploiting the power of multi-core for scientific computing in Python SciPy India SciPy India 2011 IITB, Dec 04-07 Vishal Kanaujia & Chetan Giridhar
2. Setting up the context!! Oh! My telescope generates billion bytes of data a day! • How do I process this humongous data (huge data)? • Can I become quicker (less time)? • I don’t want to compromise with a lower dataset! Attribution-NonCommercial CC BY-NC
3. Understanding the problem • Typically, scientific applications viz. modeling weather patterns, seismographic data, astronomical analysis etc, deal with huge data- set. A telescope is nothing but a huge camera generating billion bytes of data a day! generating billion bytes of data a day! • Processing of this raw data for analysis is a highly CPU-intensive task • Typically applications are not “well” designed Attribution-NonCommercial CC BY-NC
4. A good design strategy • Decomposes application in small independent chunks • Understands trade-off of using parallelism and concurrency • Aware of nuances of the programming • Aware of nuances of the programming language • Design and development of applications utilizing multiple CPU cores in an efficient manner for high performance is THE way to go! Attribution-NonCommercial CC BY-NC
5. Python with Multi-cores • Python thrives with Rapid Application Development • Python applications should exploit multi-core architecture – Let’s evaluate: • threading • multiprocessing • Beyond CPython Attribution-NonCommercial CC BY-NC
6. Getting in to the Problem Space!! Exploiting Multi-cores threading Getting in to the Problem Space!! Attribution-NonCommercial CC BY-NC
7. Multi-threading • Minimized system resource usage • Threads share same address space, memory; hence are lightweight • Context switching is less expensive • Improved performance and concurrency • Python allows writing multi-threaded applications with threading module. Attribution-NonCommercial CC BY-NC
8. Threads in Python! • Python threads are system threads (pthreads) • Each ‘running’ thread requires exclusive access to data structures in Python interpreter • Global interpreter lock (GIL) provides the • Global interpreter lock (GIL) provides the synchronization (bookkeeping) • GIL is necessary, mainly because CPython's memory management is not thread-safe. Attribution-NonCommercial CC BY-NC
9. Two CPU bound threads on single core machine Running Suspended Signals thread2 GIL released waiting for GIL thread1 Suspended Wakeup Running waiting for GIL GIL acquired time check1 thread2 check2 Attribution-NonCommercial CC BY-NC
10. GIL impact • There is considerable time lag with – Communication (Signaling) – Thread wake-up – GIL acquisition • Result – Significant overhead – Thread waits if GIL in unavailable – Threads run sequentially, rather than concurrently Attribution-NonCommercial CC BY-NC
11. Curious case of multicore system thread1, core0 thread2, core1 thread time • Conflicting goals of OS scheduler and Python interpreter • Host OS can schedule threads concurrently on multi-core • GIL battle Attribution-NonCommercial CC BY-NC
12. Python 3.2 • Better GIL arbitration • Ensures that a thread runs only for 5ms • Less context switching and fewer signals • Multicore perspective: GIL battle eliminated! • Multicore perspective: GIL battle eliminated! • More responsive threads (fair scheduling) • Convoy Effect Attribution-NonCommercial CC BY-NC
13. Comparing: Python 2.7 & Python 3.2 60 80 Execution Time – Single Core 150 Execution Time – Dual Core Python v3.2 Execution Time Single Core 55 s Dual Core 65 s Python v2.7 Execution Time Single Core 74 s Dual Core 116 s 0 20 40 60 v2.7 v3.2 Execution Time 0 50 100 v2.7 v3.2 Execution Time 50 52 54 56 58 60 62 64 66 Single Core Dual Core Execution Time – Python v3.2 Execution Time Performance dip still observed in dual cores Attribution-NonCommercial CC BY-NC
14. Multi-threading caveats • Synchronization – Locks – Semaphores • GIL • GIL Attribution-NonCommercial CC BY-NC
15. Getting in to the Problem Space!! Exploiting Multi-cores multiprocessing Getting in to the Problem Space!! Attribution-NonCommercial CC BY-NC
16. Multiprocessing • Multiprocessing - execution of multiple concurrent software processes in a system • Attractive alternative of multi-threading • True parallelism with processes • True parallelism with processes • Python supports it with the multiprocessing module.. Attribution-NonCommercial CC BY-NC
17. More into multiprocessing • multiprocessing — Process-based threading interface • “multiprocessing” module spawns a new Python interpreter instance for a process. • Each process is independent and GIL is irrelevant. • Allows leveraging multiple cores better than threads. • Allows leveraging multiple cores better than threads. • multiprocessing shares API with threading module. – “threading” => “multiprocessing” – “Thread” => “Process” Attribution-NonCommercial CC BY-NC
18. The Multiprocessing APIs • Pipe – shares data across processes • Lock – helps synchronization between processes • Value and Array – Shared memory maps for • Value and Array – Shared memory maps for sharing states between processes • Pool – offloading tasks to pool of worker processes Attribution-NonCommercial CC BY-NC
19. Impact on performance?! • Exploits the power of multi-cores much better, as GIL is avoided. Python v2.7 Single Core Dual Core threading 76 s 114 s 100 120 threading 76 s 114 s multiprocessing 72 s 43 s Cool! 40 % improvement in Execution Time on dual core!! ☺ ☺ ☺ ☺ 0 20 40 60 80 Single Core Dual Core threading multiprocessing Attribution-NonCommercial CC BY-NC
20. Multiprocessing Cons • Processes are resource intensive compared to threads • Need separate address space and resources including FHs, heap including FHs, heap • Context switching is costlier • Inter-process communication is cumbersome to implement (message passing, shared memory) Attribution-NonCommercial CC BY-NC
21. Getting in to the Problem Space!! Exploiting Multi-cores Jython concurrency Getting in to the Problem Space!! concurrency Attribution-NonCommercial CC BY-NC
22. GIL free world: Jython • Memory modeling in Java is thread safe; Jython threads are real Java threads • Uses Java GC library and no reference counting Hence, Jython is free of GIL ☺ • It can fully exploit multiple cores, as per our experiments with Jython2.5 experiments with Jython2.5 – Run with two CPU threads in tandem • Experiment shows performance improvement on a multi- core system Jython2.5 Execution time Single core 44 s Dual core 25 s Attribution-NonCommercial CC BY-NC
23. Jython is not All • Numpy doesn’t work well with Jython • Partial or no support for core modules like os, expat and mmap. • Jython-Python inconsistencies • Jython-Python inconsistencies • List of Jython issues at http://bugs.jython.org Attribution-NonCommercial CC BY-NC
24. Conclusion • Options evaluated.. • Multithreading suffers from GIL limitation • Multiprocessing and Jython…Savior but with caveats…. caveats…. • An intelligent awareness of Python (and its flavors) is helpful in exploiting multi-core opportunity in a better way! • Understand and use ☺ Attribution-NonCommercial CC BY-NC
25. Questions Thank you for your time and attention ☺ Thank you for your time and attention ☺ • Please share your feedback/ comments/ suggestions to us at: • cjgiridhar@gmail.com , http://technobeans.com • vishalkanaujia@gmail.com, http://freethreads.wordpress.com Attribution-NonCommercial CC BY-NC
26. References • Understanding the Python GIL, http://dabeaz.com/talks.html • GlobalInterpreterLock, http://wiki.python.org/moin/GlobalInterpreterLock • Thread State and the Global Interpreter Lock, http://docs.python.org/c-api/init.html#threads • Python v3.2.2 and v2.7.2 documentation, http://docs.python.org/ • Python v3.2.2 and v2.7.2 documentation, http://docs.python.org/ • Concurrency and Python, http://drdobbs.com/open- source/206103078?pgno=3 • Concurrency and Jython http://www.jython.org/jythonbook/en/1.0/Concurrency.html • Jython http://www.jython.org/faq3.html Attribution-NonCommercial CC BY-NC
27. Back up slides Attribution-NonCommercial CC BY-NC
28. Jython | more • Collection of classes in java.util.concurrent useful in concurrency can be easily leveraged • Java uses ConcurrentHashMap for implementation of dict and set for better concurrency concurrency • Plug-in small scripts and code written in jython in Java application Attribution-NonCommercial CC BY-NC