Slide 1

Slide 1 text

Boost The Performance with Codon Ikko Eltociear Ashimine GitHub: @eltociear X: @eltociear 2024/4/16 The XtremePython Online Conference

Slide 2

Slide 2 text

1. Introduction 2. About Codon 3. Comparison with other 4. Practical examples and Demo 5. Roadmap 6. Conclusion and Closing Agenda

Slide 3

Slide 3 text

1. Introduction

Slide 4

Slide 4 text

Ikko Ashimine / 一功 安次嶺 I-Tecnology Co., Ltd. Ikko Ashimine has 10 years of engineering experience, having been involved in the development of websites, mobile apps, and consumer games. Currently, as a board member at I-Tecnology Co., Ltd., he is engaged in both management and development. He has an interest in open source and has contributed to various projects. GitHub: @eltociear FaceBook: IkkoEltociearAshimine X: @eltociear YouTube: @eltociear

Slide 5

Slide 5 text

My business card

Slide 6

Slide 6 text

1. Introduction - About Python ・Simple and Readable Syntax ・Diverse Libraries and Frameworks ・Versatility ・Strong Community and Support ・Wide Range of Applications

Slide 7

Slide 7 text

So…

Slide 8

Slide 8 text

But… Python is very slow😭

Slide 9

Slide 9 text

Therefore… Let's look at how to speed up the process😎

Slide 10

Slide 10 text

2. About Codon

Slide 11

Slide 11 text

https://docs.exaloop.io/codon 2. About Codon

Slide 12

Slide 12 text

https://docs.exaloop.io/codon/general/faq 2. About Codon ・Compiles Python code to native machine code without any runtime overhead ・Speedups over vanilla Python are on the order of 10-100x or more ・Supports native multithreading ・Extensible via a plugin infrastructure ・Free of charge

Slide 13

Slide 13 text

2. About Codon - Syntax If you know Python, you already know 99% of Codon.

Slide 14

Slide 14 text

2. About Codon - Differences with Python ・Codon's int is a 64-bit signed integer, whereas Python's (after version 3) can be arbitrarily large. ・Codon currently uses ASCII strings unlike Python's unicode strings. ・Since Codon performs static type checking ahead of time, a few of Python's dynamic features are disallowed. ・For performance reasons, some numeric operations use C semantics rather than Python semantics. ・While most of the commonly used builtin modules have Codon-native implementations, a few are not yet implemented. However these can still be used within Codon via from python import. https://docs.exaloop.io/codon/general/differences

Slide 15

Slide 15 text

3. Comparison with other

Slide 16

Slide 16 text

https://docs.exaloop.io/codon/general/faq 3. Comparison - Cpython Codon tries to follow CPython's syntax, semantics and APIs as closely as possible, aside from a few cases where Codon differs from CPython for performance reasons (one example being Codon's 64-bit int vs. CPython's arbitrary- width int). Performance-wise, speedups over CPython are usually on the order of 10-100x.

Slide 17

Slide 17 text

https://docs.exaloop.io/codon/general/faq 3. Comparison - Numba While Codon does offer a JIT decorator similar to Numba's, Codon is in general an ahead-of-time compiler that compiles end-to-end programs to native code. It also supports compilation of a much broader set of Python constructs and libraries.

Slide 18

Slide 18 text

https://docs.exaloop.io/codon/general/faq 3. Comparison - PyPy PyPy strives to effectively be a drop-in replacement for CPython, whereas Codon differs in a few places in order to eliminate any dynamic runtime or virtual machine, and thereby attain much better performance.

Slide 19

Slide 19 text

https://docs.exaloop.io/codon/general/faq 3. Comparison - Cython Like Cython, Codon has a Python-extension build mode that compiles to Python extension modules, allowing Codon-compiled code to be imported and called from plain Python.

Slide 20

Slide 20 text

4. Practical examples and Demo

Slide 21

Slide 21 text

4. Install Codon Terminal: /bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)" Google Colaboratory: !echo "y" | /bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)" build from source: ->https://docs.exaloop.io/codon/advanced/build

Slide 22

Slide 22 text

4. Use Codon Terminal: # debug mode codon run {YOUR_PYTHON_FILE} # release mode codon run -release {YOUR_PYTHON_FILE} Google Colaboratory: !/root/.codon/bin/codon run prime_number.py

Slide 23

Slide 23 text

Short demonstration (Google Colab) Demo URL: https://colab.research.google.com/drive/1QFndkV9YUp-6VVEBR_bBrVLxt3lAz12s?usp=sharing

Slide 24

Slide 24 text

5. Roadmap

Slide 25

Slide 25 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap Codon's goal is to be as close to CPython as possible while still being fully statically compilable. While Codon already supports much of Python, there is still much to be done to fully realize its potential. Here is a high-level roadmap of the things we want to add, implement or explore.

Slide 26

Slide 26 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 💻 Core features 💻Type system improvements: ・First-class types and compile-time metaclasses ・Full class member deduction ・Implicit union types to support mixed-type collections ・Variadic type arguments (e.g. Foo[Bar, ...]) 💻Parallelism ・async/await support ・multiprocessing support ・Automatic locking in parallel code (e.g. if mutating a data structure shared between threads) ・Race detection

Slide 27

Slide 27 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 💻 Core features 💻Compatibility with Python 3.10+: ・Argument separators (/ and *) ・Constructor object matching in the match statement ・Support accessing various object properties (__dict__, __slots__ etc.) as much as possible in a static context 💻Better error messages ・Warning support ・Explain performance considerations ・Explain that a CPython feature is not supported 💻Modules and incremental compilation ・Cache compilation modules ・Fast generics compilation in debug mode for quick turnarounds

Slide 28

Slide 28 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 💻 Core features 💻Memory management ・Auto-tune GC ・Optional alternative memory management modes like reference counting 💻Optional automatic switching between Codon and CPython (i.e. compile only compatible functions and leave the rest to Python) 💻GPU support ・Target Apple, AMD and Intel GPUs ・GPU-specific compiler optimizations (e.g. for using various Python constructs on the GPU) 💻Interoperability with other languages ・Direct C++ interoperability via Clang ・R interoperability

Slide 29

Slide 29 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 📚 Libraries Currently, missing Python functionality can be easily accessed via a from python import foo statement, which is sufficient in most cases as many libraries are just thin wrappers around a C library and/or not performance-sensitive.

Slide 30

Slide 30 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 📚 Libraries However, in the near future, we would like to support the following modules natively: 📚Python's standard library ・Complete builtins support ・1-to-1 compatibility with existing Python functions and modules ・File modules: os, sys, struct, pathlib and so on ・Pretty much everything else on an as-needed basis 📚Native NumPy, Pandas, etc.: Having Codon-native versions of the most popular 3rd-party libraries would allow them to work with Codon's other features like multithreading and GPU. We're currently prioritizing NumPy and Pandas but aim to later target other popular libraries as well. 📚Unicode support 📚Python's testing infrastructure

Slide 31

Slide 31 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 🛠 Infrastructure & Tools 🛠Windows support 🛠A sane package manager similar to Rust's Cargo 🛠Auto-detection of installed Python libraries 🛠Improved codon.jit library support ・Better error messages ・Better installation flow 🛠Fully static binary support like Go ・Remove libcodonrt (runtime library) dependency if needed ・Remove libcpp dependency

Slide 32

Slide 32 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 🛠 Infrastructure & Tools 🛠Improved Jupyter support ・Auto-completion and code inspection ・Jupyter magic command support 🛠Plugins for Visual Studio Code, Vim, Emacs and so on

Slide 33

Slide 33 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 📝 Documentation 📝Fully document major differences with CPython 📝Document Codon IR API, with guides and tutorials 📝Document all supported modules

Slide 34

Slide 34 text

https://docs.exaloop.io/codon/general/roadmap 5. Roadmap - 👍 Nice to have 👍Implement Codon in Codon

Slide 35

Slide 35 text

6. Conclusion and Closing

Slide 36

Slide 36 text

・Can be compiled with few changes to regular python ・Codon is and will continue to be free of charge ・Codon can run 10-100 times faster in JIT mode ・Codon can output executables from source ・Many more updates are planned for Codon 6. Conclusion

Slide 37

Slide 37 text

Thank you for your attention🐈 mail: