CPython JIT_ The mystery as all we know

Transcript

1. CPython JIT: The mystery as all we know Learn | Connect | Code 11 May 2024 #Python #PyConTH
2. #Python #PyConTH Just-in-Time in CPython 3.13
3. #Python #PyConTH Just-in-Time in CPython 3.13
4. #Python #PyConTH Why is it matter? - You can compile CPython and enable it with --enable-experimental-jit - Default version will (potentially) be there in next minor release (Python 3.14) - It’s performance improve, YES. But.. how does we trade off?
5. Python Implementation
6. #Python #PyConTH Python Implementation language Implementation Cython CPython PyPy Numba This is default
7. #Python #PyConTH Python Implementation Source Program (.py) Implementation Target Intrepreter Virtual Machine Optimizer Machine Code
8. #Python #PyConTH Python (CPython) Implementation Source Program (.py) CPython Bytecode (.pyc) Machine Code Python Virtual Machine Interpreter Memory Manager …. compile Execute Target Machine
9. CPython bytecode Demo 1
10. #Python #PyConTH Python (CPython) Implementation stack LOAD_CONST ('Hello World') STORE_FAST (text) LOAD_FAST (text) RETURN_VALUE def greeting(): text = "Hello World" return text python (.py) bytecode (.pyc) local variable
11. #Python #PyConTH Python (CPython) Implementation stack LOAD_CONST ('Hello World') STORE_FAST (text) LOAD_FAST (text) RETURN_VALUE def greeting(): text = "Hello World" return text python (.py) bytecode (.pyc) local variable ‘Hello World’
12. #Python #PyConTH Python (CPython) Implementation stack LOAD_CONST ('Hello World') STORE_FAST (text) LOAD_FAST (text) RETURN_VALUE def greeting(): text = "Hello World" return text python (.py) bytecode (.pyc) local variable text = ‘Hello World’
13. #Python #PyConTH Python (CPython) Implementation stack LOAD_CONST ('Hello World') STORE_FAST (text) LOAD_FAST (text) RETURN_VALUE def greeting(): text = "Hello World" return text python (.py) bytecode (.pyc) local variable text = ‘Hello World’
14. #Python #PyConTH Python (CPython) Implementation stack LOAD_CONST ('Hello World') STORE_FAST (text) LOAD_FAST (text) RETURN_VALUE def greeting(): text = "Hello World" return text python (.py) bytecode (.pyc) local variable text = ‘Hello World’ return
15. What if metrix?
16. #Python #PyConTH Python (CPython) Implementation BUILD_LIST 0 LOAD_CONST 1 ((1, 2, 3)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 2 ((3, 1, 4)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 3 ((3, 5, 8)) LIST_EXTEND 1 BUILD_LIST 3 STORE_FAST 0 (matrixA) LOAD_FAST 0 (matrixA) RETURN_VALUE def metrix(): matrixA = [[1, 2, 3], [3, 1, 4], [3, 5, 8]] return matrixA python (.py) bytecode (.pyc)
17. #Python #PyConTH Python (CPython) Implementation BUILD_LIST 0 LOAD_CONST 1 ((1, 2, 3)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 2 ((3, 1, 4)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 3 ((3, 5, 8)) LIST_EXTEND 1 BUILD_LIST 3 STORE_FAST 0 (matrixA) LOAD_FAST 0 (matrixA) RETURN_VALUE def metrix(): matrixA = [[1, 2, 3], [3, 1, 4], [3, 5, 8]] return matrixA python (.py) bytecode (.pyc) Which scenario would we face this?
18. #Python #PyConTH Python (CPython) Problem BUILD_LIST 0 LOAD_CONST 1 ((1, 2, 3)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 2 ((3, 1, 4)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 3 ((3, 5, 8)) LIST_EXTEND 1 BUILD_LIST 3 STORE_FAST 0 (matrixA) LOAD_FAST 0 (matrixA) RETURN_VALUE bytecode (.pyc) switch (opcode) { PREDICTED(LOAD_CONST); TARGET(LOAD_CONST) { PyObject *value = GETITEM(consts, oparg); Py_INCREF(value); PUSH(value); FAST_DISPATCH(); } PREDICTED(STORE_FAST); TARGET(STORE_FAST) { PyObject *value = POP(); SETLOCAL(oparg, value); FAST_DISPATCH(); } } how CPython interpret bytecode (.pyc)
19. #Python #PyConTH Python (CPython) Problem BUILD_LIST 0 LOAD_CONST 1 ((1, 2, 3)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 2 ((3, 1, 4)) LIST_EXTEND 1 BUILD_LIST 0 LOAD_CONST 3 ((3, 5, 8)) LIST_EXTEND 1 BUILD_LIST 3 STORE_FAST 0 (matrixA) LOAD_FAST 0 (matrixA) RETURN_VALUE bytecode (.pyc) - How can we improve performance?
20. Let get back to our work. Just-in-Time (JIT) compilation
21. #Python #PyConTH What is Just-in-Time (JIT)? Ahead-of-Time Compilation #include <stdio.h> int main() { // Declare variables int num1 = 10; int num2 = 5; int sum // Perform math operations sum = num1 + num2; // Print the results printf("Sum: %d\n", sum); return 0; } compiled area target language execute compile
22. #Python #PyConTH What is Just-in-Time (JIT)? Ahead-of-Time (AOT) Compilation #include <stdio.h> int main() { // Declare variables int num1 = 10; int num2 = 5; int sum // Perform math operations sum = num1 + num2; // Print the results printf("Sum: %d\n", sum); return 0; } target language compiled area Just-in-Time (JIT) Compilation # Declare variables num1 = 10 num2 = 5 # Perform math operations sum = num1 + num2 # Print the results print("Sum:", sum) execute interpreted & executed 1st 2nd 3rd compile
23. #Python #PyConTH A-ha! So what?
24. #Python #PyConTH Demo 2: Let say… LOAD_CONST ('Public Company') LOAD_CONST ('Private Company') LOAD_CONST ('Expensive Volumn') LOAD_CONST ('Upper Cost') LOAD_CONST ('VIP') LOAD_CONST ('VVIP') LOAD_CONST (('A002', 'A003', 'T001', 'T002', 'U008', 'U009')) BUILD_CONST_KEY_MAP 6 STORE_FAST (metaTable) LOAD_FAST (metaTable) RETURN_VALUE def meta_table(): metaTable = { "A002": "Public Company", "A003": "Private Company", "T001": "Expensive Volumn", "T002": "Upper Cost", "U008": "VIP", "U009": "VVIP", } return metaTable python (.py) bytecode (.pyc)
25. #Python #PyConTH LOAD_CONST ('Public Company') LOAD_CONST ('Private Company') LOAD_CONST ('Expensive Volumn') LOAD_CONST ('Upper Cost') LOAD_CONST ('VIP') LOAD_CONST ('VVIP') LOAD_CONST (('A002', 'A003', 'T001', 'T002', 'U008', 'U009')) BUILD_CONST_KEY_MAP 6 STORE_FAST (metaTable) LOAD_FAST (metaTable) RETURN_VALUE def meta_table(): metaTable = { "A002": "Public Company", "A003": "Private Company", "T001": "Expensive Volumn", "T002": "Upper Cost", "U008": "VIP", "U009": "VVIP", } return metaTable python (.py) bytecode (.pyc) Let cache its output in compiler level Let say…
26. #Python #PyConTH Can we cache function?
27. #Python #PyConTH def meta_table(): metaTable = { "A002": "Public Company", "A003": "Private Company", "T001": "Expensive Volumn", "T002": "Upper Cost", "U008": "VIP", "U009": "VVIP", } return metaTable python (.py) Yes, if it is
28. #Python #PyConTH def simple_encrypt(message, key): encrypted_message = bytearray() key_length = len(key) for i, char in enumerate(message): encrypted_char = char ^ key[i % key_length] encrypted_message.append(encrypted_char) return encrypted_message python (.py) How about this?
29. #Python #PyConTH CPython JIT - Copy-and-Patch
30. #Python #PyConTH CPython JIT - Copy-and-Patch def simple_encrypt(message, key): encrypted_message = bytearray() key_length = len(key) for i, char in enumerate(message): encrypted_char = char ^ key[i % key_length] encrypted_message.append(encrypted_char) return encrypted_message 1) Create template 2) Fill the hole everything is done in bytecode and machine code level
31. #Python #PyConTH CPython JIT - Copy-and-Patch PREDICTED(LOAD_CONST); TARGET(LOAD_CONST) { PyObject *value = GETITEM(consts, oparg); Py_INCREF(value); PUSH(value); FAST_DISPATCH(); } how CPython interpret bytecode (.pyc) 0000000000000000 <__JIT_ENTRY>: pushq %rbp movq %rsp, %rbp movq (%rdi), %rax movq 0x28(%rax), %rax movabsq $0x0, %rcx 000000000000000d: X86_64_RELOC_UNSIGNED __JIT_OPARG movzwl %cx, %ecx movq 0x28(%rax,%rcx,8), %rax movl 0xc(%rax), %ecx incl %ecx je 0x3d <__JIT_ENTRY+0x3d> movq %gs:0x0, %r8 cmpq (%rax), %r8 jne 0x37 <__JIT_ENTRY+0x37> movl %ecx, 0xc(%rax) jmp 0x3d <__JIT_ENTRY+0x3d> lock addq $0x4, 0x10(%rax) movq %rax, (%rsi) addq $0x8, %rsi movabsq $0x0, %rax 0000000000000046: X86_64_RELOC_UNSIGNED __JIT_CONTINUE popq %rbp jmpq *%rax
32. #Python #PyConTH What are we going to see in future Python 3.14 (onward)? - It’ll be fast yes but not now. (7% improvement for JIT enable, Jan 2024) - Just–in-Time is going to be default in CPython. - Not only while function, but also other computational-heavy methods. - Python likely consumes more memory to trade off performance. - Community will adopt it such as Numpy, Tensorflow, PyTorch. - You should experiment with CPython JIT if your application are; - Long-run application process. - Numerical calculation.
33. Thank you