ABI 是什麼？跟 API 不一樣嗎？

儘管 Application Binary Interface (ABI) 通常就比較少人提起，但它的重要性不亞於更常聽到的 Application Programming Interface (API)；例如我們到現在還能跑十年前編譯好的程式這件事，就是因爲有 ABI 才有辦法達成（精確地來說是 ABI 相容性的概念）。

這次議程中我們會介紹到底什麼是 ABI、它跟 API 有什麼不同、討論 ABI 相容性及列出 ABI 不相容時會發生什麼事、還有舉出一些 ABI 的實例（Linux Kernel ABI、Python's stable ABI、Foreign Function Interface）。

Transcript

1. ABI 是什麼？ 跟 API 不一樣嗎？ What is an ABI, and how it differs from API? Shung-Hsi Yu COSCUP 2024 1
2. 2 爲什麼前人 10 年留下的程式還能跑在新系統上? Why does (compiled) program from 10 years ago still runs on the latest system? （編譯過）
3. 3 爲什麼把系統升級到下個大版我的程式就壞了？ Why does upgrading my system to the next major version broke my program? （編譯過）
4. 4 ABI
5. Shung-Hsi Yu @shunghsiyu 任職於 SUSE Kernel Engineer 維護 SLES & openSUSE 作業系統核心裡 BPF 的部分 maintains BPF in SLES & openSUSE kernel 5 自介 About me
6. 6 在講 ABI 之前 Before we talk about ABI
7. 7 API
8. 8 Application Programming Interface 應用程式介面
9. 9 Web API
10. 10 List repository tree - Repositories API | GitLab
11. 11 API
12. 12 Ranges — Built-in Types — Python 3.12.4 documentation
13. 13 Formatted Output Functions (The GNU C Library)
14. 14 ABI
15. 15 Application Binary Interface 應用二進位介面
16. 16 Application Binary Interface Application Programming Interface
17. 17 Binary Programming Application Interface
18. 18 Binary Programming Application Interface
19. 19 介面 Interface
20. 20 規格 Specification
21. 21 保證 Guarantee
22. 22 保證 Guarantee GitLab 保證 …/tree 會列出 repo 的所有檔案名稱
23. 23 保證 Guarantee Python 保證 range(2) 從 0 開始 GitLab 保證 …/tree 會列出 repo 的所有檔案名稱
24. 24 保證 Guarantee Python 保證 range(2) 從 0 開始 glibc 保證 printf() 會寫到 stdout GitLab 保證 …/tree 會列出 repo 的所有檔案名稱
25. 25 Binary Programming Application Interface
26. 26 Binary Programming Application Interface
27. 27 Binary Programming
28. 28 Binary Programming 程式碼 source code 機器碼 machine code
29. 29 Binary Programming 機器碼 machine code long y_square(struct point *p) { return p->y * p->y; }
30. 30 Binary Programming long y_square(struct point *p) { return p->y * p->y; } 00000000100001010011010100000011 00000010101001010000010100110011 00000000000000001000000001100111
31. 31 Binary Programming long y_square(struct point *p) { return p->y * p->y; } y_square: ld a0, 8(a0) mul a0, a0, a0 ret
32. 32 Binary Programming long y_square(struct point *p) { return p->y * p->y; } y_square: ld a0, 8(a0) mul a0, a0, a0 ret
33. 33 Binary Programming long y_square(struct point *p) { return p->y * p->y; } y_square: ld a0, 8(a0) mul a0, a0, a0 ret compiler
34. 34 Binary Programming
35. 35 Binary Programming 底層 low(er) level 上層 high(er) level
36. 36 Binary Programming 沒有明確宣告 implicit 明確宣告的 explicit
37. 37 舉例來說 For an example
38. 38 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
39. 39 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
40. 40 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } 函式庫 shared library
41. 41 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
42. 42 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } 函式庫 header library header
43. 43 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
44. 44 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } 應用程式 application
45. 45 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ...
46. 46 By Mc681 - Own work, CC BY-SA 4.0
47. 47 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ...
48. 48 // geo-v1.h struct point { long x; long y; }; long y_square(struct point *); // geo-v1.c -> geo.so #include <geo-v1.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
49. 49 Binary Programming long y_square(struct point *p) { return p->y * p->y; } y_square: ld a0, 8(a0) mul a0, a0, a0 ret
50. 50 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } long y_square(struct point *p) { return p->y * p->y; } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ... y_square: ld a0, 8(a0) mul a0, a0, a0 ret
51. 51 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } long y_square(struct point *p) { return p->y * p->y; } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ... y_square: ld a0, 8(a0) mul a0, a0, a0 ret
52. 52 函式庫被更新 shared library updated
53. 53 // geo-v2.h struct point { long x; long y; long z; // this is new }; long y_square(struct point *); // geo-v2.c -> geo.so #include <geo-v2.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
54. 54 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } long y_square(struct point *p) { return p->y * p->y; } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ... y_square: ld a0, 8(a0) mul a0, a0, a0 ret
55. 55 // geo-v3.h struct point { long z; // so rebellious long x; long y; }; long y_square(struct point *); // geo-v3.c -> geo.so #include <geo-v3.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
56. 56 Binary Programming int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); } long y_square(struct point *p) { return p->y * p->y; } main: li a1, 1 sd a1, 0(a0) li a1, 2 sd a1, 8(a0) ... call y_square ... y_square: ld a0, 12(a0) mul a0, a0, a0 ret
57. 57 // geo-v1.h struct point { long x; // at offset 0 long y; // at offset 8 }; long y_square(struct point *); // geo-v2.c -> geo.so #include <geo-v2.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
58. 58 // geo-v2.h struct point { long x; // at offset 0 long y; // at offset 8 long z; // at offset 16 }; long y_square(struct point *); // geo-v2.c -> geo.so #include <geo-v2.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
59. 59 // geo-v3.h struct point { long z; // at offset 0 long x; // at offset 8 long y; // at offset 16 }; long y_square(struct point *); // geo-v2.c -> geo.so #include <geo-v2.h> long y_square(struct point *p) { return p->y * p->y; } // main.c -> main #include <geo-v1.h> int main(void) { struct point p = { .x = 1, .y = 2, }; /* you get y2 == 4 */ long y2 = y_square(&p); }
60. 60 隱藏實作的細節 Hide the implementation
61. 61 // geo.h struct point; // opaque pointer struct point *init(); void set_x(struct point *, long); void set_y(struct point *, long); // main.c #include <geo.h> int main(void) { struct point *p = init(); set_x(p, 1); set_y(p, 2); long y2 = y_square(p); }
62. 62 改變函數的參數 Changing function signature
63. 63 long y_square(struct point *p); long y_square(struct point *p1, struct point *p2);
64. 64 名稱修飾 Name Mangling
65. 65 Symbol Versioning
66. 66 $ rpm -q --provides glibc libc.so.6(GLIBC_2.10)(64bit) libc.so.6(GLIBC_2.11)(64bit) ... libc.so.6(GLIBC_2.39)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.5)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.7)(64bit) libc.so.6(GLIBC_2.8)(64bit) libc.so.6(GLIBC_2.9)(64bit)
67. 67 其他特性 Other characteristics
68. 68 ABI 層層相依 ABI has a chain of dependency
69. 69 Machine ABI Kernel ABI Library ABI Application ABI
70. 70 Machine ABI Kernel ABI Library ABI Application ABI
71. 71 Machine ABI Kernel ABI Library ABI Application ABI glibc, geo.so Linux Kernel x86-64 main.c
72. 72 Machine ABI Kernel ABI Library ABI Application ABI
73. 73 Arch i386 x86-64 aarch64 riscv
74. 74 Arch i386 x86-64 aarch64 riscv bpf wasm
75. 75 呼叫慣例 Call convention
76. 76 如何傳遞引數 How to pass function arguments
77. 77 Arch/ABI arg1 arg2 arg3 arg4 arg5 arg6 i386 ebx ecx edx esi edi ebp x86-64 rdi rsi rdx r10 r8 r9 aarch64 x0 x1 x2 x3 x4 x5 riscv a0 a1 a2 a3 a4 a5
78. 78 Arch/ABI arg1 arg2 arg3 arg4 arg5 arg6 i386 ebx ecx edx esi edi ebp x86-64 rdi rsi rdx r10 r8 r9 aarch64 x0 x1 x2 x3 x4 x5 riscv a0 a1 a2 a3 a4 a5
79. 79 Arch/ABI arg1 arg2 arg3 arg4 i386 ebx ecx edx esi x86-64 rdi rsi rdx r10 aarch64 x0 x1 x2 x3 riscv a0 a1 a2 a3
80. 80 Machine ABI Kernel ABI Library ABI Application ABI 改變 changed
81. 81 Machine ABI Kernel ABI Library ABI Application ABI 改變 changed ?
82. 82 Machine ABI Kernel ABI Library ABI Application ABI 改變 changed
83. 83 底層 ABI 會影響上層 ABI affects
84. 84 凡事都有例外... There’s always exceptions...
85. 85 Cosmopolitan
86. 86 Cosmopolitan ✓Linux ✓macOS ✓Windows
87. 87 底層 ABI 會影響上層 ABI affects
88. 88 誰來保證 Who provides the guarantee
89. 89 Machine ABI Kernel ABI Library ABI Application ABI
90. 90 硬體廠商 Hardware Vendors
91. 91 Machine ABI Kernel ABI Library ABI Application ABI Linux Kernel
92. 92 Linux Kernel 開發者們 Linux Kernel Developers
93. 93 Machine ABI Kernel ABI Library ABI Application ABI
94. 94 Library 開發者們 Library Developers
95. 95 Machine ABI Kernel ABI Library ABI Application ABI
96. 96 Linux 發行版提供者 Linux Distribution Providers
97. 97 Linux 發行版提供者 Linux Distribution Providers Hi!
98. 98 Machine ABI Kernel ABI Library ABI Application ABI
99. 99 CVE
100. 100 Machine ABI Kernel ABI Library ABI Application ABI Linux Kernel Module
101. 101 Kernel ABI (kABI)
103. 103 維持相容性 Maintaining Compatibility
104. 104 向下相容 Backward Compatibility
105. 105 舊程式跑在新系統也 ok Old program should ran fine on the new system
106. 106 Linux Kernel 提供的系統呼叫 System call provided by the Linux Kernel
107. 107 int openat(int dirfd, const char *pathname, int flags, ... /* mode_t mode */ );
108. 108 需要改參數怎麼辦？ What if you need to change parameter?
109. 109 int openat(int dirfd, const char *pathname, int flags, ... /* mode_t mode */ ); int openat2(int dirfd, const char *pathname, const struct open_how *how, size_t size);
110. 110 int openat(int dirfd, const char *pathname, int flags, ... /* mode_t mode */ ); int openat2(int dirfd, const char *pathname, const struct open_how *how, size_t size);
111. 111 向上相容 Forward Compatibility
112. 112 新程式跑在舊系統也 ok New program should ran fine on the old system
113. 113 int openat(int dirfd, const char *pathname, int flags);
114. 114 上下相容 Backward & Forward Compatibility
115. 115 如何讓擴充支援多個 Python 版本 Make extension support multiple Python versions
116. 116 Python 的受限 API Python’s Limited API
117. 117 #define Py_LIMITED_API
118. 118 實際案例 Real-life cases
119. 119 Y2038 問題 Y2038 problem
121. 121 typedef time_t int32_t;
122. 122 typedef time_t int32_t; typedef time_t int64_t;
123. 123 ABI 無法相容 ABI Breakage
124. 124 typedef time_t int32_t; typedef time_t int64_t; 需要重新編譯 Need to recompile
125. 125 (好的)實際案例 Real-life cases (the good ones)
126. 126 模擬器 Emulators
127. 127 Wine
129. 129 QEMU
130. 130 Rosetta 2 (macOS)
131. 131 外部函數呼叫 (FFI) Foreign Function Interface
132. 132 #include <stdio.h> int main(void) { /* Can you use printf in Python? */ printf("Hello %s!\n", "world"); }
133. 133 #!/usr/bin/python3 import ctypes # We can load _any_ shared library # Let's use the ubiquitous libc here libc = ctypes.CDLL('libc.so.6') # This is the _exact_ same printf you # get in C libc.printf(b'Hello %s!\n', b'world')
134. 134 Ruby → C Java → C Go → C
135. 135 Ruby → C Java → C Go → C Python → Swift Julia → Rust
136. 136 結論 Conclusion
137. Thank you for listening! 137 感謝聆聽！
138. 138 eBPF BoF Day 2, 12:20-13:20 @TR613 Linux Kernel Development Day 2, 13:30-14:10 @TR411 Bird of a Feathers @COSCUP
139. 139 Binary Programming human (?) readable machine readable
140. compiler 140 Binary Programming machine code source code
141. 141 Binary Programming un-compiled compiled 簡易區分 rough differentiation
144. XX% 144 Big number here