The (not so) hidden cost of sharing code between iOS and Android

Until very recently, Dropbox had a technical strategy on mobile of sharing code between iOS and Android via C++. The idea behind this strategy was simple—write the code once in C++ instead of twice in Java and Objective C (Swift and Kotlin weren't around yet when we adopted this strategy).

We have now backed off from this strategy in favor of using each platforms’ native languages. This decision was due to the (not so) hidden cost associated with code sharing.

Come with me on a journey of how dropbox set about to share code via C++. What went well, what went wrong, what are the main lessons we learned and what this means for how we write code going forward.

Based on https://blogs.dropbox.com/tech/2019/08/the-not-so-hidden-cost-of-sharing-code-between-ios-and-android.

Transcript

1. The (not so) hidden cost of sharing code between iOS and Android
2. Eyal Guthmann Mobile Engineer
3. This talk is about How we used C++
4. This talk is about What we learned that made us stop
5. This talk is about What’s beyond C++ sharing for us
6. This talk is not about Hating on C++
7. This talk is not about A statement against code sharing
8. (War) Story Time
9. ANRs from C++
10. "main" tid=1 Blocked at MyManagerJNI.addListener (MyManagerJNI.java:384) - waiting to lock <0x01183147> (a com.dropbox.MyManagerJNI) held by thread 66 at MyLoader.onStartLoading (MyLoader.java:72) at android.support.v4.content.Loader.startLoading (Loader.java:283) at android.support.v4.app.LoaderManagerImpl$LoaderInfo.onActive (LoaderManagerImpl.java:76) ANRs from C++ :-(
11. ANRs from C++ Next step - look at thread 66 Thread 66 where are you? (Not in the thread dump) "main" tid=1 Blocked at MyManagerJNI.addListener (MyManagerJNI.java:384) - waiting to lock <0x01183147> (a com.dropbox.MyManagerJNI) held by thread 66 at MyLoader.onStartLoading (MyLoader.java:72) at android.support.v4.content.Loader.startLoading (Loader.java:283) at android.support.v4.app.LoaderManagerImpl$LoaderInfo.onActive (LoaderManagerImpl.java:76)
12. One of the most time consuming bugs we had to debug ANRs from C++
13. Took 3 rounds of debugging to fully resolve ANRs from C++
14. Had to pull in 3 senior tech leads at various stages, including one of the original architects ANRs from C++
15. So what was it?
16. We still don’t really know So what was it?
17. Fell back to eliminating deadlocks from C++ by staring at the code really hard. So what was it?
18. Best guess: A previous cleanup was blocking the app when it was foregrounded So what was it?
19. The “Solution”
20. Move all possible call site that can hang to background The “Solution”
21. That “fixed” the crash The “Solution”
22. But probably introduced a bug where data won’t refresh The “Solution”
23. Back to the trenches
24. Useless stack traces
25. Useless stack traces *** Terminating app due to uncaught exception 'DBException', reason: 'DBException: std::length_error: vector {-[DBThreadFunc run]}’ CoreFoundation __exceptionPreprocess (__offset:228) libobjc.A.dylib objc_exception_throw (__offset:56) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox -[DBThreadFunc run] (__offset:84) Foundation __NSThread__start__ (__offset:1040) libsystem_pthread.dylib _pthread_body (__offset:272) libsystem_pthread.dylib _pthread_body (__offset:0) libsystem_pthread.dylib thread_start (__offset:4) Warning: iOS stack trace ahead!
26. Useless iOS stack traces *** Terminating app due to uncaught exception 'DBException', reason: 'DBException: std::length_error: vector {-[DBThreadFunc run]}’ CoreFoundation __exceptionPreprocess (__offset:228) libobjc.A.dylib objc_exception_throw (__offset:56) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox -[DBThreadFunc run] (__offset:84) Foundation __NSThread__start__ (__offset:1040) libsystem_pthread.dylib _pthread_body (__offset:272) libsystem_pthread.dylib _pthread_body (__offset:0) libsystem_pthread.dylib thread_start (__offset:4) Useless system frames
27. *** Terminating app due to uncaught exception 'DBException', reason: 'DBException: std::length_error: vector {-[DBThreadFunc run]}’ CoreFoundation __exceptionPreprocess (__offset:228) libobjc.A.dylib objc_exception_throw (__offset:56) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox -[DBThreadFunc run] (__offset:84) Foundation __NSThread__start__ (__offset:1040) libsystem_pthread.dylib _pthread_body (__offset:272) libsystem_pthread.dylib _pthread_body (__offset:0) libsystem_pthread.dylib thread_start (__offset:4) Useless Framework frames Useless iOS stack traces
28. *** Terminating app due to uncaught exception 'DBException', reason: 'DBException: std::length_error: vector {-[DBThreadFunc run]}’ CoreFoundation __exceptionPreprocess (__offset:228) libobjc.A.dylib objc_exception_throw (__offset:56) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox -[DBThreadFunc run] (__offset:84) Foundation __NSThread__start__ (__offset:1040) libsystem_pthread.dylib _pthread_body (__offset:272) libsystem_pthread.dylib _pthread_body (__offset:0) libsystem_pthread.dylib thread_start (__offset:4) More useless system frames Useless iOS stack traces
29. *** Terminating app due to uncaught exception 'DBException', reason: 'DBException: std::length_error: vector {-[DBThreadFunc run]}’ CoreFoundation __exceptionPreprocess (__offset:228) libobjc.A.dylib objc_exception_throw (__offset:56) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox djinni::throwNSExceptionFromCurrent(char const*) (__offset:1124) Dropbox -[DBThreadFunc run] (__offset:84) Foundation __NSThread__start__ (__offset:1040) libsystem_pthread.dylib _pthread_body (__offset:272) libsystem_pthread.dylib _pthread_body (__offset:0) libsystem_pthread.dylib thread_start (__offset:4) Only useful part Useless iOS stack traces
30. Why is this happening?!?
31. We have to learn something about C++ to figure this out Caught C++ exceptions do not have a stack trace (?!?!)
32. We have to learn something about C++ to figure this out Once caught, You cannot get a stack trace
33. We have to learn something about C++ to figure this out To get a stack trace you must let it crash the app
34. How to solve this one?
35. 1) Learn way too much about the working on C++ exceptions How to solve this one?
36. 2) Roll out a change to make system exception crash the app in C++ How to solve this one?
37. 3) Get the real stack trace and easily fix the actual bug How to solve this one?
38. Now, out of trenches back to control room
39. History
40. History 2013 - C++ we choose you!
41. 2013 - C++ we choose you! Predates React Native, Flutter etc.
42. 2013 - C++ we choose you! Small team that needed to be nimble
43. 2013 - C++ we choose you! In house C++ expertise
44. 2013 - 2016: Active development
45. 2017-2018: Decreased C++ development
46. 2017-2018: Decreased C++ development Hard to keep up with ecosystem
47. 2017-2018: Decreased C++ development Core experts moved over time
48. 2017-2018: Decreased C++ development New projects stayed native
49. The technical strategy
50. Target Sharing core functionality between platforms The technical strategy
51. Create tooling to allow the C++ code and native languages to communicate The technical strategy
52. Build frameworks and libraries that the core functionality depends on (E.g: Feature flags) The technical strategy
53. Stuff we built in C++
54. Frameworks and Libraries A Framework calls into your code A library is something your code calls into
55. Frameworks Example: Lifecycle management
56. Frameworks Example: Async task execution
57. Libraries Example: C++11 Json parsing
58. Libraries Example: Non-nullable pointers for C++
59. Support tools Examples: Djinni: JNI and ObjC++ 㲗 C++ Binding Generator
60. Support tools Examples: Cross language build tools
61. Products libraries Examples: Camera roll syncing
62. Products libraries Examples: Notifications Client
63. Stats 85k LoC in C++ 35K LoC in open source project 10K for build tools
64. Results Small team successfully shipped quickly (initially)
65. Results Created an open source community for mobile C++
66. Changing direction
67. Why? High long term cost Low long term value
68. Why? Bugs coming from C++ were the most time consuming
69. Why? Only 1 engineer committed code in C++ in over 6 months
70. Lets break it down The Four types of overhead:
71. 1) Custom frameworks, libraries and tools
72. We covered this already
73. We covered this already Frameworks Libraries Support tools
74. 2) Custom development environment Some examples:
75. IDE integration Not as good as for native languages
76. Debugging Hard/Impossible to debug between languages (Especially Java 㲗 C++)
77. 3) Differences between the platforms
78. Example Background capabilities
79. Example Supported image formats
80. Example Apple Cloud vs. Google Drive/Images
81. Apple and Google are diverging Different incentives for evolution No incentive to converge We have no control over their choices
82. 4) Training, hiring, and retaining developers
83. Mobile C++ Developers
84. Challenges Senior C++ engineers are rarely mobile experts
85. Challenges More junior Mobile engineers don’t see C++ as an interesting skill
86. Beyond Code Sharing Sharing Architecture
87. Sharing Architecture Design once (together), Write twice
88. Sharing Architecture Stagger implementation (when possible)
89. Sharing Architecture Use similar libraries (RxJava + RxSwift was a big win)
90. Migration Process Goal - remove all* C++ code from mobile
91. Migration Process A/B test each migration
92. Migration Process Avoid changing C++ when possible
93. When to diverge from the ecosystem?
94. Situation 1 Large company - Long term bet (Your Googles, Facebooks, Amazons etc.)
95. Situation 2 It provides a core business advantage
96. Situation 2 And even then - keep you the surface area of you custom solution minimal
97. Build your own: Example
98. Rust at Dropbox A high-performance multi-platform programming language (Pioneered by Mozilla)
99. Rust at Dropbox Our data sync client library is written in Rust
100. Rust at Dropbox Runs on Windows/Mac/Linux
101. Rust at Dropbox Has a dedicated team of experts
102. Rust at Dropbox Provides a business advantage
103. Thank You! @EyalGuthmann (Come work with me)