Slide 1

Slide 1 text

Chasing TransactionTooLargeException in The Wild 2019-03-08 shibuya.apk Hiroshi Kurokawa

Slide 2

Slide 2 text

What’s TransactionTooLarge- Exception?

Slide 3

Slide 3 text

java.lang.RuntimeException: android.os.TransactionTooLargeException: data parcel size 592196 bytes at android.app.ActivityThread$StopInfo.run(ActivityThread.java:3752) at android.os.Handler.handleCallback(Handler.java:751) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:154) at android.app.ActivityThread.main(ActivityThread.java:6077) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755) Caused by: android.os.TransactionTooLargeException: data parcel size 592196 bytes at android.os.BinderProxy.transactNative(Native Method) at android.os.BinderProxy.transact(Binder.java:615) at android.app.ActivityManagerProxy.activityStopped(ActivityManagerNative.java:3606) at android.app.ActivityThread$StopInfo.run(ActivityThread.java:3744) at android.os.Handler.handleCallback(Handler.java:751) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:154) at android.app.ActivityThread.main(ActivityThread.java:6077) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755) What’s TransactionTooLarge- Exception?

Slide 4

Slide 4 text

What’s TransactionTooLarge- Exception? Caused by: android.os.TransactionTooLargeException: data parcel size 592196 bytes

Slide 5

Slide 5 text

What’s TTLE? https://developer.android.com/reference/android/os/TransactionTooLargeException

Slide 6

Slide 6 text

What’s TTLE? https://developer.android.com/about/versions/nougat/android-7.0-changes#other

Slide 7

Slide 7 text

What’s TTLE? • targetSdkVersion ≧ 24 • Mostly happens at Activity#onSaveInstanceState • Happens when the size of outState to save > 1MB

Slide 8

Slide 8 text

Behind TTLE java.lang.RuntimeException: android.os.TransactionTooLargeException: data parcel size 592196 bytes at android.app.ActivityThread$StopInfo.run(ActivityThread.java:3752) at android.os.Handler.handleCallback(Handler.java:751) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:154) at android.app.ActivityThread.main(ActivityThread.java:6077) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755) at android.os.BinderProxy.transact(Binder.java:615) at android.app.ActivityManagerProxy.activityStopped(ActivityManagerNative.java:3606) at android.app.ActivityThread$StopInfo.run(ActivityThread.java:3744) at android.os.Handler.handleCallback(Handler.java:751) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:154) at android.app.ActivityThread.main(ActivityThread.java:6077) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755) Caused by: android.os.TransactionTooLargeException: data parcel size 592196 bytes at android.os.BinderProxy.transactNative(Native Method)

Slide 9

Slide 9 text

Behind TTLE Caused by: android.os.TransactionTooLargeException: data parcel size 592196 bytes at android.os.BinderProxy.transactNative(Native Method)

Slide 10

Slide 10 text

Why Binder?

Slide 11

Slide 11 text

Behind onSaveInstanceState Activity Record Activity Activity Activity App Process App Process Activity Record Activity Record ActivityManagerService SystemServer Process IPC = Binder

Slide 12

Slide 12 text

The Binder transaction buffer has a limited fixed size, currently 1Mb, which is shared by all transactions in progress for the process. Consequently this exception can be thrown when there are many transactions in progress even when most of the individual transactions are of moderate size. https://developer.android.com/reference/android/os/TransactionTooLargeException Binder buffer size limit

Slide 13

Slide 13 text

How to avoid the exception? • No need to persist data across Activity/process life cycle?
 → Use LiveData instead of onSaveInstanceState • Much data to save and do not want to lose them?
 → Use local file or DB instead if possible ¯\_(π)_/¯

Slide 14

Slide 14 text

How to investigate the exception? • Investigating the cause of TransactionTooLargeException is not easy • Stacktrace does not give a hint • Hard to reproduce - kind of an exception in wild

Slide 15

Slide 15 text

Hint 1: Crash log The screen before the last screen might be the suspect

Slide 16

Slide 16 text

Hint 2: TooLargeTool

Slide 17

Slide 17 text

Hint 2: TooLargeTool * MyFragment.onSaveInstanceState wrote: Bundle@161404279 contains 3 keys and measures 0.3 KB when serialized as a Parcel * androidx.lifecycle.BundlableSavedStateRegistry.key = 0.1 KB * android:user_visible_hint = 0.1 KB * android:view_state = 0.1 KB * fragment arguments = Bundle@246765028 contains 1 keys and measures 0.0 KB when serialized as a Parcel * num = 0.0 KB

Slide 18

Slide 18 text

Hint 3: FragmentStatePagerAdapter • FragmentStatePagerAdapter saves the state of all the pages into a Bundle • Total size of Bundle
 = (Size of bundle on each page) * (the number of pages) • What if each page takes ~2KB and the ViewPager holds 500 pages?

Slide 19

Slide 19 text

Hint 3: FragmentStatePagerAdapter

Slide 20

Slide 20 text

ViewPager2

Slide 21

Slide 21 text

Reference • https://github.com/hkurokawa/TransactionTooLargeSample • https://gihyo.jp/book/2017/978-4-7741-8861-4 • https://github.com/karino2/InsideBinder/blob/master/ README.md

Slide 22

Slide 22 text

Appendix
 ~ Chasing TTLE Further ~

Slide 23

Slide 23 text

Where does 1Mb
 come from? The Binder transaction buffer has a limited fixed size, currently 1Mb, which is shared by all transactions in progress for the process. Consequently this exception can be thrown when there are many transactions in progress even when most of the individual transactions are of moderate size. https://developer.android.com/reference/android/os/TransactionTooLargeException

Slide 24

Slide 24 text

Binder.cpp https://android.googlesource.com/platform/frameworks/base/+/refs/tags/android-8.1.0_r62/ core/jni/android_util_Binder.cpp#1182 case FAILED_TRANSACTION: { ALOGE("!!! FAILED BINDER TRANSACTION !!! (parcel size = %d)", parcelSize); const char* exceptionToThrow; char msg[128]; // TransactionTooLargeException is a checked exception, only throw from certain methods. // FIXME: Transaction too large is the most common reason for FAILED_TRANSACTION // but it is not the only one. The Binder driver can return BR_FAILED_REPLY // for other reasons also, such as if the transaction is malformed or // refers to an FD that has been closed. We should change the driver // to enable us to distinguish these cases in the future. if (canThrowRemoteException && parcelSize > 200*1024) { // bona fide large payload exceptionToThrow = "android/os/TransactionTooLargeException"; snprintf(msg, sizeof(msg)-1, "data parcel size %d bytes", parcelSize);

Slide 25

Slide 25 text

IPCThreadState.cpp status_t IPCThreadState::waitForResponse(Parcel *reply, status_t *acquireResult) { int32_t cmd; int32_t err; while (1) { if ((err=talkWithDriver()) < NO_ERROR) break; err = mIn.errorCheck(); if (err < NO_ERROR) break; if (mIn.dataAvail() == 0) continue; cmd = mIn.readInt32(); switch (cmd) { case … case BR_FAILED_REPLY: err = FAILED_TRANSACTION; goto finish; https://android.googlesource.com/platform/frameworks/native/+/37b4496/libs/binder/ IPCThreadState.cpp#696

Slide 26

Slide 26 text

binder.c t->buffer = binder_alloc_new_buf(&target_proc->alloc, tr->data_size, tr->offsets_size, extra_buffers_size, !reply && (t->flags & TF_ONE_WAY)); if (IS_ERR(t->buffer)) { /* * -ESRCH indicates VMA cleared. The target is dying. */ return_error_param = PTR_ERR(t->buffer); return_error = return_error_param == -ESRCH ? BR_DEAD_REPLY : BR_FAILED_REPLY; return_error_line = __LINE__; t->buffer = NULL; goto err_binder_alloc_buf_failed; } https://github.com/torvalds/linux/blob/master/drivers/android/binder.c#L3137

Slide 27

Slide 27 text

• It seems Binder driver has a limit to the transaction buffer • Binder driver allocates some amount of memory and send/ receive a message to/from a process • Is there any kernel parameter?