Building Android and looking into the Android System

Building Android and looking into the Android System / Android “を”ビルドしてAndroid Systemを覗いてみよう

DroidKaigi 2022 Day1 (2022/10/05 14:20-14:45)
https://droidkaigi.jp/2022/timetable/364947

Transcript

1. Building Android and looking into the Android System 5th October, 2022 Kazuki Chigita(@chigichan24) Android “を”ビルドして Android Systemを覗いてみよう
2. Kazuki Chigita (@chigichan24) Android application developer at LINE Corporation DroidKaigi 2020 DroidKaigi 2021 Profile Past talks
3. IceBreak Do you have any experience to build Android System (AOSP)? Building custom Android image as work or private! I know AOSP but no experience to build I heard first time about AOSP
4. IceBreak Do you have any experience to build Android System (AOSP)? Building custom Android image as work or private! I know AOSP but no experience to build I heard first time about AOSP
5. IceBreak Do you have any experience to build Android System (AOSP)? Building custom Android image as work or private! I know AOSP but no experience to build I heard first time about AOSP
6. IceBreak Do you have any experience to build Android System (AOSP)? Building custom Android image as work or private! I know AOSP but no experience to build I heard first time about AOSP
7. The color of navigation bar changes white to black Why will I talk about this topic (background)
8. Why will I talk about this topic (background) In the basic Android app development, often approach as below (in my case) Search in the IDE for suspicious areas. Put a bite on it Modify the code slightly to check the actual behavior Learn the detail implementation background and context…
9. Why will I talk about this topic (background) In the basic Android app development, often approach as below (in my case) Search in the IDE for suspicious areas. Put a bite on it Modify the code slightly to check the actual behavior Learn the detail implementation background and context…
10. Why will I talk about this topic (background) In the basic Android app development, often approach as below (in my case) Search in the IDE for suspicious areas. Put a bite on it Modify the code slightly to check the actual behavior Learn the detail implementation background and context…
11. Why will I talk about this topic (background) In the basic Android app development, often approach as below (in my case) Search in the IDE for suspicious areas. Put a bite on it Modify the code slightly to check the actual behavior Learn the detail implementation background and context… We can approach Android inside system too
12. Table of Contents 1 2 3 What’s AOSP? Build the Android System Deep dive into the Android
13. 1. What‘s AOSP?
14. AOSP
15. https://cs.android.com/
16. AOSP
17. More detail… You can get suitable and practical information on these sessions. My Experience creating wearable IoT devices with Android AOSP 16:50-17:15 Online 15:40-16:05 Online Yoshinori Mukai Akira Kimura Day1 Day2 Extreme modification of the Android Framework to enable beamforming using 5-channel microphones
18. 2. Build the Android System
19. Step to build Custom Android There’re so many similar/unmanaged documents. We need to compare multiple documents .
20. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3
21. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3
22. 1. Prepare build environments https://source.android.com/docs/setup/start/requirements As of June 2021, Google is using 72-core machines with 64 GB of RAM internally, which take about 40 minutes for a full build (and just a few minutes for incremental builds, depending on exactly which files were modified). By contrast, a 6-core machine with a similar amount of RAM takes 3 hours. Minimum requirements OS: ubuntu LTS ver / 64bit Storage: 250GiB RAM: 16GiB
23. 1. Prepare build environments We cannot build with macOS / windows. Should we buy new high spec ubuntu machines? https://source.android.com/docs/setup/start/requirements OS: ubuntu 20.04 (64bit) Storage: 250GiB CPU: intel 32-core RAM: 128GiB Compose spot VM with this spec on GCP My recommendation
24. 1. Prepare build environments We cannot build with macOS / windows. Should we buy new high spec ubuntu machines? https://source.android.com/docs/setup/start/requirements OS: ubuntu 20.04 (64bit) Storage: 250GiB CPU: intel 32-core RAM: 128GiB Compose spot VM with this spec on GCP My recommendation We can build on AWS too but there are some problem on emulation
25. 1. Prepare build environments Step1. Prepare VM instance on GCP https://source.android.com/docs/devices/automotive/start/avd/cloud_emulator Nested Visualization allowed disk Spot Instance Needs for AVD working
26. 1. Prepare build environments Step1. Prepare VM instance on GCP https://source.android.com/docs/devices/automotive/start/avd/cloud_emulator Nested Visualization allowed disk Spot Instance Needs for AVD working
27. 1. Prepare build environments Step1. Prepare VM instance on GCP https://source.android.com/docs/devices/automotive/start/avd/cloud_emulator Nested Visualization allowed disk Spot Instance Needs for AVD working Needs for AVD working We can finish build around 1.5 hour with this pec
28. Spot Instance 1. Prepare build environments Step1. Prepare VM instance on GCP https://source.android.com/docs/devices/automotive/start/avd/cloud_emulator Nested Visualization allowed disk ← Needs for AVD working We can save cost 50-70%
29. 1. Prepare build environments Step2. Access via XRDP We can access to instance via GUI with bit setting I recommend to XRDP tunnel through ssh since security reason
30. 1. Prepare build environments Step3. Clone AOSP source AOSP source codes are managed by Repo https://gerrit.googlesource.com/git-repo/ https://source.android.com/docs/setup/create/repo https://source.android.com/docs/setup/download/downloading Repo complements Git by simplifying work across multiple repositories.
31. $ sudo apt-get install git-core gnupg flex bison build-essential zip curl zlib1g-dev gcc-multilib g++- multilib libc6-dev-i386 libncurses5 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa- dev libxml2-utils xsltproc unzip fontconfig repo $ curl https://storage.googleapis.com/git-repo- downloads/repo > ~/bin/repo && chmod a+x ~/bin/repo $ git config --global user.name John Doe $ git config --global user.email a@example.com $ repo init -u https://android.googlesource.com/platform/manifest $ repo sync -j32
32. $ sudo apt-get install git-core gnupg flex bison build-essential zip curl zlib1g-dev gcc-multilib g++- multilib libc6-dev-i386 libncurses5 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa- dev libxml2-utils xsltproc unzip fontconfig repo $ curl https://storage.googleapis.com/git-repo- downloads/repo > ~/bin/repo && chmod a+x ~/bin/repo $ git config --global user.name John Doe $ git config --global user.email a@example.com $ repo init -u https://android.googlesource.com/platform/manifest $ repo sync -j32 Install required libs for building AOSP (2-3 min)
33. $ sudo apt-get install git-core gnupg flex bison build-essential zip curl zlib1g-dev gcc-multilib g++- multilib libc6-dev-i386 libncurses5 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa- dev libxml2-utils xsltproc unzip fontconfig repo $ curl https://storage.googleapis.com/git-repo- downloads/repo > ~/bin/repo && chmod a+x ~/bin/repo $ git config --global user.name John Doe $ git config --global user.email a@example.com $ repo init -u https://android.googlesource.com/platform/manifest $ repo sync -j32 Initial setup for repo (few seconds)
34. $ sudo apt-get install git-core gnupg flex bison build-essential zip curl zlib1g-dev gcc-multilib g++- multilib libc6-dev-i386 libncurses5 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa- dev libxml2-utils xsltproc unzip fontconfig repo $ curl https://storage.googleapis.com/git-repo- downloads/repo > ~/bin/repo && chmod a+x ~/bin/repo $ git config --global user.name John Doe $ git config --global user.email a@example.com $ repo init -u https://android.googlesource.com/platform/manifest $ repo sync -j32 Clone the AOSP codes (10 min)
35. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3
36. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh
37. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh Set up your environment for building Android devices by running the command
39. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh Pass the build build target
40. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh Pass the build build target base source type archite- cture target build variant
41. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh Pass the build build target base source type archite- cture target build variant Target Product
42. 2. Build Build variants
43. $ lunch You're building on Linux Lunch menu .. Here are the common combinations: 1. aosp_arm-eng ... 9. aosp_car_x86-userdebug ... 15. aosp_cf_x86_64_phone-userdebug ... 70. yukawa_sei510-userdebug Which would you like? [aosp_arm-eng] Pick from common choices above (e.g. 13) or specify your own (e.g. aosp_barbet-eng):
44. $ lunch You're building on Linux Lunch menu .. Here are the common combinations: 1. aosp_arm-eng ... 9. aosp_car_x86-userdebug ... 15. aosp_cf_x86_64_phone-userdebug ... 70. yukawa_sei510-userdebug Which would you like? [aosp_arm-eng] Pick from common choices above (e.g. 13) or specify your own (e.g. aosp_barbet-eng): We can select Android Auto build image too
45. $ lunch aosp_cf_x86_64_phone-userdebug ============================================ PLATFORM_VERSION_CODENAME=UpsideDownCake PLATFORM_VERSION=UpsideDownCake TARGET_PRODUCT=aosp_cf_x86_64_phone TARGET_BUILD_VARIANT=userdebug TARGET_ARCH=x86_64 TARGET_ARCH_VARIANT=silvermont TARGET_2ND_ARCH=x86 TARGET_2ND_ARCH_VARIANT=silvermont HOST_OS=linux HOST_OS_EXTRA=Linux-5.15.0-1017-gcp-x86_64-Ubuntu-20.04.5-LTS HOST_CROSS_OS=windows BUILD_ID=AOSP.MASTER The platform of current master branch is set as Android 14(U)
47. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh
48. 2. Build Step2. lunch aosp_cf_x86_64_phone-userdebug Step3. m Step1. source build/envsetup.sh https://android.googlesource.com/platform/build/+/refs/heads/master/envsetup.sh It takes 1.5 - 2.0 hour
50. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3
51. 3. Check behavior on emulator 1 2 Goldfish (well-known AVD) Cuttlefish (called as CVD)
52. 3. Check behavior on emulator 1 2 Goldfish (well-known AVD) Cuttlefish (called as CVD) My recommendation
53. 3. Check behavior on emulator Cuttlefish https://source.android.com/docs/setup/create/cuttlefish Replicate the framework-based behavior of a real device with a focus on high fidelity by maintaining close alignment with the core framework. We can use acloud command support.
54. 3. Check behavior on emulator Cuttlefish https://source.android.com/docs/setup/create/cuttlefish Replicate the framework-based behavior of a real device with a focus on high fidelity by maintaining close alignment with the core framework. We can use acloud command support. Almost same with physical device behavior except hardware-specific implementation
55. 3. Check behavior on emulator CVD related commands acloud https://android.googlesource.com/platform/tools/acloud/+/refs/heads/master/README.md acloud Simple wrapper command for cvd launch Need to install chrome for launch emulator (It cannot launch with Firefox)
58. Multi-tenancy allows for host machine to launch multiple virtual guest devices with a single launch invocation.
59. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3
60. Step to build Custom Android Prepare build environment 1 Build AOSP 2 Check behavior on emulator 3 Now we can custamize Android
61. 3. Deep dive into the Android
62. The color of navigation bar changes white to black Why will I talk about this topic (background) Reprint slide
63. Search about navigation bar Navigation Bar SystemUI Application Status Bar Power UI Volume UI Battery Meter UI etc… Search Modify Learn
64. Search about navigation bar Navigation Bar SystemUI Application Status Bar Power UI Volume UI Battery Meter UI etc… Search Modify Learn
65. Search about navigation bar Search Modify Learn SystemUI/ ├── src/common/android/systemui/ │ ├── accessibility/ │ ├── ... │ ├── navigationbar/ │ │ ├── buttons/ │ │ ├── gestural/ │ │ ├── NavigationBarHelper.java │ │ ├── NavigationBar.java │ │ ├── ... │ │ └── TaskBarDelegate.java │ ├── ... │ └── wmshell/ ├── res/ └── Android.bp
66. Search about navigation bar SystemUI/ ├── src/common/android/systemui/ │ ├── accessibility/ │ ├── ... │ ├── navigationbar/ │ │ ├── buttons/ │ │ ├── gestural/ │ │ ├── NavigationBarHelper.java │ │ ├── NavigationBar.java │ │ ├── ... │ │ └── TaskBarDelegate.java │ ├── ... │ └── wmshell/ ├── res/ └── Android.bp gestural/ seems related to gesture navigation bar Search Modify Learn
67. Modify navigation bar NavigationHandle.java Search Modify Learn
68. Modify navigation bar NavigationHandle.java replace 2 to 10 makes thick navigation bar? Search Modify Learn
69. Modify navigation bar NavigationHandle.java Search Modify Learn
70. Before After thick navigation bar is appearing navigation bar is translucent
71. Modify navigation bar NavigationHandle.java this is just canvas so we can draw bitmap too Search Modify Learn
72. Modify navigation bar NavigationHandle.java androids.bmp Search Modify Learn
73. kawaii
74. Learn navigation bar bahevior Search Modify Learn
75. Learn navigation bar bahevior The color is generated by intensity Search Modify Learn
76. Learn navigation bar bahevior If the color is difference to current one, update the color Search Modify Learn
77. Learn navigation bar bahevior Search Modify Learn
78. Learn navigation bar bahevior Search Modify Learn
79. Learn navigation bar bahevior This listener receives values of Intensity? Search Modify Learn
80. The color transition is around 1.5-2.0 seconds
81. But how to detect bg change?
82. Learn navigation bar bahevior Change navigation bar when onRegionDarknessChanged called Search Modify Learn
83. Learn navigation bar bahevior !? Search Modify Learn
84. Learn navigation bar bahevior !? Threshold of color change condition is defined here Search Modify Learn
85. Learn navigation bar bahevior Search Modify Learn Notify to navigation bar
86. Learn navigation bar bahevior !? Threshold of color change condition is defined here Search Modify Learn Check diff of current Luma
87. Learn navigation bar bahevior !? Threshold of color change condition is defined here Search Modify Learn Check diff of current Luma How implemented?
88. Learn navigation bar bahevior Search Modify Learn compositionsampling listener.java android_view_Composition SamplingListener.cpp RegionSamplingThread.cpp Bind via JNI
89. Learn navigation bar bahevior Search Modify Learn Calculate luma
90. Learn navigation bar bahevior Search Modify Learn NavigationHandle.java NavigationBar.java LightBarTransitions Controller.java NavigationBar Transitions.java RegionSampling Helper.java
91. Conclusion
92. Conclusion Building AOSP makes us more deeply understanding for the Android System. In the appendix section, I added how resolved pitfall / error. Please refer when you build the AOSP.
93. Thank you If you have any questions / comments, let’s talk on ask the speaker booth.
94. @Licensing The Android robot is reproduced or modified from work created and shared by Google and used according to terms described in the Creative Commons 3.0 Attribution License. Twemoji. Copyright 2020 Twitter, Inc and other contributors Graphics licensed under CC-BY 4.0: https://creativecommons.org/licenses/by/4.0/
95. Bibliography(1) 1. https://source.android.com/ 2. https://source.android.com/docs/setup/build/downloading 3. https://source.android.com/docs/setup/start 4. https://source.android.com/docs/core/architecture 5. https://source.android.com/docs/setup/start/requirements 6. https://source.android.com/docs/devices/automotive/start/avd/cloud_emulator 7. https://www.youtube.com/watch?v=eaPUdDkE91g 8. https://2net.co.uk/slides/aosp-aaos-meetup/2022-march-cuttlefish.pdf 9. https://cs.android.com/ 10. https://cs.android.com/android/platform/superproject/+/master:frameworks/ba se/packages/SystemUI/src/com/android/systemui/navigationbar/gestural/Navig ationHandle.java;bpv=0;bpt=0 11. https://android-developers.googleblog.com/2017/05/here-comes-treble- modular-base-for.html
96. Bibliography(2) 12. https://android.googlesource.com/platform/tools/acloud/+/refs/heads/master/R EADME.md 13. https://source.android.com/docs/setup/download/downloading 14. https://source.android.com/docs/setup/start/initializing 15. https://source.android.com/docs/setup/build/known-issues 16. https://developer.android.com/studio/run/emulator-acceleration 17. https://source.android.com/docs/setup/create/cuttlefish 18. https://source.android.com/docs/setup/create/cuttlefish-ref-multi-tenancy 19. https://source.android.com/docs/setup/build
97. Appendix Q. How to setup ubuntu-desktop env? A. sudo apt update -y && sudo apt upgrade -y sudo adduser desktop_user sudo gpasswd -a desktop_user sudo sudo apt -y install ubuntu-desktop sudo apt install -y xrdp sudo sed -e 's/^new_cursors=true/new_cursors=false/g' -i /etc/xrdp/xrdp.ini
98. Appendix Q. How to setup xrdp env? A. sudo apt install -y xrdp sudo sed -e 's/^new_cursors=true/new_cursors=false/g' -i /etc/xrdp/xrdp.ini sudo systemctl restart xrdp sudo systemctl enable xrdp.service sudo systemctl enable xrdp-sesman.service https://qiita.com/osugizmo/items/21578272ae2db05ab001
99. Appendix Q. Can we build on ubuntu on VMWare? A. Yes. But it’s not recommended since we need high spec machine for building AOSP.
100. Appendix Q. Can we build on WSL? A. Yes. I didn’t try in my environment but I found the article that works on WSL2.
101. Appendix Q. Can we build on AWS instance? A. Yes but not recommended. We need to find machine spec that kvm(kernel- based virtual machine) works. It seems only bare metal instance.
102. Appendix Q. Can we build on lower version ubuntu? A. Yes but not recommended. It works fine that only focusing on build. But acloud / Cuttlefish is not work fine on ubuntu 18.04 below(in my case)
103. Appendix Q. repo is not work correctly. A. Please confirm your python version. If your python version is 2.x . It may fail some step.
104. Appendix Q. repo sync is not finish A. Sometimes, it stops 99% by network error. please try below command sysctl -w net.ipv4.tcp_window_scaling=0 https://source.android.com/docs/setup/build/known-issues
105. Appendix Q. I’m not sure kvm works fine. A. You should watch this page. https://developer.android.com/studi o/run/emulator-acceleration#vm- linux
106. Appendix Q. Build is not finish successfully(1) A. Is there enough space? You can check with df -Ph . At least 250GiB needs for build AOSP.
107. Appendix Q. Build is not finish successfully(2) A. Did you finish the pre-install step? We need to some libs for build. If the error message includes about yacc, the install is not finished yet. https://source.android.com/docs/setup/start/initializing
108. Appendix Q. acloud setup is failing A. Please confirm your ubuntu version is 20.04 or later. A. Please check kvm is correctly setup.
109. Appendix Q. acloud instance is not launch A. Please set chrome as default browser. It’s not work on FireFox which is default browser on Ubuntu.