Share
Embed
Speed up your build with remote servers
Talk where I explain how you can use a remote server to speed up your build using Mainframer script and/or Mirakle plugin
Enrique Ramirez
December 20, 2019
Transcript
37 slides
-
First approach Copy entire codebase to remote server build and
run remotely adb rsync over tcp -
$ rsync <local_project_path> <user>@<remote_server_ip>://<remote_project_path> $ rsync ./ [email protected]://home/kikermo/workspace/todoApp/ $ ssh
<user>@<remote_server_ip> $ ssh [email protected] $ ./gradlew <gradlew_task> $ ./gradlew assembleDebug $ adb install -r <path_to_apk> $ adb install -r app/build/output/apk/debug/myapp.apk $ adb shell am start -n <application_id>/<launch_activity> $ adb shell am start -n org.myapp/org.myapp.LaunchActivity -
$ ./mainframer.sh ./gradlew <gradlew_task> $ ./mainframer.sh ./gradlew assembleDebug $ adb
install -r <path_to_apk> $ adb install -r app/build/output/apk/debug/myapp.apk $ adb shell am start -n <application_id>/<launch_activity> $ adb shell am start -n org.myapp/org.myapp.LaunchActivity -
Server side 1. Choose a linux distro and install it.
2. Install Java. 3. Install android sdk. 4. Make sure you have an ssh server installed. 5. Create a user for the each dev (useradd). 6. Add sdk path to /etc/environment /etc/environments export ANDOROID_HOME=/usr/lib/android-sdk export ANDROID_SDK_ROOT=/usr/lib/android-sdk -
Client side (common) 1. Generate rsa key (if you don’t
have one already). 2. Copy public key to remote server. 3. Create an ssh config file. 4. Check you can ssh into the remote machine without using a password. -
~/.ssh/config Host * UseKeychain yes AddKeysToAgent yes IdentityFile ~/.ssh/id_rsa Host
mainframer User kikermo HostName 192.168.2.112 Port 22 IdentityFile ~/.ssh/id_rsa.pub PreferredAuthentications publickey ControlMaster auto ControlPath /tmp/%r@%h:%p ControlPersist 1 $ ssh-keygen $ ssh-copy-id <user>@<remote_machine> $ vim ~/.ssh/config $ ssh-add $ ssh mainframer -
Client side (mainframer) 1. Download mainframer v2 script 2. Put
it in your project 3. Configure ignore files for the sync a. Config b. Common ignore c. Local -> Remote ignore d. Remote -> Local ignore 4. Aliases (Optional) -
$ wget https://raw.githubusercontent.com/buildfoundation/mainframer/v2.1.0/mainframer.sh <project_dir>/.mainframer/config remote_machine=mainframer local_compression_level=1 remote_compression_level=5 <project_dir>/.mainframer/ignore .idea/* *.iml
*.DS_Store local.properties <project_dir>/.mainframer/localignore build <project_dir>/.mainframer/remoteignore src -
Aliases ~/.bash_profile or ~/.bashrc … alias mf=’./mainframer.sh’ alias installDebug=’adb install
-r app/outputs/apk/<your_build_variant_dir>’ alias runDebug=’adb shell am start -n <application_id>/<launch_activity>’ alias mfBuild=’./mainframer.sh ./gradlew assmeble<YourVariant> && installDebug && runDebug’ alias mfCleanBuild=’./mainframer.sh ./gradlew clean assmeble<YourVariant> && installDebug && runDebug’ alias mfTest=’./mainframer.sh ./gradlew <your_gradle_task_for_tests>’ alias mfSA=’./mainframer.sh ./gradlew <your_gradle_task_for_static_analysis>’ -
Client side (mirakle) 1. Create mirakle config file. ~/.gradle/init.d/mirakle_init.gradle initscript
{ repositories { jcenter() } dependencies { classpath "com.instamotor:mirakle:1.3.2" } } apply plugin: Mirakle rootProject { mirakle { host "mainframer" } } -
The Windows Problem Although there is a native implementation of
SSH, there are some required UNIX based commands that are not implemented the OS such as rsync. Possible solutions • Get a macbook • MinGW • Cygwin • Docker • WSL (Windows Subsystem for Linux) -
WSL (Windows Subsystem for Linux) Admin PowerShell Enable-WindowsOptionalFeature -Online -FeatureName
Microsoft-Windows-Subsystem-Linux -
MDNS/DNS-SD/Bonjour/Zeroconf • Allows you to discover your remote machine without
having to use an IP address. • Only work on local networks, no VPN. • AVAHI server, common open source solution. $ ping 192.168.2.112 without $ ping myserver.local with -
Aliases (extended) ~/.bash_profile or ~/.bashrc … ## Mirakle aliases alias
mirakleOff=’mv ~/.gradle/init.d/mirakle_init.gradle ~/.gradle/init.d/mirakle_init.gradle.conf’ alias mirakleOn=’mv ~/.gradle/init.d/mirakle_init.gradle.conf ~/.gradle/init.d/mirakle_init.gradle’ ## Mainframer aliases alias speedUp=’echo -e “build\n!app/build/outputs”>> .mainframer/remoteignore’ alias slowDown=’git checkout .mainframer/remoteignore’ ## discard changes alias build=’speedUp && ./mainframer.sh ./gradlew assembleDebug && installDebug && runDebug && slowDown && ./mainframer.sh’ -
Dynamic user creation on server • Compatible with AD/LDAP. •
User creation on the fly. • Kerberos. • Needs admin authorization. -
Bash script for new users • Helps onboarding new devs.
• Avoids unnecessary one time learning curve • Less troubleshooting needed -
Advantages • Laptop doesn’t heat too much & battery last
longer. • IDE doesn’t become unresponsive when building. • You can run test or static analysis in parallel while you code. • Slow networks might make the build syncing slower than building locally. • Requires configuration for every new developer. • Changing branches could generate a strange issues, needing to delete the remote workspace. • Requires a bit of maintenance. Disadvantage -
Links • @MarkAllison (Styling Android) https://blog.stylingandroid.com/mainframer-mirakle/ • @Balachandarkm https://medium.com/@balachandarkm/speed-up-android-builds -with-mainframer-docker-container-in-a-powerful-remote-ma
chine-ce7b9e122a49 • Mainframer repo https://github.com/buildfoundation/mainframer • Mirake rempo https://github.com/Instamotor-Labs/mirakle -
Links • Active Directory dynamic user creation https://help.ubuntu.com/lts/serverguide/sssd-ad.html • SSH
tunneling https://help.ubuntu.com/community/SSH/OpenSSH/PortForward ing -
Configuration Links • /etc/environmetns • ~/.ssh/config • <project_dir>/.mainframer/config • <project_dir>/.mainframer/ignore
• <project_dir>/.mainframer/localignore • <project_dir>/.mainframer/remoteignore • Aliases ~/.bash_profile or ~/.bashrc • ~/.gradle/init.d/mirakle_init.gradle