No content

Introduction 
 ● Tadashi Nemoto
 ● QA Automation Engineer / Mercari, inc.
 ○ Selenium / Appium / XCUITest
 ○ CI / CD pipeline
 ○ Cloud-based test environment
 tadashi0713 tadashi-nemoto 

Agenda
 ● About Mercari, inc.
 ● About our automation tests and problems
 ● What is Docker-Android, 
 and Docker-Android in the cloud
 ● Demo
 ● Challenges to be solved
 

https://tech.mercari.com/entry/2018/12/28/175147 

About Mercari, inc. 

About February 1st, 2013 Established Tokyo, Sendai, Fukuoka, Palo Alto, Portland, Boston Offices 1800+ Including subsidiaries Headcount 6 

allow individuals to buy and sell items and share information through their phones. We created “Mercari,” a C2C marketplace app where users can use their smartphones to easily buy and sell secondhand goods. By linking sellers who own items to buyers who see the value in these objects, our goal is to provide an easy, fun, and safe and secure service that creates new value and redefines the cycle of consumption. In modern society, where mass production and mass consumption have become commonplace, the value of items that fall into disuse each year is expected to grow to ¥7.6 trillion in Japan alone.* Of these items, there are many that are no longer useful to one person, but could be used by someone else who values them. The Mercari Group aims to use the popularization of smartphones and social media to Create value in a global marketplace where anyone can buy & sell Mission Source: Ministry of Economy, Trade and Industry (April 2018). Estimated total value of items described as “items that became no longer needed over the past year”. Excludes aggregate value of automobiles, motorcycles and mopeds 7 

What Is Mercari? ●Service start: July 2013 ●OS: Android, iOS *Can also use from web browsers ●Usage fee: Free *Commission fee for sold items: 10% of the sales price ●Regions/languages supported: Base specs for Japan/Japanese Many sellers enjoy having the items they no longer need purchased and used by buyers who need them, and buyers enjoy the feeling of hunting for treasure as they search through unique and diverse items for lucky finds. In addition to buying and selling, users actively communicate through the buyer/seller chat and the “Like” feature. The Mercari app is a C2C marketplace where individuals can easily sell used items. We want to provide both buyers and sellers with a service where they can enjoy safe and secure transactions. Mercari offers a unique customer experience, with a transaction environment that uses the payments Mercari holds in escrow, and simple and affordable shipping options. 8 

By the Numbers (JP/Full Year) 232 346.8 GMV¹ 490.2 B
 In: billion JPY FY 06/2017 FY 06/2018 21.2 33.4 Net Sales 46.2 B
 In: billion JPY FY 06/2017 FY 06/2018 8.45 10.75 MAU² 13.57 M
 In: Million people FY 06/2017 FY 06/2018 Source: Internal documents, from FY2018.6 Presentation Material
 1. GMV after cancellation
 2. Monthly Active Users in June. Number of registered users that used our app in the month
 490.2 FY 06/2019 46.2 FY 06/2019 13.57 FY 06/2019 9 

Building a new ecosystem through Merpay Merpay is a mobile payments service operated by Merpay, Inc. The company belongs to the Mercari Group, widely recognized for its service “Mercari,” the top C2C marketplace app in Japan. Money earned by selling unwanted items on the Mercari app and money charged to the app through users’ bank accounts can be used to make payments at stores and on the Mercari app itself. Our future goal is to provide a comprehensive financial service by combining primary and secondary distribution based on data, and supporting the business of merchants while utilizing users’ credit information, such as their transaction history and ratings. Primary distribution Manufactures, retail, distributors, etc. Secondary distribution Marketplace app Attract customers Marketing Transaction data Future Business Development Building an Ecosystem 

“The Selling App.” Mercari’s mission is to “Create value in a global marketplace where anyone can buy & sell,” and the year after our founding, we began to expand overseas. We started our business in the US in April 2014, and we adapted our branding, improved our UI/UX, and created a delivery network to match local preferences and the characteristics of the US market. We recognize that succeeding in the US, with its enormous and diverse population, is an important milestone to achieve our mission, so that is where we are devoting our energy. 11 Future Business Development Strengthening Overseas Expansion To Make Selling Easier than Buying. 

About our automation tests and problems 

No content

UI・E2E Tests of Android
 Build & upload apk Kick tests 

No content

No content

Problems
 ● Increasing test execution time
 ● Maintenance cost of real devices
 ● Test failures due to differences of real devices
 (e.g. display size, OS version, etc.)
 ● Maintenance cost of local environments
 (e.g. Appium, Ruby, etc.)
 ● These problems often made tests flaky and unreliable
 

Our headcount tripled in 3 years
 1,826 376 462 555 596 712 795 1,014 1,140 1,357 1,639 1,786 Changes in total headcount 1Q 2Q 3Q 4Q FY2016.6 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q FY2017.6 FY2018.6 FY2019.6 from Mercari Bold Challenge 

The number of engineers also increased greatly
 FY2018 FY2019 FY2017 FY2016 from Mercari Bold Challenge 

We need faster and more stable QA feedback
 

Test Pyramid
 https://martinfowler.com/bliki/TestPyramid.html 

Mobile Testing Pyramid
 http://elementalselenium.com/tips/71-mobile-testing-pyramid 

Mobile Testing Pyramid
 http://elementalselenium.com/tips/71-mobile-testing-pyramid 

Azure, Docker, Android Emulator
 

Azure, Docker, Android Emulator
 

Summary
 ● We first created a local environment (local PC, real devices) to run Android E2E・UI Tests
 ● We faced some problems, and needed faster and more stable QA feedback
 ● Based on the Mobile Testing Pyramid, we tried creating parallelized test environment using Azure, Docker, and Android Emulator
 

What is Docker-Android, and Docker-Android in the cloud 

Advantages of using
 ● Docker
 ● Android Emulator
 ● Cloud server
 

Docker
 ● Easy to scrap and build
 ● Easily recreate same
 environment(local, CI)
 ● Isolated environment
 ● Infrastructure as Code
 

Cloud server(IaaS)
 ● Reduces maintenance cost compared to on-premises server
 ● Easy to scale
 ● Cost determined by usage
 

Android Emulator
 ● Can reduce maintenance cost
 of real devices
 ● Clean state
 ● Can emulate
 ○ SMS
 ○ Push notification
 

Docker-Android
 github.com/docker-android
 

Advantages of Docker-Android
 ● Uses fast Android x86 emulator
 ● Starts Appium server inside container
 ● Can connect to Selenium Grid easily
 ● Remote connect using noVNC
 ● Can capture videos
 

Docker-Android Selenium Grid
 

How to run Docker-Android
 in the cloud 
 ● Android x86 emulator needs hardware virtualization
 ● Can only run limited cloud service / Instances
 ○ Bare metal Instance(AWS)
 ○ Nested virtualization(Azure)
 
 

AWS
 https://aws.amazon.com/jp/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/ 

Azure (Nested Virtualization)
 https://azure.microsoft.com/en-us/blog/nested-virtualization-in-azure/ 

Run Docker-Android in Nested Virtualization
 https://stackoverflow.com/questions/49099428/is-it-possi ble-to-run-kvm-on-azure 

Summary
 ● With Docker-Android and Cloud server(Azure), we could create parallelized testing environments
 ● We can also benefit from using
 ○ Docker
 ○ Android Emulator
 ○ Cloud server
 

Demo 

https://github.com/tadashi0713/webdriverio-appium-on-docker-android Demo 

Challenges to be solved 

Challenges to be solved
 ● Test report
 ● Optimise distributed tests
 ● Server load on QA environment
 ● Divide role of emulators and real devices
 

Test report 

Rspec HTML Reporter
 ● Problems
 ○ Difficult to find history and trend
 ○ Generate reports by number of parallel
 

Allure
 

Optimise distributed tests 

https://docs.knapsackpro.com/2018/improve-circleci-para llelisation-for-rspec-minitest-cypress Optimise distributed tests
 

https://docs.knapsackpro.com/2018/improve-circleci-para llelisation-for-rspec-minitest-cypress Knapsack (Pro)
 

Server load on QA environments 

● Solution
 ○ Use mock server instead
 ○ Improve QA environment infracture
 Server load to QA Environments
 API server
 (QA)
 

Divide roles of emulators and real devices 

Mobile Testing Pyramid
 http://elementalselenium.com/tips/71-mobile-testing-pyramid 

Real devices: real-time conditions(Headspin)
 https://tech.mercari.com/entry/2019/02/18/173236 

Summary
 ● We still need to solve the below challenges for faster and more stable QA feedback
 ○ Test report
 ○ Optimise distributed tests
 ○ Server load on QA environment
 ○ Divide roles of emulators and real devices
 

Conclusion 

Message(Key takeaways)
 ● We faced problems in testing environments, and need faster, more stable QA feedback
 ● Following the Mobile Testing Pyramid, we tried creating parallelized test environment using Azure, Docker, and Android Emulator
 ● There are challenges to be solved apart from the test environment for faster and more stable QA feedback
 

No content