Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Автоматизация регрессии аналитики

Автоматизация регрессии аналитики

Talk by Andrii Rogulin

Андрей расскажет нам об автоматизации регрессии аналитики, а именно:
- Как упростить жизнь Data Analytics и QA.
- Как (почти) избавиться от Charles при проверке аналитики
- Как подружить UI тесты, сбор и проверку аналитики
- Решения и инструменты

This talk was made for CocoaFriday #4 ( https://cocoaheads.org.ua/cocoafriday/4) which took place June 14, 2019

Video: https://youtu.be/jwsohbSNZ2M

CocoaHeads Ukraine

June 14, 2019
Tweet

More Decks by CocoaHeads Ukraine

Other Decks in Programming

Transcript

  1. Better Analytics
    Andrii Rogulin

    View full-size slide

  2. – Arthur C. Nielsen, Market Researcher & Founder of ACNielsen
    The price of light is less than the cost of darkness.

    View full-size slide

  3. Why is it so important?
    A/B tests
    Conversion to subscription
    Retention
    User behaviors

    View full-size slide

  4. Analytic events manual debug in Charles

    View full-size slide

  5. Basically… it works

    View full-size slide

  6. - But what can we do?
    - Automate it!
    - But what will we get?
    Reduced time for analytics testing
    CI-driven analytics checks
    Reduced risk of human-driven mistakes
    Avoid breaks in analytics during refactoring

    View full-size slide

  7. Appium
    Why not native?
    It’s cross-platform
    It can work with Java

    View full-size slide

  8. Setup
    Homebrew
    Node+npm
    Maven (or Gradle)
    Appium
    Appium Desktop (optional)
    Appium Doctor (optional but recommended)
    System variables:
    - JAVA_HOME
    - M2_HOME
    - M2
    Java libs
    selenium-java
    TestNG
    Appium
    Allure
    ...

    View full-size slide

  9. Selenium

    (Open-Source lib)
    Web Driver Agent
    iOS Simulator
    Your Application
    Appium

    Server
    Your

    UI tests
    code
    TestNG + Maven + Java
    How it works

    View full-size slide

  10. {
    "platformName": "iOS",
    "platformVersion": "12.1",
    "deviceName": "iPhone XR",
    "app": "/Users/Me/Dev/MyAwesomeProject/Project.ipa"
    }
    Basic Setup
    Appium server

    View full-size slide

  11. WebDriverAgent

    View full-size slide

  12. Inspecting UI elements

    View full-size slide

  13. Run from scripts

    View full-size slide

  14. Build fo Simulator (fastlane)
    fastlane `gym`
    command does not
    work
    You’ll get an .app file

    View full-size slide

  15. Setting up device capabilities in your UI tests project

    View full-size slide

  16. Setting up profiles (for Maven)
    pom.xml

    My-UItests=Profile-12.1

    http://127.0.0.1:4723/wd/hub
    project.xml
    ${project.basedir}/../Project.ipa
    en


    TestNG xml
    path to your .ipa file

    View full-size slide

  17. and... run
    mvn clean test -P 'My-UItests-Profile-12.1'

    View full-size slide

  18. Unit tests 

    vs 

    High order tests
    (that’s all about analytics)

    View full-size slide

  19. Unit tests
    - Tests will vary depending on
    application’s architecture
    - Tests will be different on each
    platform (iOS, Android, etc)

    View full-size slide

  20. High order tests
    - Architecturally independent
    - Platform independent (in some way)

    View full-size slide

  21. Get the most out of your 

    High order tests

    View full-size slide

  22. The touchpoint
    is in files

    View full-size slide

  23. CocoaLumberjack

    View full-size slide

  24. File rolling by:
    - file size
    - time
    - session
    Multiple files management
    - number of files
    File reuse policy

    View full-size slide

  25. How things are going… generally
    You’re likely to have such code somewhere regardless to your app architecture

    View full-size slide

  26. Or how things can go

    View full-size slide

  27. Couple words about CI
    How Jenkins see it:

    View full-size slide

  28. Couple words about CI
    What actually happens:
    1) Cleaning build folder
    2) Updating dependencies
    3) Build for simulator
    4) Wrapping .app to .ipa

    View full-size slide

  29. Couple words about CI
    5) Pulling Automation tests repo to subfolder
    6) Checking out needed automation tests branch
    7) Starting Appium
    8) Running UI tests
    9) Killing Appium after all

    View full-size slide

  30. Allure report of failed test

    View full-size slide

  31. Expected result:
    Actual result: Duplicated events

    View full-size slide

  32. While you sleep, CI runs your tests

    View full-size slide

  33. Bonus
    - Handy analytics log to be passed as
    a new expected result to automation
    tests
    - On-device autonomous analytics
    testing
    DON’T FORGET TO REMOVE IT IN THE
    APP STORE BUILDS

    View full-size slide

  34. Results
    - Analytics regression tests that can be
    run on CI
    - Analytics log file to be passed around
    - Time economy for manual tests
    - Handy real-time analytics logging for
    manual tests
    - Autonomous analytics on-device
    testing and sharing
    - Reduced chance of human-driven
    mistakes while doing routine analytics
    checks
    - Platform and architectural
    independent tests
    and
    Analytics in NOT the only thing
    you can log and share with your 

    UI tests ;)

    View full-size slide

  35. Thanks!
    This presentation would not be possible

    without our QA team
    Andrii Rogulin

    View full-size slide