Making an Efficient Deploy Bot

These are the slides for my presentation at "Build Eureka.swift × Bitrise"
Event page: https://eure.connpass.com/event/121871/

Note: This topic is an update to my previous presentations at other meetups. We are continuously improving our deployment builds and trying out different tools and services.

Transcript

1. MAKING AN EFFICIENT
   DEPLOYMENT BOT
   build eureka.swi. × Bitrise 2019/3/28
   @JohnEstropia
   

   View Slide

2. @JOHNESTROPIA
   ➤ Senior Engineer @
   ➤
   ➤ !
   ➤ Dev since 2008 (mostly mobile apps)
   ➤ iAppli, BREW, Windows CE/Mobile
   ➤ iOS 4.0~ (Swift beta~)
   ➤ http://github.com/JohnEstropia/CoreStore
   

   View Slide

3. AGENDA
   ➤ Why we needed a deploy bot in Pairs JP
   ➤ Meet our deploy bot
   ➤ Future features
   

   View Slide

4. WHY WE NEEDED
   A DEPLOY BOT
   

   View Slide

5. PAIRS JP IOS TEAM
   

   View Slide

6. SERVER ENVIRONMENT (XCODE SCHEMES)
   Staging (Development) Production
   

   View Slide

7. DEPLOYMENT TARGETS
   AppStore
   Crashlytics TestFlight
   

   View Slide

8. PROBLEM 1: COMPLEX BUILD PATTERNS
   Staging
   Production
   ?
   ?
   ? ?
   ? ?
   

   View Slide

9. PROBLEM 1: COMPLEX BUILD PATTERNS
   ➤ Solution:
   ➤ Fastlane
   

   View Slide

10. PROBLEM 2: WAITING FOR BUILD SCRIPTS
    Staging
    Production
    

    View Slide

11. PROBLEM 2: WAITING FOR BUILD SCRIPTS
    ➤ Pairs JP project:
    • *.swift: 130,000~ lines of code
    • *.m: 8,000~ lines of code
    • Dependencies:
    • Pods: 50~ frameworks
    • Carthage: 20~ frameworks
    ➤ Build Time:
    • Debug: 5~ minutes (+ preparation + upload time)
    • Release: 15~ minutes (+ preparation + upload time)
    

    View Slide

12. PROBLEM 2: WAITING FOR BUILD SCRIPTS
    Staging
    Production
    ɾɾɾ
    

    View Slide

13. PROBLEM 2: WAITING FOR BUILD SCRIPTS
    ➤ Solution:
    ➤ Continuous Integration / Delivery
    ➤ We currently use
    ➤ Work on something else while CI/CD runs
    

    View Slide

14. ALL PROBLEM SOLVED?
    

    View Slide

15. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    Code
    Freeze
    

    View Slide

16. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    Code
    Freeze
    QA Test
    

    View Slide

17. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    Code
    Freeze
    QA Test
    AppStore
    Submission
    

    View Slide

18. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    Code
    Freeze
    QA Test
    AppStore
    Submission
    
    

    View Slide

19. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    AppStore
    Release
    Code
    Freeze
    QA Test
    AppStore
    Submission
    
    

    View Slide

20. OUR WEEKLY DEPLOYMENT SCHEDULE
    Mon Tue Wed Thur Fri Sat Sun
    AppStore
    Release
    Code
    Freeze
    QA Test
    AppStore
    Submission
    

    View Slide

21. OUR WEEKLY DEPLOYMENT SCHEDULE
    ➤ Wednesdays: Code-freeze
    ➤ Finish new features for next version
    ➤ Deploy testing binary (for QA)
    ➤ Fridays: AppStore Submission
    ➤ Finish hotfixes for those features
    ➤ Deploy production binary (for AppStore review)
    ➤ Delays = overtime / reschedule update
    ➤ Monday ~ Friday: Anyone can deploy their own testing binary
    

    View Slide

22. PAIRS JP IOS TEAM: PARALLEL DEVELOPMENT
    

    View Slide

23. PROBLEM 3: PRIORITY BUILDS
    Staging
    Production
    

    View Slide

24. MEET OUR
    SLACKBOT
    @porygon
    

    View Slide

25. @PORYGON: SLACKBOT
    ➤ Written in Swift!
    ➤ Runs on a dedicated local iMac
    ➤ Own file system
    ➤ Own keys/accounts
    ➤ Own integrations
    

    View Slide

26. PAIRS JP DEPLOYMENT FLOWS
    

    View Slide

27. PAIRS JP DEPLOYMENT FLOWS
    

    View Slide

28. @PORYGON SKILLS
    

    View Slide

29. DEPLOY: VIA CHAT COMMAND
    

    View Slide

30. DEPLOY: ANIMATED PROGRESS
    

    View Slide

31. DEPLOY: HTML LOGS
    

    View Slide

32. DEPLOY: CUSTOM REPORTING
    

    View Slide

33. DEPLOY: REPORTING FAILURES
    

    View Slide

34. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

35. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

36. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

37. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

38. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

39. YOUR OWN BOT = SKY’S THE LIMIT
    

    View Slide

40. FUTURE
    FEATURES
    (WIP)
    

    View Slide

41. @Z: FASTER BRANCH-SWITCHING/CHECKOUTS
    ➤ Using git worktrees
    ➤ Load all repositories on initial execution only
    ➤ fetch+worktree magic on branch change
    

    View Slide

42. @Z: THREADING MICRO-MANAGEMENT
    ➤ fastlane scripts are broken down by relevant queues
    %FQMPZ
    %FQMPZ
    #&'03&
    

    View Slide

43. @Z: THREADING MICRO-MANAGEMENT
    ➤ fastlane scripts are broken down by relevant queues
    %FQMPZ
    %FQMPZ
    "'5&3
    

    View Slide

44. @Z: THREADING MICRO-MANAGEMENT
    ➤ fastlane scripts are broken down by relevant queues
    ➤ Run unrelated tasks in parallel
    ➤ No need to wait for whole deploy scripts before running the next
    

    View Slide

45. @Z: BETTER, INFORMATIVE PROGRESS REPORTING
    

    View Slide

46. @Z: BETTER, INFORMATIVE PROGRESS REPORTING
    

    View Slide

47. @Z: OTHER FEATURES
    ➤ Self-updating
    ➤ Type-safe Swift config file
    ➤ User permissions (by Slack user ID, workgroup)
    ➤ Team-specific features
    ➤ Example: generating QR codes for testing universal deeplinks
    

    View Slide

48. WRAP-UP
    ➤ We’re designing our build process to
    balance speed & convenience
    ➤ WIP
    ➤ Exploring Bitrise workflows to
    optimize micro-tasks
    ➤ Optimize our dedicated bot to fill in
    the gaps
    ➤ super-customizations
    ➤ Also hoping Bitrise would fill in
    some of those gaps
    

    View Slide

49. THANKS FOR YOUR TIME!
    ➤ Twitter: @JohnEstropia
    ➤ Github: github.com/JohnEstropia
    

    View Slide