Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Integrating and Releasing code at Tuenti: an automated pipeline

Tuenti
March 08, 2014

Integrating and Releasing code at Tuenti: an automated pipeline

Integrar, probar y distribuir el código en el que trabajan unos 30 desarrolladores supone un reto. Diseñando procesos flexibles, pero cuyo cumplimiento sea estricto, y automatizándolos, en Tuenti hemos aplicado al desarrollo de aplicaciones móviles las lecciones que nos permitieron incrementar el número de releases de código de nuestra web de dos a la semana a 2 al día, resultando en un proceso transparente al desarrollador, y con todas las características que se exigen de un proceso de release: seguridad, trazabilidad, repetibilidad, automatización y eficiente flujo de información de arriba a abajo y de abajo arriba.

Tuenti

March 08, 2014
Tweet

More Decks by Tuenti

Other Decks in Technology

Transcript

  1. Integrating and Releasing code at
    Tuenti: an automated pipeline
    @oscarsj

    View Slide

  2. Problem
    ● Lots of developers writing code at the same
    time
    ● Buggy code gets to main development
    branch
    ● A lot of time wasted checking test results,
    performing builds, merging branches,
    publishing versions…
    ● Information missing, lots of meetings
    required

    View Slide

  3. Solution
    Create a process that is:
    ● Well-defined
    ● Repeteable
    ● Trackable
    ● Automated (as far as possible)
    How?

    View Slide

  4. Agenda
    1. Overview
    a. Dev/Release system architecture
    b. Flow
    2. Process
    a. Development
    b. Integration
    c. Distribution
    d. Release
    e. Publishing

    View Slide

  5. Dev/Release system architecture

    View Slide

  6. Dev/Release system architecture

    View Slide

  7. Dev/Release system architecture

    View Slide

  8. Dev/Release system architecture

    View Slide

  9. Dev/Release system architecture

    View Slide

  10. Flow

    View Slide

  11. Dev/Release system architecture

    View Slide

  12. Mercurial (SCM)
    ● Mercurial/Git: distributed scm
    ○ every repo is a full copy, with history
    ○ Developers work on their copy, and frequently
    synchronize with central repo (push/pull)
    ● Protected branches:
    ○ integration
    ○ release-*
    ○ hotfix-*
    ○ live

    View Slide

  13. Mercurial (SCM)

    View Slide

  14. Mercurial (SCM)
    ● Sanity check “hooks”
    ○ Syntax checks
    ○ No conflict markers
    ○ All commits with ticket identifier
    ○ Data formats validation
    ● Tags:
    ○ All versions published are uniquely identified by
    version
    ○ This allow to track back errors to the code

    View Slide

  15. Dev/Release system architecture

    View Slide

  16. Jenkins
    mvn

    View Slide

  17. Dev/Release system architecture

    View Slide

  18. Hockeyapps
    ● Development versions distribution system
    ○ http://hockeyapp.net/
    ● Stores versioned apks, along with symbol
    tables
    ● Provides an API to include in the app:
    ○ Send crash reports to hockeyapps
    ○ Notifies and installs new versions from server

    View Slide

  19. Hockeyapps
    ● Projects:
    ○ alphas: loose versioning, teams manage it
    themselves
    ○ preproduction: strict, production versioning. Only
    versions built from integration/release pipeline
    ○ production: no distribution, only to store crashes

    View Slide

  20. Hockeyapps
    ● Development versions generated locally, by
    devs
    ● Preproduction versions generated from
    integration and release branches
    ○ Tagged, and version increased after successful
    publish
    ● Versioning in integration and release branch
    ○ Major.minor.build

    View Slide

  21. Dev/Release system architecture

    View Slide

  22. Jira
    ● Issue tracker
    ○ Rest API
    ○ Hooks -> events
    ● All info of project stored here
    ○ Human generated: Content of branches, risks,
    effects on customers, etc
    ○ Auto generated: Results of builds, times of
    deployment, etc
    ● Process triggered from here
    ○ Integrate branch
    ○ Create releases
    ○ Publish versions

    View Slide

  23. Dev/Release system architecture

    View Slide

  24. Release Manager
    ● Controls processes
    ● Checks tests status
    ● Checks integrity of builds and branches
    ● Checks for abnormal situations
    ● Broadcast info for stakeholders
    ● Operates scripts to build and deploy to
    different environments

    View Slide

  25. Flow

    View Slide

  26. Flow

    View Slide

  27. Process
    ● From development (my laptop) to production
    ● As fast as possible: all code finished but not
    published gets old
    ● Quality is integral part of the process:
    ○ devs: automated testing
    ○ product: ensure quality as a feature
    ○ leads: provide means with scheduling and formation
    ● Makes sure code in integration branch is
    always “good” (passes tests)

    View Slide

  28. Process: Development
    1. Developer branches from integration
    2. Creates a INT (integration request) ticket
    3. Develops, tests
    4. Peer code reviews
    5. Pull from integration branch
    6. Translations
    7. Ready (reviewed+tests passing+feature
    accepted): pull request
    8. Waits for feedback

    View Slide

  29. Process: Integration
    ● All info to merge a branch in jira ticket:
    ○ Changeset
    ○ Author/Reviewer
    ○ Contents
    ○ Risks...
    ● Create temporal branch with integration +
    feature branch
    ● Test that temporal branch
    ● Blue ball: integrated, otherwise: rejected
    ● Feedback:
    ○ Jira
    ○ Mail

    View Slide

  30. Process: Distribution
    ● Integration latest changeset gets build in
    “production” mode (no debug, ofuscated,
    prod. key)
    ● If build succeeds:
    ○ Pushed to hockeyapps
    ○ Version (build number) increased
    ○ Tag in mercurial with version number
    ○ Feedback via mail, with information, changelog and
    link to donwload
    ● Periodically, at least once a week
    ○ Too often may be counterproductive

    View Slide

  31. Process: Release
    ● Integration latest changeset get branched
    ● Latest translations included
    ● Latest hotfixes included
    ● Product version set (major.minor) if needed
    ● Build/distribution
    ○ all company encouraged to install
    ● Stabilization:
    ○ 3 days
    ○ devs must check and fix crashes related to release
    ○ at the end of each day a new build is distributed

    View Slide

  32. Process: Release
    ● Release stable: agreement between
    developers and product owners
    ● Google play submit
    ○ progressive roll-out:
    ■ 8 hours to 5%
    ■ increased to 100% during next 8 hours
    ● Release branch is merged to live branch
    ○ hotfixes branched from here
    ● Release branch is merged to integration
    ○ via pull request

    View Slide

  33. Process: Publishing
    ● multiple architectures
    ● Progressive roll-out
    ○ 5% for 24 hours
    ○ Progressive increase during another 8 hours
    ● Manual process :(

    View Slide

  34. QUESTIONS?
    @oscarsj

    View Slide