Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introducing Continuous Delivery of Mobile Apps
 in a Corporate Environment: A Case Study

Introducing Continuous Delivery of Mobile Apps
 in a Corporate Environment: A Case Study

Presentation at RCoSE workshop 2015 in Florence, Italy

Stephan Krusche

May 23, 2015
Tweet

More Decks by Stephan Krusche

Other Decks in Research

Transcript

  1. 1
    Sebastian Klepper, Stephan Krusche, Sebastian Peters, Bernd Bruegge and Lukas Alperowitz
    Technische Universität München, Chair for Applied Software Engineering
    Introducing Continuous Delivery of Mobile Apps

    in a Corporate Environment: A Case Study
    23 May 2015
    RCoSE 2015
    Stephan Krusche
    Twitter: @skrusche
    Email: [email protected]
    Web: www.skrusche.de

    View full-size slide

  2. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study 2
    Digital technologies become key factors for
    companies, in particular mobile computing
    Mobile-Computing is a strong
    driver for digital transformation
    Agile Development is the key to
    an effective realization

    View full-size slide

  3. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Rugby’s Release Management Workflow
    3
    Version
    Control
    Server
    Developer
    1
    notify
    upload
    build 5
    download
    6
    Issue
    Tracker
    notify
    store feedback
    10
    release
    4
    Release

    Manager
    checkout, compile
    and test build
    2
    upload feedback
    give
    feedback
    9
    Continuous
    Integration
    Server
    Continuous
    Delivery
    Server
    7
    Device
    8
    inform about
    build status
    commit
    3
    11
    User
    RCoSE 2014
    Evaluation in university projects —> leads to more releases & feedback

    View full-size slide

  4. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Situation at Capgemini revealed with interviews
    Continuous integration and delivery of mobile apps in a (heterogeneous) project-
    based organization is complex and expensive and therefore neglected at Capgemini
    4
    Process
    No standardized development process for mobile applications,
    often isolated custom solutions
    Continuity Continuity is missing in the development
    Infrastructure Existing project environments are not sufficient
    Prioritization
    Automatic delivery and testing is neglected in favor of
    development speed

    View full-size slide

  5. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Additional Requirements at Capgemini
    • Support of multiple mobile platforms (including cross-platform)
    • High security for business critical applications
    • Consideration of access control and data privacy
    • Support of different project environments
    • Consideration of dependency management
    • Ability to distinguish between user-generated and automatic feedback
    • Ability to manually deliver an application
    ➡Project managers want the ability to tailor the workflow to their own needs
    5

    View full-size slide

  6. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study 6
    User
    Developer
    Delivery
    Integration
    Configuration
    Management
    Feedback
    Components of the release management workflow

    View full-size slide

  7. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Rugby’s extended and tailorable release management workflow
    7
    Developer
    Test
    2.2
    Delivery Service
    Integration System
    Repository
    Push

    changes
    1.1
    Observe/notify

    to trigger build
    1.2
    Fetch changes
    1.3
    Upload app
    3.1
    Package
    2.3
    Collect

    metrics
    2.4
    Issue Tracker
    Observe changes
    1.5
    Observe/notify

    about build
    2.5
    Resolve

    dependencies
    1.4
    Update

    status
    4.4
    Build
    2.1
    Release Manager
    Send email
    3.2
    Update status
    4.2
    Integration
    Configuration
    Management
    Delivery Feedback
    Send automatic

    feedback
    4.1
    User
    Forward email
    3.3
    Send user-generated feedback
    4.3
    Download app
    3.4

    View full-size slide

  8. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    1) Example of tailored workflow for a larger project
    8
    Developer
    Build
    2.1
    Upload app
    3.1
    Test
    2.2
    Delivery Service User
    Download app
    3.4
    Send email
    3.2
    Integration System
    Repository
    Push

    changes
    1.1
    Package
    2.3
    Collect

    metrics
    2.4
    Issue Tracker
    Observe changes
    1.5
    Resolve

    dependencies
    1.4 Send automatic

    feedback
    4.1
    Forward email
    3.3
    Update status
    4.2
    Release
    Manager
    Trigger build
    1.2
    Email Server
    Fetch changes
    1.3
    Notify

    about build
    2.5
    Integration
    Configuration
    Management
    Delivery Feedback
    manual build only automatic
    feedback
    automatic
    distribution

    View full-size slide

  9. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    2) Example of tailored workflow for a smaller project
    9
    Developer
    Build 2.1
    Test 2.2
    Delivery Service
    Integration System
    Repository
    Push

    changes
    1.1
    Trigger build
    1.2
    Fetch changes
    1.3
    Upload app 3.1
    Package 2.3
    Send user-generated feedback
    4.3
    Issue Tracker
    Forward email
    3.3
    Update

    status
    4.4
    Release Manager
    Send email
    3.2
    User
    Download app
    3.4
    Download app
    3.0
    Integration
    Configuration
    Management
    Delivery Feedback
    no dependency
    management
    manual
    distribution
    only user-
    generated
    feedback
    no metrics

    View full-size slide

  10. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Evaluation
    10

    View full-size slide

  11. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Research Questions
    1) Integration: How are projects building and testing changes and how often and timely
    are they doing it?
    2) Testing: Which types of tests are projects employing and how are these tests run
    respectively?
    3) Metrics: Which metrics are collected, how are they collected and how is the
    information used?
    4) Delivery: How are projects delivering new builds to users, how much time and effort
    does it cost and how many users can be reached?
    5) Feedback: Which feedback channels are used, how frequently is feedback collected
    and what is the quality of feedback?
    11

    View full-size slide

  12. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Study Design
    1) Investigation of 8 projects developing mobile applications (mostly iOS and Android) at
    Capgemini with customers from different industrial sectors
    2) Introduction of Rugby’s extended and tailorable release management workflow
    3) Conduction of personal interviews about the use of the workflow
    • Interviewees with multiple years of experience in the mobile domain
    • Familiar with agile methodologies and continuous integration
    • Little experience with automated delivery
    12

    View full-size slide

  13. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Findings: Integration
    • More projects use integration systems
    (88% instead of 25% before)
    • Frequency of integration increased
    • More projects use dependency
    management (75% instead of 13% before)
    ➡Reduced time for integration and
    immediate feedback about the build state
    13
    38%$
    13%$
    38%$
    88%$
    25%$
    0%$
    10%$
    20%$
    30%$
    40%$
    50%$
    60%$
    70%$
    80%$
    90%$
    100%$
    Before$ After$
    Immediately$ Regularly$ Sporadically$

    View full-size slide

  14. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Findings: Testing
    • Effort and implementation of tests
    remain unchanged
    • Automation of test cases greatly
    increased
    ➡High time savings and immediate
    visibility of regression failures
    • Acceptance, usability and exploratory
    test execution remain manually
    14
    13%$
    38%$ 38%$
    13%$
    13%$
    38%$
    13%$
    38%$
    13%$
    38%$
    50%$
    63%$
    13%$
    75%$
    38%$
    0%$
    10%$
    20%$
    30%$
    40%$
    50%$
    60%$
    70%$
    80%$
    90%$
    100%$
    Before$After$ Before$After$ Before$After$
    Unit$ Integration$ System$
    Automated$ Semi@automated$ Manual$

    View full-size slide

  15. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Findings: Metrics
    Majority of projects makes use of automatic metrics:
    • Static analysis tools provide codebase metrics
    like code quality and test coverage
    ➡ Continuous quality assurance
    • The integration system reports build success
    rate and duration of build and test steps
    ➡ Status monitoring
    • Project metrics (e.g. velocity, cycle time) are
    obtained from the issue tracker
    ➡Decision support
    15
    8%#
    65%#
    75%#
    38%#
    13%#
    5%#
    6%# 8%#
    5%#
    0%#
    10%#
    20%#
    30%#
    40%#
    50%#
    60%#
    70%#
    80%#
    90%#
    100%#
    Before#After# Before#After# Before#After#
    Codebase# Integration# Project#
    Automated# SemiCautomated# Manual#

    View full-size slide

  16. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Findings: Delivery
    • About 60% use automated delivery, about
    40% semi-automated delivery (before 0%)
    • Involvement of team members reduced by
    25% (median)
    • Delivery time reduced to 5 min (median)
    ➡High time savings and immediate
    availability of changes in releases
    16
    43% 38%
    57%
    50%
    100% 100%
    13%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    70%
    80%
    90%
    100%
    Before After Before After
    Internal External
    Automated Semi>automated Manual

    View full-size slide

  17. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Findings: Feedback
    • The feedback collection did not significantly change
    ➡Unstructured feedback channels are preferred
    • The quality of the feedback did not significantly change
    ➡Synchronous communication of feedback has the highest quality
    17
    88% 88%
    13%
    50%
    25%
    13%
    38%
    13%
    13%
    25%
    25%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    70%
    80%
    90%
    100%
    Phone Email Virtual:
    meetings
    Real:meetings Custom:tools Dedicated:
    tools
    At:least:weekly At:least:monthly Less:than:monthly
    25%
    38%
    25%
    50%
    13%
    38%
    75%
    13%
    13%
    38%
    13%
    25%
    25%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    70%
    80%
    90%
    100%
    Phone Email Virtual:
    meetings
    Real:meetings Custom:tools Dedicated:
    tools
    Excellent Good Acceptable Insufficient Useless

    View full-size slide

  18. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Threats to Validity
    • Reliability: Duration of the evaluation period, number of metrics, or level of detail may
    have influence on the reliability of our results
    • Generalizability: Number of projects and variation of project characteristics may be
    too low in order to achieve generalizable results
    • Selection bias: Projects participating in our survey may have already worked in an
    agile fashion and had interest in automated integration and delivery
    • Researcher bias: An appreciation for agile in general or our solution in particular and
    positive results of our previous study may have influenced our evaluation
    18

    View full-size slide

  19. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Conclusion
    • We showed how to introduce continuous delivery in a corporate environment where no
    standardized workflow for mobile projects existed before
    • Duration of integration and delivery decreased while their frequency increased
    • Automated testing and metrics collection improved
    • Due to the close evaluation after the introduction of the workflows we could not find
    significant changes in the feedback collection
    ➡The tailorable workflow provides a good compromise between technical promises
    and actual business needs
    ➡The value of the introduced workflow is appreciated, especially low time and effort
    required for setup that allows projects to get started quickly
    19

    View full-size slide

  20. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    Outlook
    • Rugby’s components have the potential for highly complex and individual setups, e.g.
    • Finer control over the build process with build matrices and branch filters
    • Automated user interface tests
    • Integration tests with a backend built in parallel
    • Longer observation period in the evaluation
    • Improve the incorporation of user needs
    • Evaluate whether development behavior and feedback change over time
    20

    View full-size slide

  21. 21
    Introducing Continuous Delivery of Mobile Apps

    in a Corporate Environment: A Case Study
    Thank you!
    Sebastian Klepper, Stephan Krusche, Sebastian Peters, Bernd Bruegge and Lukas Alperowitz
    Technische Universität München, Chair for Applied Software Engineering
    RCoSE 2015
    Stephan Krusche
    Twitter: @skrusche
    Email: [email protected]
    Web: www.skrusche.de

    View full-size slide

  22. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    References (1)
    (1) M. Fitzgerald, N. Kruschwitz, D. Bonnet, and M. Welch, “Embracing digital technology: A new strategic imperative,” MIT Sloan Management Review
    in collaboration with Capgemini Consulting, 2013.
    (2) S. Khalaf, “Flurry analytics insights,” 2015, retrieved February 26, 2015 from http://www.flurry.com/blog/flurry-insights.
    (3) S. Krusche and B. Bruegge, “User feedback in mobile development,” in Proceedings of the 2nd International Workshop on Mobile Development
    Lifecycle. ACM, 2014, pp. 25–26.
    (4) D. Pagano and W. Maalej, “User feedback in the appstore: An empirical study,” in Proceedings of the 21st International Requirements Engineering
    Conference. IEEE, 2013, pp. 125–134.
    (5) D. Pagano and B. Bruegge, “User involvement in software evolution practice: a case study,” in Proceedings of the 2013 international conference on
    Software engineering. IEEE, 2013, pp. 953–962.
    (6) K. Schwaber and M. Beedle, Agile software development with Scrum. Prentice Hall, 2002.
    (7) VersionOne, “8th annual state of agile survey,” 2014, http://www.versionone.com/pdf/2013-state-of-agile-survey.pdf.
    (8) R. Pichler, Agile product management with scrum: Creating products that customers love. Addison-Wesley Professional, 2010.
    (9) J. Bosch, “Continuous software engineering: An introduction,” in Continuous Software Engineering. Springer, 2014, pp. 3–13.
    (10) J. Humble and D. Farley, Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison Wesley, 2010.
    (11) C. Handy, Understanding organizations: managing differentiation and integration. Oxford University Press, 1993.
    (12) W. Brown, H. McCormick, T. Mowbray, and R. Malveau, AntiPatterns: refactoring software, architectures, and projects in crisis. Wiley, 1998.
    (13) O. Pedreira, M. Piattini, M. Luaces, and N. Brisaboa, “A systematic review of software process tailoring,” SIGSOFT SE Notes, vol. 32, no. 3, pp. 1–6,
    2007.
    (14) S. Krusche, L. Alperowitz, B. Bruegge, and M. Wagner, “Rugby: An Agile Process Model Based on Continuous Delivery,” in Proceedings of the 1st
    International Workshop on RCoSE. ACM, 2014, pp. 42–50.
    (15) G. Kalus and M. Kuhrmann, “Criteria for software process tailoring: A systematic review,” in Proceedings of ICSSP. ACM, 2013, pp. 171–180.
    22

    View full-size slide

  23. RCoSE, ICSE’15 - Sebastian Klepper, Stephan Krusche et al. - Introducing Continuous Delivery of Mobile Apps in a Corporate Environment: A Case Study
    References (2)
    (16) I. Jacobson, G. Booch, J. Rumbaugh, J. Rumbaugh, and G. Booch, The unified software development process. Addison-Wesley, 1999.
    (17) B. Bruegge, S. Krusche, and M. Wagner, “Teaching Tornado: from communication models to releases” in Proceedings of the 8th edition of
    the Educators’ Symposium. ACM, 2012, pp. 5–12.
    (18) J. Feller, Perspectives on free and open source software. MIT, 2005.
    (19) L. Zhao and S. Elbaum, “A survey on quality related activities in open source,” SIGSOFT Software Engineering Notes, vol. 25, no. 3, pp. 54–
    57, 2000.
    (20) M. Aberdour, “Achieving quality in open-source software,” Software, vol. 24, no. 1, pp. 58–64, 2007.
    (21) S. Krusche and L. Alperowitz, “Introduction of Continuous Delivery in Multi-Customer Project Courses,” in Proceedings of the 36th ICSE.
    IEEE, 2014, pp. 335–343.
    (22) M. Poppendiek and T. Poppendiek, Implementing Lean Software Development: From Concept to Cash. Addison Wesley, 2006.
    (23) L. Van Audenhove, “Expert interviews and interview techniques for policy analysis,” Vrije Universiteit Brussel, 2013.
    (24) P. Duvall, S. Matyas, and A. Glover, Continuous integration: improving software quality and reducing risk. Pearson Education, 2007.
    (25) L. Crispin and J. Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams. Addison Wesley, 2009.
    (26) B.Marick,“Agile testing directions: tests and examples,”2003,retrieved February 26, 2015 from http://www.exampler.com/old-blog/2003/08/22.
    (27) N. Fenton and J. Bieman, Software metrics: a rigorous and practical approach, 3rd ed. CRC Press, 2014.
    (28) M. Lorenz and J. Kidd, Object-oriented software metrics: a practical guide. Prentice-Hall, 1994.
    (29) M. Poppendiek and T. Poppendiek, Lean Software Development: An Agile Toolkit. Addison Wesley, 2003.
    (30) D. Willimack et al., “Evolution and adaption of questionnaire development, evaluation, and testing methods for establishment surveys,”
    Methods for Testing and Evaluating Questionnaires, pp. 385–407, 2004.
    (31) P. Gorans and P. Kruchten, “A guide to critical success factors in agile delivery,” IBM Center for The Business of Government, 2014.
    23

    View full-size slide