Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The New Hybrid Future of Software Engineering

The New Hybrid Future of Software Engineering

Keynote at ESEM 2022 in Helsinki, Finland.

The COVID-19 pandemic shook the world to its core and provoked an overnight exodus of software developers who normally worked in an office setting to working from home. In this talk, I will present the results of several studies at Microsoft on how software developers and their teams have been affected by this change. I will discuss benefits and challenges of remote work and the impact of productivity during the pandemic. I will conclude with research opportunities and implications for the new hybrid work in software engineering as many developers are returning to work in offices.

https://conf.researchr.org/home/esem-2022

Thomas Zimmermann

September 22, 2022
Tweet

More Decks by Thomas Zimmermann

Other Decks in Research

Transcript

  1. The New
    Hybrid Future of
    Software
    Engineering
    Thomas Zimmermann
    #ESEM2022

    View Slide

  2. 2
    «Software is eating
    the world.»
    Image: Pixels Movie

    View Slide

  3. The SPACE of developer productivity
    3
    Make higher-quality
    software
    Produce software
    more efficiently
    Make smarter
    planning decisions
    Improve
    the well-being and
    satisfaction of
    developers
    Facilitate team
    collaboration
    and success
    Transition to a
    fully remote
    working world
    Why developer
    productivity
    matters
    Learning how to define, measure,
    and predict developer productivity
    can help organizations:

    View Slide

  4. The SPACE of developer productivity
    4
    Productivity is all about
    developer activity
    You can’t improve what you don’t
    understand. Here are the common
    myths and misconceptions around
    developer productivity:
    Myths and
    misconceptions
    about developer
    productivity
    01
    Productivity is only about
    individual performance
    02
    One productivity metric
    can tell you everything
    03
    Productivity measures are
    only useful for managers
    04
    Productivity comes down
    to engineering systems
    and developer tools
    05

    View Slide

  5. The SPACE of developer productivity
    5
    SPACE: A
    framework for
    understanding
    developer
    productivity
    The SPACE framework offers a
    new way of thinking about
    developer productivity.
    https://aka.ms/SpaceFramework
    ACMQUEUE – March 2021

    View Slide

  6. The SPACE of developer productivity
    6
    SPACE: A
    framework for
    understanding
    developer
    productivity
    The SPACE framework offers a
    new way of thinking about
    developer productivity.
    https://aka.ms/SpaceFramework
    Satisfaction and well-being
    Performance
    Activity
    Communication and collaboration
    Efficiency and flow

    View Slide

  7. The SPACE of developer productivity
    7
    SPACE framework in action
    Individual
    One person
    Team or group
    People that
    work together
    Level
    System
    End-to-end
    work through a
    system (like a
    development
    pipeline)
    Satisfaction & Well-being
    How fulfilled, happy, and
    healthy one it
    Developer satisfaction
    Retention*
    Satisfaction with code
    reviews assigned
    Perception of code reviews
    Developer satisfaction
    Retention*
    Satisfaction with
    engineering system
    (e.g., CI/CD pipeline)
    Performance
    An outcome of
    a process
    Code review
    velocity
    Code review
    velocity
    Story points
    shipped*
    Code review velocity
    Code review
    (acceptance rate)
    Customer satisfaction
    Reliability (uptime)
    Activity
    The count of
    actions or outputs
    Number of code
    reviews completed
    Coding time
    # commits
    Lines of code*
    # story points
    completed*
    Frequency of
    deployments
    Communication &
    collaboration
    How people talk and
    work together
    Code review score (quality or
    thoughtfulness)
    PR merge times
    Quality of meetings*
    Knowledge sharing, discoverability
    (quality of documentation)
    Efficiency & flow
    Doing work with minimal
    delays or interruptions
    Code review timing
    Productivity perception
    Lack of interruptions
    PR merge times
    Quality of meetings*
    Knowledge sharing, discoverability
    (quality of documentation)
    Code review timing
    Handoffs
    Knowledge sharing, discoverability
    (quality of documentation)
    Code review timing
    Velocity/flow
    through the system
    * Use these metrics with (even more) caution – they can proxy more things.

    View Slide

  8. Impact of
    work from home
    on software
    engineers at
    Microsoft (*)
    (*) during the pandemic

    View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. Work from Home
    Denae Ford, Margaret-Anne D. Storey, Thomas
    Zimmermann, Christian Bird, Sonia Jaffe, Chandra
    Shekhar Maddila, Jenna L. Butler, Brian Houck,
    Nachiappan Nagappan: A Tale of Two Cities: Software
    Developers Working from Home During the COVID-19
    Pandemic. ACM TOSEM (2021)
    https://arxiv.org/abs/2008.11147

    View Slide

  15. Multi-survey study
    How has productivity changed? Why?
    What is good about working from home?
    What is bad about working from home?
    What challenges have you encountered
    working from home?
    How has communication with your team
    members changed since working from home?
    How do you stay socially engaged with your
    colleagues (in place of hallway conversation)?

    View Slide

  16. Multi-survey study
    How has productivity changed? Why?
    What is good about working from home?
    What is bad about working from home?
    What challenges have you encountered
    working from home?
    How has communication with your team
    members changed since working from home?
    How do you stay socially engaged with your
    colleagues (in place of hallway conversation)?

    View Slide

  17. Compared to working in office, how has your
    productivity changed? (N=2,254, April/May)
    11.18%
    25.60%
    31.72%
    25.51%
    5.99%
    Significantly more productive
    More productive
    About the same
    Less productive
    Significantly less productive
    Answer Percent
    Significantly less
    productive
    6.06%
    Less productive 31.79%
    About the same 31.79%
    More productive 22.06%
    Significantly more
    productive
    8.30%
    Previous survey:
    March 16-23, N=1,357

    View Slide

  18. What is good about working from home?
    Workplace – In the current WFH environment respondents reported advantages including
    now having a window office, no longer being in an open-office space, and feeling safer.
    Well Being – Quality of a healthy lifestyle has increased (e.g., enabling ease of physical
    fitness and more healthy diets), they are taking more breaks, as well as reducing health
    risks from the crisis.
    Beyond Work – Respondents reported feeling as though they enabled at better work-life
    balance by having an increased proximity to family, enabled access to higher quality food,
    and the ability to manage home needs.
    Flexibility – Respondents described having more control over their tasks and schedule
    which in turn made them feel more productive.
    Efficient Interactions – With the increased uncertainty of daily needs, respondents have
    found it helpful to reduce spontaneous meetings, interruptions, and distractions.
    Convenience & Commute – Over half of respondents mentioned the reduced commute as
    a great benefit of now WFH. Other participants in this cluster described the convenient
    clothing of PJs that have made them feel more comfortable during times of work.

    View Slide

  19. How frequent
    (Prevalence)
    and how important
    (Importance)
    are these benefits?
    How has productivity
    changed (Delta) when a
    benefit is present?
    Statistically significant differences
    indicated with (*)
    What work-related benefits have you experienced working from home?

    View Slide

  20. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased.
    The explanatory variables are the direct effects and interactions for major benefits experiences and reported to be important.

    View Slide

  21. What is bad about working from home? /
    What challenges have you encountered working from home?
    Communication and Coordination – Friction interacting with colleagues when working from home. This
    includes not being able to just drop in on an office, not being able to find them on teams, or not being able
    to have a rich in person conversation. Lack of ad-hoc meetings and social interactions. Respondents missed
    the Hallway and other ad-hoc impromptu conversations. Various meeting issues. Respondents mentioned
    challenges with having online meetings (people speaking over each other, not being able to "read" people
    when no video is in use, and not being able to brainstorm as easily as when in the same physical room.)
    Workplace Setup – The work environment at home is worse than the office environment. This includes no
    specific area to work (e.g., having to sit at a kitchen table or on a couch), temperature, noise, light, poor
    chair, etc. In some cases, the environment does not provide same ergonomic affordances as the work office
    (pain from poor chairs or uncomfortable work positions). Many people complained that they had to work on
    just their laptop, and they missed the power of their desktop setup or having multiple monitors.
    Hard to Focus, Distractions, and Interruptions – It is harder to maintain focus when working from home.
    Non-family distractions around the house (laundry, TV, Facebook, etc.) Distractions specifically related to
    family members or other people living in the same residence. Distractions that arise from work such as
    emails, Teams chats, etc.

    View Slide

  22. What is bad about working from home? /
    What challenges have you encountered working from home?
    Connectivity – Working from home is hindered by poor Internet connection, SAW and VPN drops, and having
    to use remote desktops. Reboots/updates sometimes required employees drive to get machines working
    again.
    Work-life balance – Not being able to establish a clear boundary being work life and home life. This includes
    feeling "always on", putting in more hours for work so that there is less "balance", or switching back and
    forth between work and home responsibilities frequently.
    Lack of routine – Working different hours than normal, not having a consistent schedule. Staying in the same
    place for too long, forgetting to take a lunch or snack break, not moving between meetings. Missing
    convenience of food/coffee/drinks on campus. Annoyance of having to make one's own food.
    Physical and mental health – Feeling lonely or isolated. Feeling pressure to be as productive from home as in
    office, feeling stress/anxiety about job or current virus situation. Finding it more difficult to stay motivated to
    get work done. People miss exercise in the form of walking around campus, walking between office, or to the
    break room when working from home.
    None – Respondents explicitly mentioned no challenges and/or nothing bad about working from home.

    View Slide

  23. How frequent
    (Prevalence) and
    how impactful (Impact)
    are these challenges?
    How has productivity
    changed (Delta) when a
    challenge is present?
    Statistically significant differences
    indicated with (*)
    What challenges have you experienced working from home?

    View Slide

  24. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased.
    The explanatory variables are the direct effects and interactions for major challenges.

    View Slide

  25. What could be improved about working from home? (Choose up to three items.)

    View Slide

  26. work autonomy
    work environment
    meetings
    ability to focus
    work life balance
    childcare
    The Yin and Yang of Working from Home

    View Slide

  27. Charles Dickens’ “A Tale of
    Two Cities” begins,
    “It was the best of times, it
    was the worst of times.”
    Adapting Dickens’ line to
    leading an engineering
    team during the global
    pandemic, I’d say “We’re
    doing very well, we’re
    barely hanging in there.”
    – Shane O’Flynn
    What It’s Like to Lead a 100-Person Engineering Team
    During the Pandemic (lohika.com)

    View Slide

  28. Team Productivity
    Courtney Miller, Paige Rodeghero, Margaret-Anne D.
    Storey, Denae Ford, Thomas Zimmermann:
    "How Was Your Weekend?" Software Development
    Teams Working From Home During COVID-19.
    ICSE (2021). https://arxiv.org/abs/2101.05877

    View Slide

  29. Team productivity
    Compared to working in office, how has
    the productivity of your team changed?
    3.37%
    19.58%
    53.77%
    21.51%
    1.77%
    Significantly more
    productive
    More productive
    About the same
    Less productive
    Significantly less
    productive
    70% say that the ability of the team to meet
    milestones has not changed. However, there
    are concerns that this may not be sustainable.
    As a team we are taking longer to
    meet milestones mostly due to
    people's availability changing with
    having to take care of family, mental
    health issues, or simply burning out
    from blurred work/home boundaries.
    I think that for certain roles which rely on in-person collaboration and
    also need to align people it's harder and takes more time. So even if the
    milestones are achieved, the workload increase in effort makes this
    unsustainable over time. We need to slow down and learn how to get
    as efficient and smart on doing 100% remote work.
    My feeling is that our reliance on email and
    individual contributors instead of teams have
    delayed our timeliness to resolve issues, and a tool
    like DrWhom is not enough to identify knowledgeable
    alternate points of contact.

    View Slide

  30. View Slide

  31. Compared to working in office, do you think the ability of
    the team to reach milestones has changed?
    We have lost somewhere between 20%-40% effectiveness
    in use of time. In order to keep up, people are working
    longer hours. We are starting to see burnout.
    (S1384)
    It is more difficult to land nuanced discussions.
    1:1 is OK, but in a multi-person discussion it is
    very, very hard to keep focus. (S1777)
    The time pressure due to child care expectations and
    more-frequent scheduled meetings (vs. quick hallway
    chats) has definitely slowed the team down. We have
    explicitly and officially postponed some work, citing
    Coronavirus-rooted challenges as reasons.
    (S887)
    Brainstorming activities that need whiteboard and involve
    more passionate conversation progresses slower than
    before and can be frustrating.
    (S1833)
    Trouble with collaboration
    and communication

    View Slide

  32. How has your team culture changed?
    We used to have lunches together, and had good
    comeraderie. No more lunches together, less comeradie,
    and less incidental ‘water cooler’ information exchange.
    (S2049)
    We are trying to adapt to daily sync ups and face time
    between all team members. which helps with
    communication but adds into the million meetings.
    (S1722)
    More explicit efforts to facilitate social events
    and foster team relationships.
    (S1676)
    I feel no difference.
    (S1801)
    Change in Social Connections
    Change in Communication

    View Slide

  33. How has your team supported you during this crisis?
    35
    We have weekly social hours, which helps with some of
    the social isolation I’ve been feeling.
    (S1562)
    Everyone is supportive of each other. It is easy to talk
    with someone if needed. Feel supported.
    (S1053)
    Lots of teams meetings.
    (S169)
    More meetings with our direct manager’s
    team just to talk and check in.
    (S1636)
    Focus on Social Connections
    Maintaining Communication

    View Slide

  34. How has communication with your team members changed
    since working from home?
    Frequency:
    Much less communication for most, especially less
    informal/social communication, more meetings for some
    Tools: Heavy use of Teams, more video, more emails (some
    less), some miss whiteboards, more miss F2F interactions
    Challenges: Communication tool friction, feeling isolated
    (while a few mention they feel more connected)
    Adapting: Emerging new practices such as daily standups,
    specific Teams channels for topics/social interactions, and
    more intentional communication (can’t rely on drivebys)
    Effectiveness: Varies between same, worse, better
    "Less frequent. More
    intentional. You don't overhear
    as many team conversations
    unless those happen to be
    posted in a Teams channel.
    Sometimes there is an overload
    of too many communication
    channels: email, multiple
    Teams' teams, channels, chat
    groups, Slack for some external
    partnerships, GitHub issue
    discussions, Azure DevOps issue
    discussions

    View Slide

  35. How has communication with your team changed since working from home?
    37

    View Slide

  36. AIC Backwards Stepwise Regression
    Team culture factor
    Model with All Factors
    Team culture factor
    Team culture factor
    Team culture factor
    Final Reduced Model

    View Slide

  37. Explaining Team Productivity
    Ability to make decisions
    as a team
    Impactful contributions
    I make to team
    Ability to brainstorm
    with team External team bonding
    activities
    Difficulty communicating
    with colleagues
    Less awareness of what
    colleagues are working on
    Start/end meetings with
    non-work talk
    Frequency of scheduled
    social activities
    Ability to brainstorm
    with team
    Team members are
    Supportive of new ideas

    View Slide

  38. Recommendations for
    effective remote teams
    Build and maintain team culture.
    Include social activities as part of “work.”
    Be mindful of other people’s time.
    Actively work to be inclusive.
    Managers should stay aware of contributions.
    Managers should hold weekly 1:1 meetings.
    Hold regular team meetings.
    Create communication standards.

    View Slide

  39. Onboarding
    Paige Rodeghero, Thomas Zimmermann,
    Brian Houck, Denae Ford: Please Turn Your Cameras
    On: Remote Onboarding of Software Developers
    during a Pandemic. ICSE SEIP (2021)
    https://arxiv.org/abs/2011.08130

    View Slide

  40. Methodology
    Sent a detailed survey to
    FTEs who have recently
    onboarded
    Performed open coding
    on the data collected for
    a qualitative analysis
    Conducted a quantitative
    analysis on the closed
    question responses
    Conducted interviews
    with new hires and early-
    career employees
    43

    View Slide

  41. Survey
    Participant
    Overview
    • 1000 emails sent for recruitment
    • MSFT FTE developers
    • North America
    • Start date had to be in 2020
    • 267 responses collected (26.7% response rate)
    • All participants were working at home
    at the time of the survey
    • Gender of participants
    • 57 women
    • 162 man
    • 1 non-binary/ gender diverse
    44

    View Slide

  42. RQ1: Pandemic
    Onboarding Challenges
    What challenges have you faced while
    onboarding to your team and project?
    Communication and Collaboration
    Asking for Help
    Building Team Connections
    Finding Documentation
    Technical Issues
    Hardware and Permissions
    “Because everyone is
    working from home I
    don’t run into people
    randomly or walk by
    their office, which has
    led me to ask for less
    help than I usually
    would have because I
    don’t like asking small,
    half-thought out
    questions virtually.”
    (S106)
    “During meetings, most teammates
    keep the camera off. This makes
    collaboration and remote experience
    difficult.” (S228)
    “Onboarding remotely is
    definitely more challenging. The
    inability to truly interact with
    coworkers, the in-ability to really
    have traditional team building,
    and the disconnection from
    everyone contributes to feeling
    like my work is kind of isolated
    from everyone else. It also
    makes it harder to get a broader,
    high level understanding of the
    team and organization.” (S91)
    45

    View Slide

  43. RQ2: Team
    Onboarding Efforts
    What has the team done to make you feel connected
    to the team?
    Frequent 1:1 meetings with manager
    Remote team meeting 1st day
    Checking in to prevent blockages
    Social virtual events
    Onboarding buddy
    Assigned mentor
    “The team has been extremely welcoming. My very first day we met
    over teams and each person introduced themselves and welcomed me
    personally. Several people have reached out to ask how my onboarding
    is going and offered to answer any questions I have. Several members of
    my team invited me to join them in a hackathon. Generally everyone has
    been incredibly helpful and patient when teaching me.” (S191)
    46

    View Slide

  44. RQ3: Team Interaction
    with New Hires
    129 (58.1%) new hires reported 1:1 meetings daily
    Communication focused on work related topics
    72.9% of the communication lasted < 30 minutes
    43.4% of the communication was < than 15 minutes
    “Formally there are scheduled meetings - mostly
    with the whole team, but I do have a scheduled
    1:1 with my manager. There are informal chats
    and some email exchanges between me and
    individuals to help me.” (S18)
    47

    View Slide

  45. RQ4: Social
    Connection to Team
    83.3% of the new hires felt connection to their team
    “Unfortunately I guess I
    haven’t taken active
    steps to do so since I feel
    like I’m spending al l of
    my time trying to ramp
    up and learn.” (S92)
    “I try to reach out to random team
    members. I’m soon going to ask my
    manager if people can start turning
    on their cameras so I can see facial
    expressions.” (S75)
    48

    View Slide

  46. Recommendations for
    remote onboarding
    Promote communication & asking for help.
    Encourage teams to turn cameras on.
    Schedule frequent 1:1 meetings.
    Provide information about the organization.
    Emphasize team building.
    Assign an onboarding buddy.
    Assign an onboarding technical mentor.
    Support multiple onboarding speeds.
    Assign a simple first task.
    Provide up-to-date documentation.

    View Slide

  47. Other research on
    the impact of
    work from home on
    software engineers
    (*an incomplete list)

    View Slide

  48. “Software companies should focus on maximizing employee
    wellbeing and improving the ergonomics of employees’ home
    offices. Women, parents and disabled persons may require
    extra support.”
    Empir. Softw. Eng. 25(6): 4927-4961 (2020)

    View Slide

  49. “We find that WFH has both positive and negative impacts
    on developer productivity in terms of different metrics,
    e.g., the number of builds/commits/code reviews.”
    Science China Information (2022)

    View Slide

  50. “We identified that 74.1% of the participants consider
    their productivity remains good or excellent, and 84.5%
    feel motivated and have easy communication with their
    co-workers. Furthermore, the main factors influencing the
    productivity are external interruption, environment
    adaptation, and emotional issues.”
    SBES (2020)

    View Slide

  51. “Overall, we conclude that
    working from home was
    per se not a significant
    challenge for software
    engineers.”
    Empir. Softw. Eng. (2021)
    ICSE SEIP (2021)

    View Slide

  52. Key Findings: (1) Developer activity remains largely consistent or
    increased compared to last year. (2) Flux in GitHub issues likely reflect
    the shift to distributed work and local virus responses.
    (3) The cadence of work has changed.
    (4) Patterns of developer activity have implications for burnout.
    (5) People are collaborating more, especially on open-source projects.

    View Slide

  53. Findings: (1) Behind the average “no change” lays a
    large variation of experiences. (2) Productive engineers
    benefit from a better organization of work, increased
    flexibility and focus. (3) Poor collaboration, emotional
    issues, distractions and poor equipment hinder
    productivity. (4) Average productivity in later months is
    higher than the early months of the pandemic.
    JSS (2022)

    View Slide

  54. New Future of Work Symposium (2020)

    View Slide

  55. CHI EA (2021)
    Identified people’s diverse strategies and
    reasons behind the decisions of where to work
    during the hybrid work phase.

    View Slide

  56. http://aka.ms/newfutureofwork

    View Slide

  57. The New Future of Work
    Hybrid

    View Slide

  58. Everything
    Everywhere
    All at Once
    A 2022 American absurdist science fiction comedy-drama film written
    and directed by Daniel Kwan and Daniel Scheinert (collectively known
    as "Daniels"). It stars Michelle Yeoh as a Chinese American woman
    being audited by the IRS who discovers that she must connect with
    parallel universe versions of herself to prevent a powerful being from
    causing the destruction of the multiverse.

    View Slide

  59. Enable everyone
    to work anywhere
    with anyone
    at anytime
    on any device

    View Slide

  60. Enable everyone
    Broaden the participation of developers.
    How can we leverage AI to allow more people to contribute to software?

    View Slide

  61. To work anywhere

    View Slide

  62. With anyone
    Need for physical co-location less important. Teams will be more distributed.
    Inclusive communities as a tool to facilitate this work.
    Collaborators can be anywhere as well.
    At anytime
    Asynchronous collaboration
    On any device
    Desktop, laptop, phone

    View Slide

  63. What are research opportunities for ESEM?
    • Empirical studies of the changing work practices.
    • Well-being, mental health, and burnout in software engineering.
    • How to design inclusive communities.
    • Leverage AI to improve software development productivity throughout the
    entire development lifecycle
    • Broaden participation in software development, for example, by leveraging
    AI and similar techniques.

    View Slide

  64. View Slide