The New Hybrid Future of Software Engineering

Keynote at ESEM 2022 in Helsinki, Finland.

The COVID-19 pandemic shook the world to its core and provoked an overnight exodus of software developers who normally worked in an office setting to working from home. In this talk, I will present the results of several studies at Microsoft on how software developers and their teams have been affected by this change. I will discuss benefits and challenges of remote work and the impact of productivity during the pandemic. I will conclude with research opportunities and implications for the new hybrid work in software engineering as many developers are returning to work in offices.

https://conf.researchr.org/home/esem-2022

Transcript

1. The New
   Hybrid Future of
   Software
   Engineering
   Thomas Zimmermann
   #ESEM2022
   

   View Slide

2. 2
   «Software is eating
   the world.»
   Image: Pixels Movie
   

   View Slide

3. The SPACE of developer productivity
   3
   Make higher-quality
   software
   Produce software
   more efficiently
   Make smarter
   planning decisions
   Improve
   the well-being and
   satisfaction of
   developers
   Facilitate team
   collaboration
   and success
   Transition to a
   fully remote
   working world
   Why developer
   productivity
   matters
   Learning how to define, measure,
   and predict developer productivity
   can help organizations:
   

   View Slide

4. The SPACE of developer productivity
   4
   Productivity is all about
   developer activity
   You can’t improve what you don’t
   understand. Here are the common
   myths and misconceptions around
   developer productivity:
   Myths and
   misconceptions
   about developer
   productivity
   01
   Productivity is only about
   individual performance
   02
   One productivity metric
   can tell you everything
   03
   Productivity measures are
   only useful for managers
   04
   Productivity comes down
   to engineering systems
   and developer tools
   05
   

   View Slide

5. The SPACE of developer productivity
   5
   SPACE: A
   framework for
   understanding
   developer
   productivity
   The SPACE framework offers a
   new way of thinking about
   developer productivity.
   https://aka.ms/SpaceFramework
   ACMQUEUE – March 2021
   

   View Slide

6. The SPACE of developer productivity
   6
   SPACE: A
   framework for
   understanding
   developer
   productivity
   The SPACE framework offers a
   new way of thinking about
   developer productivity.
   https://aka.ms/SpaceFramework
   Satisfaction and well-being
   Performance
   Activity
   Communication and collaboration
   Efficiency and flow
   

   View Slide

7. The SPACE of developer productivity
   7
   SPACE framework in action
   Individual
   One person
   Team or group
   People that
   work together
   Level
   System
   End-to-end
   work through a
   system (like a
   development
   pipeline)
   Satisfaction & Well-being
   How fulfilled, happy, and
   healthy one it
   Developer satisfaction
   Retention*
   Satisfaction with code
   reviews assigned
   Perception of code reviews
   Developer satisfaction
   Retention*
   Satisfaction with
   engineering system
   (e.g., CI/CD pipeline)
   Performance
   An outcome of
   a process
   Code review
   velocity
   Code review
   velocity
   Story points
   shipped*
   Code review velocity
   Code review
   (acceptance rate)
   Customer satisfaction
   Reliability (uptime)
   Activity
   The count of
   actions or outputs
   Number of code
   reviews completed
   Coding time
   # commits
   Lines of code*
   # story points
   completed*
   Frequency of
   deployments
   Communication &
   collaboration
   How people talk and
   work together
   Code review score (quality or
   thoughtfulness)
   PR merge times
   Quality of meetings*
   Knowledge sharing, discoverability
   (quality of documentation)
   Efficiency & flow
   Doing work with minimal
   delays or interruptions
   Code review timing
   Productivity perception
   Lack of interruptions
   PR merge times
   Quality of meetings*
   Knowledge sharing, discoverability
   (quality of documentation)
   Code review timing
   Handoffs
   Knowledge sharing, discoverability
   (quality of documentation)
   Code review timing
   Velocity/flow
   through the system
   * Use these metrics with (even more) caution – they can proxy more things.
   

   View Slide

8. Impact of
   work from home
   on software
   engineers at
   Microsoft (*)
   (*) during the pandemic
   

   View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. Work from Home
    Denae Ford, Margaret-Anne D. Storey, Thomas
    Zimmermann, Christian Bird, Sonia Jaffe, Chandra
    Shekhar Maddila, Jenna L. Butler, Brian Houck,
    Nachiappan Nagappan: A Tale of Two Cities: Software
    Developers Working from Home During the COVID-19
    Pandemic. ACM TOSEM (2021)
    https://arxiv.org/abs/2008.11147
    

    View Slide

15. Multi-survey study
    How has productivity changed? Why?
    What is good about working from home?
    What is bad about working from home?
    What challenges have you encountered
    working from home?
    How has communication with your team
    members changed since working from home?
    How do you stay socially engaged with your
    colleagues (in place of hallway conversation)?
    

    View Slide

16. Multi-survey study
    How has productivity changed? Why?
    What is good about working from home?
    What is bad about working from home?
    What challenges have you encountered
    working from home?
    How has communication with your team
    members changed since working from home?
    How do you stay socially engaged with your
    colleagues (in place of hallway conversation)?
    

    View Slide

17. Compared to working in office, how has your
    productivity changed? (N=2,254, April/May)
    11.18%
    25.60%
    31.72%
    25.51%
    5.99%
    Significantly more productive
    More productive
    About the same
    Less productive
    Significantly less productive
    Answer Percent
    Significantly less
    productive
    6.06%
    Less productive 31.79%
    About the same 31.79%
    More productive 22.06%
    Significantly more
    productive
    8.30%
    Previous survey:
    March 16-23, N=1,357
    

    View Slide

18. What is good about working from home?
    Workplace – In the current WFH environment respondents reported advantages including
    now having a window office, no longer being in an open-office space, and feeling safer.
    Well Being – Quality of a healthy lifestyle has increased (e.g., enabling ease of physical
    fitness and more healthy diets), they are taking more breaks, as well as reducing health
    risks from the crisis.
    Beyond Work – Respondents reported feeling as though they enabled at better work-life
    balance by having an increased proximity to family, enabled access to higher quality food,
    and the ability to manage home needs.
    Flexibility – Respondents described having more control over their tasks and schedule
    which in turn made them feel more productive.
    Efficient Interactions – With the increased uncertainty of daily needs, respondents have
    found it helpful to reduce spontaneous meetings, interruptions, and distractions.
    Convenience & Commute – Over half of respondents mentioned the reduced commute as
    a great benefit of now WFH. Other participants in this cluster described the convenient
    clothing of PJs that have made them feel more comfortable during times of work.
    

    View Slide

19. How frequent
    (Prevalence)
    and how important
    (Importance)
    are these benefits?
    How has productivity
    changed (Delta) when a
    benefit is present?
    Statistically significant differences
    indicated with (*)
    What work-related benefits have you experienced working from home?
    

    View Slide

20. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased.
    The explanatory variables are the direct effects and interactions for major benefits experiences and reported to be important.
    

    View Slide

21. What is bad about working from home? /
    What challenges have you encountered working from home?
    Communication and Coordination – Friction interacting with colleagues when working from home. This
    includes not being able to just drop in on an office, not being able to find them on teams, or not being able
    to have a rich in person conversation. Lack of ad-hoc meetings and social interactions. Respondents missed
    the Hallway and other ad-hoc impromptu conversations. Various meeting issues. Respondents mentioned
    challenges with having online meetings (people speaking over each other, not being able to "read" people
    when no video is in use, and not being able to brainstorm as easily as when in the same physical room.)
    Workplace Setup – The work environment at home is worse than the office environment. This includes no
    specific area to work (e.g., having to sit at a kitchen table or on a couch), temperature, noise, light, poor
    chair, etc. In some cases, the environment does not provide same ergonomic affordances as the work office
    (pain from poor chairs or uncomfortable work positions). Many people complained that they had to work on
    just their laptop, and they missed the power of their desktop setup or having multiple monitors.
    Hard to Focus, Distractions, and Interruptions – It is harder to maintain focus when working from home.
    Non-family distractions around the house (laundry, TV, Facebook, etc.) Distractions specifically related to
    family members or other people living in the same residence. Distractions that arise from work such as
    emails, Teams chats, etc.
    

    View Slide

22. What is bad about working from home? /
    What challenges have you encountered working from home?
    Connectivity – Working from home is hindered by poor Internet connection, SAW and VPN drops, and having
    to use remote desktops. Reboots/updates sometimes required employees drive to get machines working
    again.
    Work-life balance – Not being able to establish a clear boundary being work life and home life. This includes
    feeling "always on", putting in more hours for work so that there is less "balance", or switching back and
    forth between work and home responsibilities frequently.
    Lack of routine – Working different hours than normal, not having a consistent schedule. Staying in the same
    place for too long, forgetting to take a lunch or snack break, not moving between meetings. Missing
    convenience of food/coffee/drinks on campus. Annoyance of having to make one's own food.
    Physical and mental health – Feeling lonely or isolated. Feeling pressure to be as productive from home as in
    office, feeling stress/anxiety about job or current virus situation. Finding it more difficult to stay motivated to
    get work done. People miss exercise in the form of walking around campus, walking between office, or to the
    break room when working from home.
    None – Respondents explicitly mentioned no challenges and/or nothing bad about working from home.
    

    View Slide

23. How frequent
    (Prevalence) and
    how impactful (Impact)
    are these challenges?
    How has productivity
    changed (Delta) when a
    challenge is present?
    Statistically significant differences
    indicated with (*)
    What challenges have you experienced working from home?
    

    View Slide

24. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased.
    The explanatory variables are the direct effects and interactions for major challenges.
    

    View Slide

25. What could be improved about working from home? (Choose up to three items.)
    

    View Slide

26. work autonomy
    work environment
    meetings
    ability to focus
    work life balance
    childcare
    The Yin and Yang of Working from Home
    

    View Slide

27. Charles Dickens’ “A Tale of
    Two Cities” begins,
    “It was the best of times, it
    was the worst of times.”
    Adapting Dickens’ line to
    leading an engineering
    team during the global
    pandemic, I’d say “We’re
    doing very well, we’re
    barely hanging in there.”
    – Shane O’Flynn
    What It’s Like to Lead a 100-Person Engineering Team
    During the Pandemic (lohika.com)
    

    View Slide

28. Team Productivity
    Courtney Miller, Paige Rodeghero, Margaret-Anne D.
    Storey, Denae Ford, Thomas Zimmermann:
    "How Was Your Weekend?" Software Development
    Teams Working From Home During COVID-19.
    ICSE (2021). https://arxiv.org/abs/2101.05877
    

    View Slide

29. Team productivity
    Compared to working in office, how has
    the productivity of your team changed?
    3.37%
    19.58%
    53.77%
    21.51%
    1.77%
    Significantly more
    productive
    More productive
    About the same
    Less productive
    Significantly less
    productive
    70% say that the ability of the team to meet
    milestones has not changed. However, there
    are concerns that this may not be sustainable.
    As a team we are taking longer to
    meet milestones mostly due to
    people's availability changing with
    having to take care of family, mental
    health issues, or simply burning out
    from blurred work/home boundaries.
    I think that for certain roles which rely on in-person collaboration and
    also need to align people it's harder and takes more time. So even if the
    milestones are achieved, the workload increase in effort makes this
    unsustainable over time. We need to slow down and learn how to get
    as efficient and smart on doing 100% remote work.
    My feeling is that our reliance on email and
    individual contributors instead of teams have
    delayed our timeliness to resolve issues, and a tool
    like DrWhom is not enough to identify knowledgeable
    alternate points of contact.
    

    View Slide

30. View Slide

31. Compared to working in office, do you think the ability of
    the team to reach milestones has changed?
    We have lost somewhere between 20%-40% effectiveness
    in use of time. In order to keep up, people are working
    longer hours. We are starting to see burnout.
    (S1384)
    It is more difficult to land nuanced discussions.
    1:1 is OK, but in a multi-person discussion it is
    very, very hard to keep focus. (S1777)
    The time pressure due to child care expectations and
    more-frequent scheduled meetings (vs. quick hallway
    chats) has definitely slowed the team down. We have
    explicitly and officially postponed some work, citing
    Coronavirus-rooted challenges as reasons.
    (S887)
    Brainstorming activities that need whiteboard and involve
    more passionate conversation progresses slower than
    before and can be frustrating.
    (S1833)
    Trouble with collaboration
    and communication
    

    View Slide

32. How has your team culture changed?
    We used to have lunches together, and had good
    comeraderie. No more lunches together, less comeradie,
    and less incidental ‘water cooler’ information exchange.
    (S2049)
    We are trying to adapt to daily sync ups and face time
    between all team members. which helps with
    communication but adds into the million meetings.
    (S1722)
    More explicit efforts to facilitate social events
    and foster team relationships.
    (S1676)
    I feel no difference.
    (S1801)
    Change in Social Connections
    Change in Communication
    

    View Slide

33. How has your team supported you during this crisis?
    35
    We have weekly social hours, which helps with some of
    the social isolation I’ve been feeling.
    (S1562)
    Everyone is supportive of each other. It is easy to talk
    with someone if needed. Feel supported.
    (S1053)
    Lots of teams meetings.
    (S169)
    More meetings with our direct manager’s
    team just to talk and check in.
    (S1636)
    Focus on Social Connections
    Maintaining Communication
    

    View Slide

34. How has communication with your team members changed
    since working from home?
    Frequency:
    Much less communication for most, especially less
    informal/social communication, more meetings for some
    Tools: Heavy use of Teams, more video, more emails (some
    less), some miss whiteboards, more miss F2F interactions
    Challenges: Communication tool friction, feeling isolated
    (while a few mention they feel more connected)
    Adapting: Emerging new practices such as daily standups,
    specific Teams channels for topics/social interactions, and
    more intentional communication (can’t rely on drivebys)
    Effectiveness: Varies between same, worse, better
    "Less frequent. More
    intentional. You don't overhear
    as many team conversations
    unless those happen to be
    posted in a Teams channel.
    Sometimes there is an overload
    of too many communication
    channels: email, multiple
    Teams' teams, channels, chat
    groups, Slack for some external
    partnerships, GitHub issue
    discussions, Azure DevOps issue
    discussions
    

    View Slide

35. How has communication with your team changed since working from home?
    37
    

    View Slide

36. AIC Backwards Stepwise Regression
    Team culture factor
    Model with All Factors
    Team culture factor
    Team culture factor
    Team culture factor
    Final Reduced Model
    

    View Slide

37. Explaining Team Productivity
    Ability to make decisions
    as a team
    Impactful contributions
    I make to team
    Ability to brainstorm
    with team External team bonding
    activities
    Difficulty communicating
    with colleagues
    Less awareness of what
    colleagues are working on
    Start/end meetings with
    non-work talk
    Frequency of scheduled
    social activities
    Ability to brainstorm
    with team
    Team members are
    Supportive of new ideas
    

    View Slide

38. Recommendations for
    effective remote teams
    Build and maintain team culture.
    Include social activities as part of “work.”
    Be mindful of other people’s time.
    Actively work to be inclusive.
    Managers should stay aware of contributions.
    Managers should hold weekly 1:1 meetings.
    Hold regular team meetings.
    Create communication standards.
    

    View Slide

39. Onboarding
    Paige Rodeghero, Thomas Zimmermann,
    Brian Houck, Denae Ford: Please Turn Your Cameras
    On: Remote Onboarding of Software Developers
    during a Pandemic. ICSE SEIP (2021)
    https://arxiv.org/abs/2011.08130
    

    View Slide

40. Methodology
    Sent a detailed survey to
    FTEs who have recently
    onboarded
    Performed open coding
    on the data collected for
    a qualitative analysis
    Conducted a quantitative
    analysis on the closed
    question responses
    Conducted interviews
    with new hires and early-
    career employees
    43
    

    View Slide

41. Survey
    Participant
    Overview
    • 1000 emails sent for recruitment
    • MSFT FTE developers
    • North America
    • Start date had to be in 2020
    • 267 responses collected (26.7% response rate)
    • All participants were working at home
    at the time of the survey
    • Gender of participants
    • 57 women
    • 162 man
    • 1 non-binary/ gender diverse
    44
    

    View Slide

42. RQ1: Pandemic
    Onboarding Challenges
    What challenges have you faced while
    onboarding to your team and project?
    Communication and Collaboration
    Asking for Help
    Building Team Connections
    Finding Documentation
    Technical Issues
    Hardware and Permissions
    “Because everyone is
    working from home I
    don’t run into people
    randomly or walk by
    their office, which has
    led me to ask for less
    help than I usually
    would have because I
    don’t like asking small,
    half-thought out
    questions virtually.”
    (S106)
    “During meetings, most teammates
    keep the camera off. This makes
    collaboration and remote experience
    difficult.” (S228)
    “Onboarding remotely is
    definitely more challenging. The
    inability to truly interact with
    coworkers, the in-ability to really
    have traditional team building,
    and the disconnection from
    everyone contributes to feeling
    like my work is kind of isolated
    from everyone else. It also
    makes it harder to get a broader,
    high level understanding of the
    team and organization.” (S91)
    45
    

    View Slide

43. RQ2: Team
    Onboarding Efforts
    What has the team done to make you feel connected
    to the team?
    Frequent 1:1 meetings with manager
    Remote team meeting 1st day
    Checking in to prevent blockages
    Social virtual events
    Onboarding buddy
    Assigned mentor
    “The team has been extremely welcoming. My very first day we met
    over teams and each person introduced themselves and welcomed me
    personally. Several people have reached out to ask how my onboarding
    is going and offered to answer any questions I have. Several members of
    my team invited me to join them in a hackathon. Generally everyone has
    been incredibly helpful and patient when teaching me.” (S191)
    46
    

    View Slide

44. RQ3: Team Interaction
    with New Hires
    129 (58.1%) new hires reported 1:1 meetings daily
    Communication focused on work related topics
    72.9% of the communication lasted < 30 minutes
    43.4% of the communication was < than 15 minutes
    “Formally there are scheduled meetings - mostly
    with the whole team, but I do have a scheduled
    1:1 with my manager. There are informal chats
    and some email exchanges between me and
    individuals to help me.” (S18)
    47
    

    View Slide

45. RQ4: Social
    Connection to Team
    83.3% of the new hires felt connection to their team
    “Unfortunately I guess I
    haven’t taken active
    steps to do so since I feel
    like I’m spending al l of
    my time trying to ramp
    up and learn.” (S92)
    “I try to reach out to random team
    members. I’m soon going to ask my
    manager if people can start turning
    on their cameras so I can see facial
    expressions.” (S75)
    48
    

    View Slide

46. Recommendations for
    remote onboarding
    Promote communication & asking for help.
    Encourage teams to turn cameras on.
    Schedule frequent 1:1 meetings.
    Provide information about the organization.
    Emphasize team building.
    Assign an onboarding buddy.
    Assign an onboarding technical mentor.
    Support multiple onboarding speeds.
    Assign a simple first task.
    Provide up-to-date documentation.
    

    View Slide

47. Other research on
    the impact of
    work from home on
    software engineers
    (*an incomplete list)
    

    View Slide

48. “Software companies should focus on maximizing employee
    wellbeing and improving the ergonomics of employees’ home
    offices. Women, parents and disabled persons may require
    extra support.”
    Empir. Softw. Eng. 25(6): 4927-4961 (2020)
    

    View Slide

49. “We find that WFH has both positive and negative impacts
    on developer productivity in terms of different metrics,
    e.g., the number of builds/commits/code reviews.”
    Science China Information (2022)
    

    View Slide

50. “We identified that 74.1% of the participants consider
    their productivity remains good or excellent, and 84.5%
    feel motivated and have easy communication with their
    co-workers. Furthermore, the main factors influencing the
    productivity are external interruption, environment
    adaptation, and emotional issues.”
    SBES (2020)
    

    View Slide

51. “Overall, we conclude that
    working from home was
    per se not a significant
    challenge for software
    engineers.”
    Empir. Softw. Eng. (2021)
    ICSE SEIP (2021)
    

    View Slide

52. Key Findings: (1) Developer activity remains largely consistent or
    increased compared to last year. (2) Flux in GitHub issues likely reflect
    the shift to distributed work and local virus responses.
    (3) The cadence of work has changed.
    (4) Patterns of developer activity have implications for burnout.
    (5) People are collaborating more, especially on open-source projects.
    

    View Slide

53. Findings: (1) Behind the average “no change” lays a
    large variation of experiences. (2) Productive engineers
    benefit from a better organization of work, increased
    flexibility and focus. (3) Poor collaboration, emotional
    issues, distractions and poor equipment hinder
    productivity. (4) Average productivity in later months is
    higher than the early months of the pandemic.
    JSS (2022)
    

    View Slide

54. New Future of Work Symposium (2020)
    

    View Slide

55. CHI EA (2021)
    Identified people’s diverse strategies and
    reasons behind the decisions of where to work
    during the hybrid work phase.
    

    View Slide

56. http://aka.ms/newfutureofwork
    

    View Slide

57. The New Future of Work
    Hybrid
    

    View Slide

58. Everything
    Everywhere
    All at Once
    A 2022 American absurdist science fiction comedy-drama film written
    and directed by Daniel Kwan and Daniel Scheinert (collectively known
    as "Daniels"). It stars Michelle Yeoh as a Chinese American woman
    being audited by the IRS who discovers that she must connect with
    parallel universe versions of herself to prevent a powerful being from
    causing the destruction of the multiverse.
    

    View Slide

59. Enable everyone
    to work anywhere
    with anyone
    at anytime
    on any device
    

    View Slide

60. Enable everyone
    Broaden the participation of developers.
    How can we leverage AI to allow more people to contribute to software?
    

    View Slide

61. To work anywhere
    

    View Slide

62. With anyone
    Need for physical co-location less important. Teams will be more distributed.
    Inclusive communities as a tool to facilitate this work.
    Collaborators can be anywhere as well.
    At anytime
    Asynchronous collaboration
    On any device
    Desktop, laptop, phone
    

    View Slide

63. What are research opportunities for ESEM?
    • Empirical studies of the changing work practices.
    • Well-being, mental health, and burnout in software engineering.
    • How to design inclusive communities.
    • Leverage AI to improve software development productivity throughout the
    entire development lifecycle
    • Broaden participation in software development, for example, by leveraging
    AI and similar techniques.
    

    View Slide

64. View Slide