Starting an Ops Team @emmajanehw Emma Jane Hogbin Westby www.unocha.org

Agenda 0. Origin story Lessons Learned 1. Building inclusive teams 2. Starting a community of practice 3. Growing your tech with your team

Origin story

Humanitarian Programme Cycle https://www.humanitarianresponse.info/en/programme-cycle/space

Humanitarian Programme Cycle https://interagencystandingcommittee.org/system/files/hpc_reference_module_2015_final_.pdf

http://www.unocha.org/what-we-do/coordination-tools/cluster-coordination

(Most of) the Teams We Support

Pre 2014 Fast-moving, disparate teams. Duplicate costs for hosting. No infrastructure support for: - very large (high traffic) sites;
 - for small sites. Origin Story

Circa 2014 All web properties consolidated into a single infrastructure. Opted to use a managed hosting company already in use by OCHA with containers running on racked hardware. Origin Story

Circa 2014 All web properties consolidated into a single infrastructure. Opted to use a managed hosting company already in use by OCHA with containers running on racked hardware. Origin Story we’ll come back to this later

Transition Ops support was handled by - managed hosting staff
 - third-party ops consulting company with the intention of bringing this support in-house 
 “at some point”. Origin Story

Genesis of the Ops Team (2014) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). 
 Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story FTE = Full Time Equivalent

Genesis of the Ops Team (March 2016) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). 
 Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story That’s us! FTE = Full Time Equivalent

Genesis of the Ops Team (June 2016) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 0.5 FTE ops
 specialist Drupal ops in Australia 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story That’s us now!

Timeline Consolidation Understanding Building Trust Disparate teams Origin Story < 2014 2014 - 2015 2016

Building an inclusive team myself my team my community of practice

No matter your rank or role you are the start of an inclusive team. community of practice team self

Interlude The moment you’re asked to
 “show progress” is the moment you realise you have no metrics.

Resource Rich Archbold, Intercom.io Leading a Team with Values https://www.youtube.com/watch?v=vb8uuMSb55E

version 1.0 of my values- based metrics world premiere!

Knowing my values significantly reduced my option paralysis Set boundaries: When devs have root access to a shared service what’s “allowed”; developing an on-call schedule Define roles: according to the capacity of each team, what’s “ops” and what’s “dev”? Collaborate on special projects: ops pair with developers on configuration improvements no dev wanted to be part of the on-call roster

Building an inclusive team community of practice team self

Lesson 1: I thought I was being consistent, but as I grew,
 I began valuing different attributes and skills.

Question: How will you develop and nurture today the skills you will need three months from now?

Lesson 2: The skills I nurtured on my team three months ago are not necessarily the skills I need today.

Question: What action(s) can you take to ensure you have a diverse team who will help your team grow as the situation changes.

Starting a community of practice community of practice team self

define: community of practice A community of practice is a group of people who share a concern or a passion for something they do, and learn how to do it better as they interact regularly. This definition reflects the fundamentally social nature of human learning. It is very broad. http://wenger-trayner.com/resources/what-is-a-community-of-practice/

Establishing Communication Channels (biweekly) Community of Practice calls Flowdock discussion / chat channels Documentation on how to report an ops problem Weekly newsletter Incident reports Performance investigations Communities of Practice

Growing the practice The practice:
 their interactions produce resources that affect their practice (whether they engage in actual practice together or separately) http://wenger-trayner.com/resources/what-is-a-community-of-practice/

Resources Produced by the Community Tools Flowdock discussion / chat channels “Brown bag” calls Shared documentation Process Established procedures on how to work with ops (e.g., reporting an incident) Artefacts Weekly newsletter Incident reports Infra / app performance investigations Communities of Practice

The development stages of a CoP http://tacitlondon.com/publications/community-of-practice-maturity-model/ we’re about here

Timeline Consolidation Understanding Building Trust Disparate teams Origin Story < 2014 2014 - 2015 2016 All of the communication tools I was using to improve shared understanding
 helped to make our team more consistent and we began to establish trust.

Tuckman’s stages of group development Consolidation Understanding Building Trust Disparate teams Building Teams

Tuckman’s stages of group development Consolidation Understanding Building Trust Disparate teams Storm Form Norm Perform Building Teams

Growing your tech with your team

Understanding what we had inherited CentOS 6 + Docker 1.6 + GlusterFS 3 + Insufficient RAM Sadness 1.0 Growing your tech

Understanding what we had inherited CentOS 6 + Docker 1.6 + GlusterFS 3 + Insufficient RAM Sadness 1.0 Growing your tech on three peers

Interlude You get better at the thing you practice. I’m much better at “incidents” than I would like to be.

Resource Scott Klein, StatusPage.io
 Effective Incident Communication https://www.youtube.com/watch?v=ySSdqfZlC7Y

It wasn’t actually a bad decision.

But it’s time to move on.

Lesson 3: Having the wrong technology today, doesn’t mean you made a bad decision.

Question: How will you establish the trust within your team, and community of practice, to be able to iterate and improve your infrastructure?

Summary 0. Response to humanitarian crisis is publicly documented, and ever-evolving. 1. If you don’t know where to start in building an inclusive team,
 start by defining your values. 2. A strong, inclusive community of practice doesn't happen on its own. Transparency and trust are essential. 3. Your infrastructure is used to deploy applications;
 without the trust of the developers, you cannot iterate and improve.

I’m hiring! If you are, or know, an API developer who would be interested in working on an amazing (distributed) team. Please reach out!

emma@humanitarianresponse.info