Slide 1

Slide 1 text

Starting an Ops Team @emmajanehw Emma Jane Hogbin Westby www.unocha.org

Slide 2

Slide 2 text

Agenda 0. Origin story Lessons Learned 1. Building inclusive teams 2. Starting a community of practice 3. Growing your tech with your team

Slide 3

Slide 3 text

Origin story

Slide 4

Slide 4 text

Humanitarian Programme Cycle https://www.humanitarianresponse.info/en/programme-cycle/space

Slide 5

Slide 5 text

Humanitarian Programme Cycle https://interagencystandingcommittee.org/system/files/hpc_reference_module_2015_final_.pdf

Slide 6

Slide 6 text

http://www.unocha.org/what-we-do/coordination-tools/cluster-coordination

Slide 7

Slide 7 text

(Most of) the Teams We Support

Slide 8

Slide 8 text

Pre 2014 Fast-moving, disparate teams. Duplicate costs for hosting. No infrastructure support for: - very large (high traffic) sites;
 - for small sites. Origin Story

Slide 9

Slide 9 text

Circa 2014 All web properties consolidated into a single infrastructure. Opted to use a managed hosting company already in use by OCHA with containers running on racked hardware. Origin Story

Slide 10

Slide 10 text

Circa 2014 All web properties consolidated into a single infrastructure. Opted to use a managed hosting company already in use by OCHA with containers running on racked hardware. Origin Story we’ll come back to this later

Slide 11

Slide 11 text

Transition Ops support was handled by - managed hosting staff
 - third-party ops consulting company with the intention of bringing this support in-house 
 “at some point”. Origin Story

Slide 12

Slide 12 text

Genesis of the Ops Team (2014) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). 
 Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story FTE = Full Time Equivalent

Slide 13

Slide 13 text

Genesis of the Ops Team (March 2016) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). 
 Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story That’s us! FTE = Full Time Equivalent

Slide 14

Slide 14 text

Genesis of the Ops Team (June 2016) OCHA Ops Team 0.5 FTE ops
 from one of the existing dev teams 0.5 FTE ops
 specialist Drupal ops in Australia 1 FTE coordination / project management
 front end developer turned dev team project manager turned ops maven (that's me). Phase2 Ops Team 0.2 FTE ops
 consulting firm 0.1 FTE coordination / project management
 experienced ops PM BlackMesh Ops Team as-needed ops support
 managed hosting company 0.1 FTE coordination / project management
 experienced ops PM Origin Story That’s us now!

Slide 15

Slide 15 text

Timeline Consolidation Understanding Building Trust Disparate teams Origin Story < 2014 2014 - 2015 2016

Slide 16

Slide 16 text

Building an inclusive team myself my team my community of practice

Slide 17

Slide 17 text

No matter your rank or role you are the start of an inclusive team. community of practice team self

Slide 18

Slide 18 text

Interlude The moment you’re asked to
 “show progress” is the moment you realise you have no metrics.

Slide 19

Slide 19 text

Resource Rich Archbold, Intercom.io Leading a Team with Values https://www.youtube.com/watch?v=vb8uuMSb55E

Slide 20

Slide 20 text

version 1.0 of my values- based metrics world premiere!

Slide 21

Slide 21 text

Knowing my values significantly reduced my option paralysis Set boundaries: When devs have root access to a shared service what’s “allowed”; developing an on-call schedule Define roles: according to the capacity of each team, what’s “ops” and what’s “dev”? Collaborate on special projects: ops pair with developers on configuration improvements no dev wanted to be part of the on-call roster

Slide 22

Slide 22 text

Building an inclusive team community of practice team self

Slide 23

Slide 23 text

Lesson 1: I thought I was being consistent, but as I grew,
 I began valuing different attributes and skills.

Slide 24

Slide 24 text

Question: How will you develop and nurture today the skills you will need three months from now?

Slide 25

Slide 25 text

Lesson 2: The skills I nurtured on my team three months ago are not necessarily the skills I need today.

Slide 26

Slide 26 text

Question: What action(s) can you take to ensure you have a diverse team who will help your team grow as the situation changes.

Slide 27

Slide 27 text

Starting a community of practice community of practice team self

Slide 28

Slide 28 text

define: community of practice A community of practice is a group of people who share a concern or a passion for something they do, and learn how to do it better as they interact regularly. This definition reflects the fundamentally social nature of human learning. It is very broad. http://wenger-trayner.com/resources/what-is-a-community-of-practice/

Slide 29

Slide 29 text

Establishing Communication Channels (biweekly) Community of Practice calls Flowdock discussion / chat channels Documentation on how to report an ops problem Weekly newsletter Incident reports Performance investigations Communities of Practice

Slide 30

Slide 30 text

Growing the practice The practice:
 their interactions produce resources that affect their practice (whether they engage in actual practice together or separately) http://wenger-trayner.com/resources/what-is-a-community-of-practice/

Slide 31

Slide 31 text

Resources Produced by the Community Tools Flowdock discussion / chat channels “Brown bag” calls Shared documentation Process Established procedures on how to work with ops (e.g., reporting an incident) Artefacts Weekly newsletter Incident reports Infra / app performance investigations Communities of Practice

Slide 32

Slide 32 text

The development stages of a CoP http://tacitlondon.com/publications/community-of-practice-maturity-model/ we’re about here

Slide 33

Slide 33 text

Timeline Consolidation Understanding Building Trust Disparate teams Origin Story < 2014 2014 - 2015 2016 All of the communication tools I was using to improve shared understanding
 helped to make our team more consistent and we began to establish trust.

Slide 34

Slide 34 text

Tuckman’s stages of group development Consolidation Understanding Building Trust Disparate teams Building Teams

Slide 35

Slide 35 text

Tuckman’s stages of group development Consolidation Understanding Building Trust Disparate teams Storm Form Norm Perform Building Teams

Slide 36

Slide 36 text

Growing your tech with your team

Slide 37

Slide 37 text

Understanding what we had inherited CentOS 6 + Docker 1.6 + GlusterFS 3 + Insufficient RAM Sadness 1.0 Growing your tech

Slide 38

Slide 38 text

Understanding what we had inherited CentOS 6 + Docker 1.6 + GlusterFS 3 + Insufficient RAM Sadness 1.0 Growing your tech on three peers

Slide 39

Slide 39 text

Interlude You get better at the thing you practice. I’m much better at “incidents” than I would like to be.

Slide 40

Slide 40 text

Resource Scott Klein, StatusPage.io
 Effective Incident Communication https://www.youtube.com/watch?v=ySSdqfZlC7Y

Slide 41

Slide 41 text

It wasn’t actually a bad decision.

Slide 42

Slide 42 text

But it’s time to move on.

Slide 43

Slide 43 text

Lesson 3: Having the wrong technology today, doesn’t mean you made a bad decision.

Slide 44

Slide 44 text

Question: How will you establish the trust within your team, and community of practice, to be able to iterate and improve your infrastructure?

Slide 45

Slide 45 text

Summary 0. Response to humanitarian crisis is publicly documented, and ever-evolving. 1. If you don’t know where to start in building an inclusive team,
 start by defining your values. 2. A strong, inclusive community of practice doesn't happen on its own. Transparency and trust are essential. 3. Your infrastructure is used to deploy applications;
 without the trust of the developers, you cannot iterate and improve.

Slide 46

Slide 46 text

I’m hiring! If you are, or know, an API developer who would be interested in working on an amazing (distributed) team. Please reach out!

Slide 47

Slide 47 text