Keeping Users at the Forefront While Scaling Services

Slide 1

Slide 1 text

Keeping Users at the Forefront While Scaling Services @bensheldon

Slide 2

Slide 2 text

“Building resilient and usable systems requires a balancing act between responding to the needs of technical systems and users. This session shares practices and principles that helped the GetCalFresh engineering team scale and improve their service. We'll discuss practical implementations of "Leave a Seam" (creating paths within a system for people talk to you) and daily and weekly operations rituals for keeping technical teams oriented towards meeting the needs of users and technical systems.” – Session Synopsis, 2018

Slide 3

Slide 3 text

1. Background 2. Values & Principles 3. Rituals & Practices 4. Revisit Principles 5. Questions “The idea behind Honest Practice methodology is to investigate the specific context in which a practice was used, not just to report the outcome – be it success or failure.” – Howie Fisher Honest Practice: How the Public Sector Can Look at Itself Outline

Slide 4

Slide 4 text

Background

Slide 5

Slide 5 text

About your presenter College of Public and Community Service

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

What is GetCalFresh? ● A service provided by Code for America that acts as a digital assister, available at: https://www.getcalfresh.org (demo here) ● It allows anyone to apply for CalFresh using a mobile phone, and guides them through the eligibility process using email, text, and live chat support ● Like an assister, it works on top of existing systems, translating required questions to plain language, and submitting them to CalWIN or C4Yourself

Slide 10

Slide 10 text

GetCalFresh focuses on... 1 2 3 Getting more people through the door Supporting clients through the eligibility process Ensuring clients don’t churn for administrative reasons* *currently offering an easier SAR-7 in three counties

Slide 11

Slide 11 text

GetCalFresh is working who we help apply ● 16,000 applications submitted monthly ● 33,000 people helped monthly ● 55% avg. approval rate across 33 counties ● 32% increase of avg. online applications ● 20% of all apps are submitted via GCF ● 60% are applying for CalFresh for the first time

Slide 12

Slide 12 text

GetCalFresh is people June 1, 2018 March 6, 2017 Director Engineer Design & Research Engineer Client Success Program & Partnerships Engineer Engineer Engineer Product Program & Partnerships Client Success Data Science Program & Partnerships

Slide 13

Slide 13 text

Values & Principles

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Operations

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

The right place at the right time... ● 2 different CalFresh application webforms ● Fax Machines ● Email ● “Secure” Email ● “Secure” Dropbox ● County-specific upload websites ● SFTP ● Lots of humans and their own operational systems and business processes. ...which is at the county office by 5pm that business day.

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

“Understand Meet User Needs First and foremost, we talk to users and understand who they are. We’ve found that much of the time the best way to get started is not to write any code, but instead to just help people, manually. Once we have a live service, we make sure to insert a “seam” such as live chat that allows us to talk to users regularly.

Slide 21

Slide 21 text

“Optimize for Speed of Learning Armed with a clear North Star and a plan to reach it, we can focus on possibly the most important part of our process — learning at the fastest rate possible. We’re big fans of the Lean Startup approach and we’re always trying to accelerate the speed at which we can get through Build-Measure-Learn loops.

Slide 22

Slide 22 text

“Build Visibility In ...we make sure we have good visibility into problems users may be having with our services. Quality monitoring is key when it comes to providing a high quality service to our customers.

Slide 23

Slide 23 text

Rituals & Practices

Slide 24

Slide 24 text

By the numbers Ben's Start Date March 6, 2017 Presentation Date June 1, 2018 Calendar Days 452 Business Days Excluding Holidays 313 Number of Weeks 60 Applications Assisted 131,513 Deploys 1,044 Issues / Stories Written 1,437 Daily Standups 246 Bi-Weekly Planning Meeting 30 Bi-Weekly Team Review and Retro 30 Bi-Weekly Engineering Retro 29 Morning Rituals 183 Daily Drives 273 Incident Postmortems 10 Days Client Success Coverage 29 Impact / Complexity meetings 4 Strategic Planning Onsites 1 Team Offsites 3

Slide 25

Slide 25 text

Actual growth Active in 9 counties Active in 33 counties

Slide 26

Slide 26 text

Meetings

Slide 27

Slide 27 text

Meetings Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building

Slide 28

Slide 28 text

March 6, 2017 - June 1, 2018 ➔ 1,044 production deploys ➔ 3.3 deploys per workday ➔ 1,437 new stories / issues Pace of development

Slide 29

Slide 29 text

30 Bi-Weekly Planning Meeting

Slide 30

Slide 30 text

183 Morning Rituals

Slide 31

Slide 31 text

Meetings - Daily Drive Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building

Slide 32

Slide 32 text

Slide 33

Slide 33 text

We call submittal “Driving” Automated Driving Manual Driver

Slide 34

Slide 34 text

Common “Driving” Problems ● Mailing Address Parsing ● Document Formats (bmp, gif, multi-age tiffs) ● Document Sizes (100 page leases), ● Spécial Cháråcters ● Webform updates ● Webform degradations ● Fax machine errors ● “Rare” eligibility cases

Slide 35

Slide 35 text

Actual growth Active in 9 counties Active in 33 counties

Slide 36

Slide 36 text

March 6, 2017 June 1, 2018 273 Daily Drives April 17, 2017

Slide 37

Slide 37 text

273 Daily Drives

Slide 38

Slide 38 text

273 Daily Drives

Slide 39

Slide 39 text

273 Daily Drives

Slide 40

Slide 40 text

273 Daily Drives

Slide 41

Slide 41 text

Meetings - Retro and Consensus Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building

Slide 42

Slide 42 text

30 + 29 = 59 Retros

Slide 43

Slide 43 text

29 Days on Client Success

Slide 44

Slide 44 text

10 Incident Postmortems

Slide 45

Slide 45 text

4 Impact / Complexity Matrices

Slide 46

Slide 46 text

1 Strategic Planning Onsite

Slide 47

Slide 47 text

3 Offsites

Slide 48

Slide 48 text

Revisit Principles

Slide 49

Slide 49 text

“Understand Meet User Needs ... just help people, manually... talk to users regularly… Examples: Daily Drive, Customer Success Shifts

Slide 50

Slide 50 text

“Optimize for Speed of Learning ... accelerate the speed at which we can get through Build-Measure-Learn loops... Examples: Morning Rituals, Development ⇔ Daily Drive, Retrospectives, Postmortems

Slide 51

Slide 51 text

“Build Visibility In ...have good visibility into problems users may be having with our services... Examples: Daily Drive, Morning Rituals, Consensus Building