Keeping Users at the Forefront While Scaling Services

Building resilient and usable systems requires a balancing act between responding to the needs of technical systems and users. This session shares practices and principles that helped the GetCalFresh engineering team scale and improve their service. We'll discuss practical implementations of "Leave a Seam" (creating paths within a system for people talk to you) and daily and weekly operations rituals for keeping technical teams oriented towards meeting the needs of users and technical systems.

Transcript

1. Keeping Users at the Forefront While Scaling Services @bensheldon
2. “Building resilient and usable systems requires a balancing act between responding to the needs of technical systems and users. This session shares practices and principles that helped the GetCalFresh engineering team scale and improve their service. We'll discuss practical implementations of "Leave a Seam" (creating paths within a system for people talk to you) and daily and weekly operations rituals for keeping technical teams oriented towards meeting the needs of users and technical systems.” – Session Synopsis, 2018
3. 1. Background 2. Values & Principles 3. Rituals & Practices 4. Revisit Principles 5. Questions “The idea behind Honest Practice methodology is to investigate the specific context in which a practice was used, not just to report the outcome – be it success or failure.” – Howie Fisher Honest Practice: How the Public Sector Can Look at Itself Outline
4. Background
5. About your presenter College of Public and Community Service
9. What is GetCalFresh? • A service provided by Code for America that acts as a digital assister, available at: https://www.getcalfresh.org (demo here) • It allows anyone to apply for CalFresh using a mobile phone, and guides them through the eligibility process using email, text, and live chat support • Like an assister, it works on top of existing systems, translating required questions to plain language, and submitting them to CalWIN or C4Yourself
10. GetCalFresh focuses on... 1 2 3 Getting more people through the door Supporting clients through the eligibility process Ensuring clients don’t churn for administrative reasons* *currently offering an easier SAR-7 in three counties
11. GetCalFresh is working who we help apply • 16,000 applications submitted monthly • 33,000 people helped monthly • 55% avg. approval rate across 33 counties • 32% increase of avg. online applications • 20% of all apps are submitted via GCF • 60% are applying for CalFresh for the first time
12. GetCalFresh is people June 1, 2018 March 6, 2017 Director Engineer Design & Research Engineer Client Success Program & Partnerships Engineer Engineer Engineer Product Program & Partnerships Client Success Data Science Program & Partnerships
13. Values & Principles
16. Operations
18. The right place at the right time... • 2 different CalFresh application webforms • Fax Machines • Email • “Secure” Email • “Secure” Dropbox • County-specific upload websites • SFTP • Lots of humans and their own operational systems and business processes. ...which is at the county office by 5pm that business day.
20. “Understand Meet User Needs First and foremost, we talk to users and understand who they are. We’ve found that much of the time the best way to get started is not to write any code, but instead to just help people, manually. Once we have a live service, we make sure to insert a “seam” such as live chat that allows us to talk to users regularly.
21. “Optimize for Speed of Learning Armed with a clear North Star and a plan to reach it, we can focus on possibly the most important part of our process — learning at the fastest rate possible. We’re big fans of the Lean Startup approach and we’re always trying to accelerate the speed at which we can get through Build-Measure-Learn loops.
22. “Build Visibility In ...we make sure we have good visibility into problems users may be having with our services. Quality monitoring is key when it comes to providing a high quality service to our customers.
23. Rituals & Practices
24. By the numbers Ben's Start Date March 6, 2017 Presentation Date June 1, 2018 Calendar Days 452 Business Days Excluding Holidays 313 Number of Weeks 60 Applications Assisted 131,513 Deploys 1,044 Issues / Stories Written 1,437 Daily Standups 246 Bi-Weekly Planning Meeting 30 Bi-Weekly Team Review and Retro 30 Bi-Weekly Engineering Retro 29 Morning Rituals 183 Daily Drives 273 Incident Postmortems 10 Days Client Success Coverage 29 Impact / Complexity meetings 4 Strategic Planning Onsites 1 Team Offsites 3
25. Actual growth Active in 9 counties Active in 33 counties
26. Meetings
27. Meetings Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building
28. March 6, 2017 - June 1, 2018 ➔ 1,044 production deploys ➔ 3.3 deploys per workday ➔ 1,437 new stories / issues Pace of development
29. 30 Bi-Weekly Planning Meeting
30. 183 Morning Rituals
31. Meetings - Daily Drive Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building
32. The right place at the right time... • 2 different CalFresh application webforms • Fax Machines • Email • “Secure” Email • “Secure” Dropbox • County-specific upload websites • SFTP • Lots of humans and their own operational systems and business processes. ...which is at the county office by 5pm that business day.
33. We call submittal “Driving” Automated Driving Manual Driver
34. Common “Driving” Problems • Mailing Address Parsing • Document Formats (bmp, gif, multi-age tiffs) • Document Sizes (100 page leases), • Spécial Cháråcters • Webform updates • Webform degradations • Fax machine errors • “Rare” eligibility cases
35. Actual growth Active in 9 counties Active in 33 counties
36. March 6, 2017 June 1, 2018 273 Daily Drives April 17, 2017
37. 273 Daily Drives
38. 273 Daily Drives
39. 273 Daily Drives
40. 273 Daily Drives
41. Meetings - Retro and Consensus Morning Rituals Daily Drive Planning Review and Retrospective Development Consensus Building
42. 30 + 29 = 59 Retros
43. 29 Days on Client Success
44. 10 Incident Postmortems
45. 4 Impact / Complexity Matrices
46. 1 Strategic Planning Onsite
47. 3 Offsites
48. Revisit Principles
49. “Understand Meet User Needs ... just help people, manually... talk to users regularly… Examples: Daily Drive, Customer Success Shifts
50. “Optimize for Speed of Learning ... accelerate the speed at which we can get through Build-Measure-Learn loops... Examples: Morning Rituals, Development ⇔ Daily Drive, Retrospectives, Postmortems
51. “Build Visibility In ...have good visibility into problems users may be having with our services... Examples: Daily Drive, Morning Rituals, Consensus Building
53. Thank you! @bensheldon