Keeping Users at the Forefront While Scaling Services

Building resilient and usable systems requires a balancing act between responding to the needs of technical systems and users. This session shares practices and principles that helped the GetCalFresh engineering team scale and improve their service. We'll discuss practical implementations of "Leave a Seam" (creating paths within a system for people talk to you) and daily and weekly operations rituals for keeping technical teams oriented towards meeting the needs of users and technical systems.

Transcript

1. Keeping Users at the
   Forefront While Scaling
   Services
   @bensheldon
   

   View Slide

2. “Building resilient and usable systems requires a balancing act between
   responding to the needs of technical systems and users. This session
   shares practices and principles that helped the GetCalFresh engineering
   team scale and improve their service. We'll discuss practical
   implementations of "Leave a Seam" (creating paths within a system for
   people talk to you) and daily and weekly operations rituals for keeping
   technical teams oriented towards meeting the needs of users and
   technical systems.”
   – Session Synopsis, 2018
   

   View Slide

3. 1. Background
   2. Values & Principles
   3. Rituals & Practices
   4. Revisit Principles
   5. Questions “The idea behind Honest Practice methodology
   is to investigate the specific context in which a
   practice was used, not just to report the outcome
   – be it success or failure.”
   – Howie Fisher
   Honest Practice: How the Public Sector Can Look at Itself
   Outline
   

   View Slide

4. Background
   

   View Slide

5. About your presenter
   College of Public
   and Community
   Service
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. What is GetCalFresh?
   ● A service provided by Code for America that acts as a digital
   assister, available at: https://www.getcalfresh.org (demo here)
   ● It allows anyone to apply for CalFresh using a mobile phone, and
   guides them through the eligibility process using email, text, and
   live chat support
   ● Like an assister, it works on top of existing systems, translating
   required questions to plain language, and submitting them to
   CalWIN or C4Yourself
   

   View Slide

10. GetCalFresh focuses on...
    1
    2
    3
    Getting more people through the door
    Supporting clients through the eligibility process
    Ensuring clients don’t churn for administrative reasons*
    *currently offering an easier SAR-7 in three counties
    

    View Slide

11. GetCalFresh is working
    who we help apply
    ● 16,000 applications submitted monthly
    ● 33,000 people helped monthly
    ● 55% avg. approval rate across 33 counties
    ● 32% increase of avg. online applications
    ● 20% of all apps are submitted via GCF
    ● 60% are applying for CalFresh for the first
    time
    

    View Slide

12. GetCalFresh is people
    June 1, 2018
    March 6, 2017
    Director Engineer
    Design & Research Engineer
    Client Success
    Program & Partnerships
    Engineer Engineer
    Engineer
    Product
    Program & Partnerships
    Client Success
    Data Science
    Program & Partnerships
    

    View Slide

13. Values & Principles
    

    View Slide

14. View Slide

15. View Slide

16. Operations
    

    View Slide

17. View Slide

18. The right place at the right time...
    ● 2 different CalFresh application webforms
    ● Fax Machines
    ● Email
    ● “Secure” Email
    ● “Secure” Dropbox
    ● County-specific upload websites
    ● SFTP
    ● Lots of humans and their own operational
    systems and business processes.
    ...which is at the county office by 5pm
    that business day.
    

    View Slide

19. View Slide

20. “Understand Meet User Needs
    First and foremost, we talk to users and understand
    who they are. We’ve found that much of the time the best
    way to get started is not to write any code, but instead to
    just help people, manually. Once we have a live service,
    we make sure to insert a “seam” such as live chat that
    allows us to talk to users regularly.
    

    View Slide

21. “Optimize for Speed of Learning
    Armed with a clear North Star and a plan to reach it, we
    can focus on possibly the most important part of our
    process — learning at the fastest rate possible. We’re big
    fans of the Lean Startup approach and we’re always trying
    to accelerate the speed at which we can get through
    Build-Measure-Learn loops.
    

    View Slide

22. “Build Visibility In
    ...we make sure we have good visibility into problems
    users may be having with our services. Quality
    monitoring is key when it comes to providing a high quality
    service to our customers.
    

    View Slide

23. Rituals & Practices
    

    View Slide

24. By the numbers
    Ben's Start Date March 6, 2017
    Presentation Date June 1, 2018
    Calendar Days 452
    Business Days Excluding Holidays 313
    Number of Weeks 60
    Applications Assisted 131,513
    Deploys 1,044
    Issues / Stories Written 1,437
    Daily Standups 246
    Bi-Weekly Planning Meeting 30
    Bi-Weekly Team Review and Retro 30
    Bi-Weekly Engineering Retro 29
    Morning Rituals 183
    Daily Drives 273
    Incident Postmortems 10
    Days Client Success Coverage 29
    Impact / Complexity meetings 4
    Strategic Planning Onsites 1
    Team Offsites 3
    

    View Slide

25. Actual growth
    Active in 9 counties
    Active in 33 counties
    

    View Slide

26. Meetings
    

    View Slide

27. Meetings
    Morning Rituals
    Daily Drive
    Planning Review and
    Retrospective
    Development
    Consensus
    Building
    

    View Slide

28. March 6, 2017 - June 1, 2018
    ➔ 1,044 production deploys
    ➔ 3.3 deploys per workday
    ➔ 1,437 new stories / issues
    Pace of development
    

    View Slide

29. 30 Bi-Weekly Planning Meeting
    

    View Slide

30. 183 Morning Rituals
    

    View Slide

31. Meetings - Daily Drive
    Morning Rituals
    Daily Drive
    Planning Review and
    Retrospective
    Development
    Consensus
    Building
    

    View Slide

32. The right place at the right time...
    ● 2 different CalFresh application webforms
    ● Fax Machines
    ● Email
    ● “Secure” Email
    ● “Secure” Dropbox
    ● County-specific upload websites
    ● SFTP
    ● Lots of humans and their own operational
    systems and business processes.
    ...which is at the county office by 5pm
    that business day.
    

    View Slide

33. We call submittal “Driving”
    Automated Driving
    Manual Driver
    

    View Slide

34. Common “Driving” Problems
    ● Mailing Address Parsing
    ● Document Formats (bmp, gif, multi-age
    tiffs)
    ● Document Sizes (100 page leases),
    ● Spécial Cháråcters
    ● Webform updates
    ● Webform degradations
    ● Fax machine errors
    ● “Rare” eligibility cases
    

    View Slide

35. Actual growth
    Active in 9 counties
    Active in 33 counties
    

    View Slide

36. March 6, 2017
    June 1, 2018
    273 Daily Drives
    April 17, 2017
    

    View Slide

37. 273 Daily Drives
    

    View Slide

38. 273 Daily Drives
    

    View Slide

39. 273 Daily Drives
    

    View Slide

40. 273 Daily Drives
    

    View Slide

41. Meetings - Retro and Consensus
    Morning Rituals
    Daily Drive
    Planning Review and
    Retrospective
    Development
    Consensus
    Building
    

    View Slide

42. 30 + 29 = 59 Retros
    

    View Slide

43. 29 Days on Client Success
    

    View Slide

44. 10 Incident Postmortems
    

    View Slide

45. 4 Impact / Complexity Matrices
    

    View Slide

46. 1 Strategic Planning Onsite
    

    View Slide

47. 3 Offsites
    

    View Slide

48. Revisit Principles
    

    View Slide

49. “Understand Meet User Needs
    ... just help people, manually... talk to users regularly…
    Examples:
    Daily Drive, Customer Success Shifts
    

    View Slide

50. “Optimize for Speed of Learning
    ... accelerate the speed at which we can get through
    Build-Measure-Learn loops...
    Examples:
    Morning Rituals, Development ⇔ Daily Drive,
    Retrospectives, Postmortems
    

    View Slide

51. “Build Visibility In
    ...have good visibility into problems users may be
    having with our services...
    Examples:
    Daily Drive, Morning Rituals,
    Consensus Building
    

    View Slide

52. View Slide

53. Thank you!
    @bensheldon
    

    View Slide