Reaching Millions of Girls with Girl Effect

Slide 1

Slide 1 text

Reaching Millions of Girls with Girl Effect Codie Roelf & Jamie Hewland ScaleConf 8 March 2019

Slide 2

Slide 2 text

Introduction Praekelt.org, Girl Effect, and Springster 01

Slide 3

Slide 3 text

Codie Roelf Jamie Hewland Site Reliability Engineer @jayhewland Back-end Engineer @ItsCodieee

Slide 4

Slide 4 text

Praekelt.org uses mobile technology to solve some of the world's largest social problems. Our Mission

Slide 5

Slide 5 text

We build open- source, scalable platforms that allow anyone with a mobile phone to access vital information and essential services— putting wellbeing in the palm of their hands. Our Technologies

Slide 6

Slide 6 text

Girl Effect builds youth brands and mobile platforms to empower girls to change their lives.

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

67 countries 16 languages 16m users

Slide 9

Slide 9 text

This talk is about the many things we did to reach these users

Slide 10

Slide 10 text

This talk is about the many things we did to reach these users —there was no silver bullet

Slide 11

Slide 11 text

The Molo Platform Shared tools for publishing mobisites 02

Slide 12

Slide 12 text

The Molo Platform • Community features • Proﬁles, comments, polls, surveys, tips, reaction questions • Focus on user-generated content • Many sites of the same shape but with heavy customisation • Skins/CSS • Plugins • Internationalisation • Content

Slide 13

Slide 13 text

Internet of Good Things Unicef Tuneme UNFPA BabyCenter Johnson & Johnson

Slide 14

Slide 14 text

The year is 2014… • Facebook launches Free Basics • Simultaneous launch in many countries • Localisation + URL for each country • Incubator with 100 new sites all built with Molo

Slide 15

Slide 15 text

Countries 0 20 40 60 80 December February April June Launched countries "Priority countries"

Slide 16

Slide 16 text

Hosting many sites Scale by replicating 03

Slide 17

Slide 17 text

web01 Nginx Molo PostgreSQL The Internet M Funder Project Server/VM 1:n 1:1

Slide 18

Slide 18 text

Puppet Conﬁguration management web01 M web02 M web03 M web04 M Replicate The Internet

Slide 19

Slide 19 text

web01 Supervisor Nginx web02 db02 web03 M M M M M M M M M M M M M M db01

Slide 20

Slide 20 text

Hoped to automate our problems away with container orchestration • Easy, replicable deployments • Resource-aware scheduling • Health checks & failover

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Peak containers • Johannesburg Mesos/Marathon cluster peaks at 30 worker nodes • ~255GB RAM, 120 cores, ~900 containers • Bare-metal Hetzner hosts • VMs on self-managed XenServer • Shared with other services (not just Springster or Molo sites) • Team of 4 SREs

Slide 27

Slide 27 text

Reducing the number of sites Mo’ containers mo’ problems 04

Slide 28

Slide 28 text

E_TOO_MANY_APPS • Could run many sites/containers easily and quickly • Running many similar sites as completely separate services • Big maintenance overhead for content managers

Slide 29

Slide 29 text

CMS Application x 67

Slide 30

Slide 30 text

CMS Application x 15

Slide 31

Slide 31 text

CMS Application x 15

Slide 32

Slide 32 text

Beneﬁts • Cost efﬁciencies • Less content errors • Less frustration for content managers • User expectation management

Slide 33

Slide 33 text

Consolidation Core Infrastructure 05

Slide 34

Slide 34 text

Core Infrastructure Girl Effect wanted to consolidated their services on one platform • AWS-hosted • Shared container infrastructure • Centralise data and auth for GDPR

Slide 35

Slide 35 text

Moving to AWS • Made much easier by Docker • Adopted Spinnaker for improved CD • Heavier use of AWS services including emerging services • High-availability

Slide 36

Slide 36 text

GitHub Docker Hub Travis CI Mesosphere DC/OS

Slide 37

Slide 37 text

Common base images springster molo-bootstrap django-bootstrap python-base python:slim Developers Product: Source code, app settings SRE Best practices: Webserver conﬁg, users/groups, init Upstream bit.ly/DjangoDocker

Slide 38

Slide 38 text

Multiple channels Keeping Springster relevant 06

Slide 39

Slide 39 text

Then Now Community School Church Newspaper Magazine Television Radio Google  Blogs  Websites Facebook Messenger Twitter Instagram YouTube WhatsApp

Slide 40

Slide 40 text

more access to more information, more often and faster than ever before

Slide 41

Slide 41 text

• Large quantity of high-quality content in our CMS • Need to rearchitect our system to deliver content on more than just mobi-sites • Want swappable front-ends for different channels Bringing Springster to new channels

Slide 42

Slide 42 text

CMS Application WhatsApp Web app KaiOS

Slide 43

Slide 43 text

Reassessing open source • Everything Praekelt.org does is open source • Have to always keep up with changes in OSS communities • Balance between custom/differentiated code & leveraging tools in ecosystem

Slide 44

Slide 44 text

vs. vs. Mesosphere DC/OS Molo Wagtail Kubernetes

Slide 45

Slide 45 text

Conclusions Lessons learned 07

Slide 46

Slide 46 text

Lessons learned • Scaling can happen in different, unpredictable ways • Flexibility and ability to adapt are often critical • Incremental consolidation/optimisation can come later • Balance building your own system vs using what’s available