Community Engineering

I spoke at iOS Con SG in Singapore & shared technical stories about my experience building social media apps

Transcript

1. Community Engineering Tamar Nachmany
2. Seke (sek-ey) • An indigenous language of Nepal • Estimated to have 700 speakers • One building in Brooklyn has 50 of the world’s 700 Seke speakers
3. For many communities, social media is that building
4. 1st generation American + bilingual New Yorker + community app developer
5. What languages do you speak? 1, 2, 3!
6. Today I’m going to be talking about building tech for communities
7. Today I’m going to share some stories about technical decisions & how they evolved
8. Building social media is a dialogue
9. “Evolving community” • The number of users changes • Consumer tech changes • Users seek new ways of expressing themselves • Bad actors join the community • Sub-communities with different needs form
10. 1. Anecdote 1: Scaling new social media 2. Anecdote 2: Rearchitecting posts @ Tumblr Community Engineering
11. Senior Engineer @ Glossier, a digital-first beauty company Formerly iOS @ Tumblr, small startups & the New York Times About Me
12. 1. Scaling
13. or, Why scaling communities is not like romanesco broccoli
14. What is romanesco broccoli?
16. What is scaling?
17. Creating a great experience for as many users as possible
18. We were building a new social network from the ground up
19. After we conceptualized the iOS app, built a frontend & backend and set up the infrastructure, we realized our API requests were slow.
20. Really slow
21. After some investigation, we noticed an issue with our infrastructure
22. Event-driven, serverless computing platform AWS Lambda λ
23. Amazon EC2 vs. AWS Lambda Amazon EC2 • Infrastructure as a service • Computing resources always on • Manual scaling & configuration • More customizable AWS Lambda • Infrastructure platform as a service • Computing resources only on demand • Implicit scaling • Not customizable
24. const handler = async (event: any = {}, _context: Context, callback: Callback) => { } Lambda function
25. What happens when a user makes a request?
26. COLD START WARM START 1st call 2nd call 3rd call 4th call Lambda Initialized User makes a request Code Code Code Code
27. COLD START WARM START • Lambda code is executed • Lambda code is downloaded • Container is mounted • with language • & runtime • & database • Lambda code is executed
28. So why was our app very slow?
29. Nobody* used it
30. Nobody* used it
32. So most starts were cold starts and cold starts were very slow
33. Can it scale up?
34. Can it scale down ?
35. People often skip ahead and build experiences that they imagine will work well at a large scale Communities need to be successful at a small scale before they can grow to a large scale
36. Think about the mechanics of one user’s experience SHARING CONNECTING READING CREATING
37. Scale impacts each area & the way they interconnect SHARING CONNECTING READING CREATING
38. • Bring paid creators onto your platform • Distribute the responsibility of giving folks a good experience (uplift community leaders) • Bring an existing, external community onto your platform Create a large(r) scale “manually”
39. • Small, trustworthy community makes people feel more comfortable sharing • Levers for joy & interaction don’t need to be about popularity like likes and comments Create a compelling small-scale
40. “Single-player experience”
41. Why is scaling communities not like romanesco broccoli
43. Romanesco is a natural fractal • Fractals are shapes with self-similar forms at different scales • At every scale, a fractal looks the same
44. Communities don’t scale like fractals. They’re complex, and different at every scale. Scale has a huge, non-linear impact on users.
45. Imagine scaling up a dinner
46. Takeaways • New community products can fail because of scale • We chose a good technology for scaling up, but a bad technology for scaling down • Moreover, the technology we used is probably not optimized for social media • The reality is, you will probably start at a small scale. Don’t design an experience or build tech that is doomed at a small scale.
47. 2. Evolving Features
48. User generated content
49. This next example is also a story of scale. But more than just scale, it’s a story about how the early technical decisions teams make and how those decisions can be gracefully transformed over time.
51. Story of an evolution of social media
52. Story of two technology stacks
53. Let’s go back
54. 2004 - Facebook is launched
55. 2006 - Twitter is launched
56. 2007 - Tumblr is launched
57. 2007 - The iPhone is launched
58. 2007 - Social media is all about self-expression through custom HTML
59. Internet of <tags>
60. Source: cameronsworld.net
62. Tumblr began as a tool for building creative standalone websites…
63. …and became a community platform, with a feed of interweaving content
66. Tumblr revolves around posts
67. The early vision of Tumblr is encapsulated by its original post architecture
68. When I joined Tumblr, posts were stored as HTML
69. <li /> <div /> <img> <quoteblock / > <h1 /> <video /> <h2 /> These are HTML tags
70. HTML • Stands for Hyper Text Markup Language • Describes the structure of a web page • Is made up of a series of elements • …which tell the browser what to display & how
71. Why? • HTML was an important part of what expression meant in online culture when Tumblr was founded • One of Tumblr’s original forms of expression was building themes • Browsers are built with this exact purpose
72. This made a lot of sense… until mobile apps came along
73. By definition, browsers know how to build a UI using HTML but native iOS apps don’t. Rendering HTML was a complex, inefficient, multi-step process for our iOS app.
74. There was one more important detail to the way posts were represented
75. type 1 2 3 Post table was designed for the early realities of Tumblr: for a small scale and small number of post types HTML HTML HTML HTML Image Data HTML Text Post Photo Post
76. type 1 2 3 Adding new content to a post type became impossible at scale HTML HTML HTML HTML Image Data HTML Text Post Photo Post ? New HTML New HTML
77. Tumblr needed to evolve to a mobile- optimized post architecture that created a better user experience
78. Tumblr Neue Post Format
79. You can check out Tumblr’s public post format spec here: https://github.com/tumblr/docs/blob/master/npf-spec.md
80. Post <img src=“img.jpg” /><p><div />
81. Post JSON JSON JSON
82. Post Content Block Content Block Content Block
83. Post Content Block Content Block Content Block Image Text Video Audio Link Content types:
84. Post "content": [ { "type": "text", "text": “Here’s some text" }, { "type": "text", "subtype": "quote", “text": “Here’s a quote.", } ]
85. class PhotoPost: Post { photo: Photo }
86. class PhotoPost: Post { content: [ContentBlock] } class PhotoContentBlock: ContentBlock { photo: Photo } protocol ContentBlock {}
87. class PhotoContentView { init(contentBlock: ContentBlock) { ... } }
88. Post Photo Content Block Text Content Block
89. Message Photo Content Block Text Content Block
90. This concept of a client that uses a content type & some styling metadata to figure out how to render stuff sounds familiar…
91. With NPF, Tumblr clients behave like simplified browsers, taking arrays of JSON objects and figuring out how to render each type of supported content type. Sounds like a browser!
92. type 1 2 3 The post database was simplified as well HTML HTML HTML HTML Image Data HTML Text Post Photo Post
93. type 1 JSON Array Text Post Photo Post JSON Array The post database was simplified as well
94. type 1 {}, {} Text Photo {}, {} Photo Video Audio Text {}, {} {}, {} {}, {} {}, {} Video {}, {} Photo {}, {}
95. And all this happened without users noticing a thing
96. NPF was a smart and thoughtful refactor that became a foundation for supporting the features of the future
97. What are the takeaways from this refactor?
98. Takeaways 1. You are who you are 2. Decouple evergreen and ephemeral features 3. Not everything needs to be strict 5. Leverage Swift 6. How do you make fundamental changes to a product without anybody noticing?
99. Every social media app has the DNA of its early vision & early community Tumblr began as a website building tool, founded during the ‘internet of tags’ era. As users shifted to mobile, Tumblr’s underlying technology had to evolve. 1.You are who you are
100. The desire to share beauty and ideas won’t change but the vehicle for that expression will. Don’t tightly couple creative content to the current vehicle for that content such as a post 2. Decouple evergreen & ephemeral features
101. By using a more flexible data structure to represent post data (an array of validated JSON objects) and a simplified database strategy (a single column for post contents), this refactor made introducing new features far faster & easier to do at Tumblr’s scale 3. Not everything needs to be strict
102. Compilers are our friends. Swift has many language features that are conducive to creating content blocks that can be used throughout an app. Switch cases (without a default) for example make it easier to ensure that new content types enums are handled absolutely everywhere. Protocols enabled the creation of a flexible yet structured contents array, which created an easy workflow of adding support for new content types 4. Leverage Swift
103. 5. Learn how to make fundamental changes to a product without anybody noticing
104. Shout out to the entire Tumblr engineering team & to the Tumblr Expression Team folks who implemented this project
105. ✨
106. What did we discuss today? • We started by flipping the scale mantra on its head and discussing AWS Lambda and the importance of building community tech that can scale down. • Then we discussed Tumblr’s post spec evolution, from HTML to JSON, from tightly coupled post types to extensible content blocks and what this refactor & the challenges of Tumblr’s scale teach us about evolving from early technical decisions • & languages, fractals, and more
107. Your app will have the DNA of its history And of the future as it evolves
108. Building community apps is an exercise in preparing for the future
109. It will be easier to create the future we want if we’re not doing it alone
110. Thanks Tamar Nachmany @tamarshmallows
111. Photo Credits • Seke photography by Diana Zeyneb Alhindawi for the New York Times • Dinner photography by Michael Graydon and Nikole Herriot • Font is Dark Mono