Open Source Craft at Twitter

Slide 1

Slide 1 text

Open Source Craft at Twitter Chris Aniszczyk (@cra) http://aniszczyk.org #monkigras

Slide 2

Slide 2 text

Twistory Twitter History

Slide 3

Slide 3 text

2006: A simple idea...

Slide 4

Slide 4 text

2008: Growing Pains

Slide 5

Slide 5 text

2009... Crazy Growth

Slide 6

Slide 6 text

2010+: Shit, build a company!

Slide 7

Slide 7 text

Employee Growth...

Slide 8

Slide 8 text

Now: Growth Continues... 1400+ Employees Worldwide 50% Employees are Engineers 200M+ Active Users 400M+ Tweets per Day 33+ Languages Supported 60% Active Users are on Mobile 100+ Open Source Projects

Slide 9

Slide 9 text

Engineers run the asylum...

Slide 10

Slide 10 text

Code dumping happens...

Slide 11

Slide 11 text

Code dumping happens...

Slide 12

Slide 12 text

No Ownership = Problems Start with ownership. Created an Open Source Office in 2011

Slide 13

Slide 13 text

Open Source Craft and Culture How we roll...

Slide 14

Slide 14 text

Open Source Craft (operating principles) Use Open Assume Open Deﬁne Secret Sauce Measure Everything Default to GitHub Default to Permissive Acquire and Open Pay it Forward

Slide 15

Slide 15 text

Use Open Use and benchmark open source software by default. When starting a new initiative, always evaluate open source options before going to reinvent the wheel. (e.g., if redis doesn’t work for you, you better have solid evidence)

Slide 16

Slide 16 text

Twitter Runs on Open Source

Slide 17

Slide 17 text

Define Secret Sauce Don’t open source anything that represents a core business value. Define your secret sauce so there’s a shared understanding that can guide decisions. Embed this secret sauce within your culture and company via training.

Slide 18

Slide 18 text

Secret Sauce, what is it? What’s yours?

Slide 19

Slide 19 text

If you know your secret sauce...

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Assume Open Assume that what you are developing will be opened in the future. Pretend the whole world will be watching. Use reasonable third party dependencies to prevent pain down the road. (we mostly use Apache’s Third Party Guidelines as a starting point)

Slide 22

Slide 22 text

Default to GitHub The GitHub community is the largest open source community, with over three million users. You would be stupid to ignore that fact. Embrace social coding tools to lower the barrier to contribution and participation.

Slide 23

Slide 23 text

Foundations are Good* We just prefer not to default to them. We view them as a place for stable projects that grow into maturity, not to incubate new projects. Our goal is to gain traction first as fast as possible. If not, fail fast and carry on.

Slide 24

Slide 24 text

Default to Permissive

Slide 25

Slide 25 text

Be Permissive For outbound open source software, we default to OSI permissive licenses (the ALv2 in the majority of cases). We do this so we can maximize adoption and participation, which we favor instead of control.

Slide 26

Slide 26 text

See http://blogs.the451group.com/opensource/2011/12/19/the-future-of-commercial-open-source-business-strategies/ Embrace the Trend

Slide 27

Slide 27 text

See http://antirez.com/news/48 Notes from Antirez (1) “First of all, open source for me is not a way to contribute to the free software movement, but to contribute to humanity. This means a lot of things, for instance I don't care about what people do with my code, nor if they'll release back their modiﬁcations. I simply want people to use my code in one way or the other. Especially I want people to have fun, learn new stuff, and make money with my code. For me other people making money out of something I wrote is not something that I lost, it is something that I gained.”

Slide 28

Slide 28 text

See http://antirez.com/news/48 Notes from Antirez (2) 1) I'm having a bigger effect in the world if somebody can pay the bills using my code. 2) If there are N subjects making money with my code, maybe they will be happy to share some of this money with me, or will be more willing to hire me. 3) I can be myself one of the subjects making money with my code, and with other open source software code. For all this reasons my license of choice is the BSD licensed, that is the perfect incarnation of do whatever you want as a license.”

Slide 29

Slide 29 text

Acquire and Open* Include open sourcing software in M&A discussions, especially if you’re mainly acquiring talent or shelving the product. There’s no need for software to go to waste.

Slide 30

Slide 30 text

Acquire and Open: RedPhone See https://github.com/WhisperSystems/RedPhone

Slide 31

Slide 31 text

Acquire and Open: Clutch.IO See http://engineering.twitter.com/2012/10/open-sourcing-clutchio.html See http://www.prweb.com/releases/2012/10/prweb10067693.htm

Slide 32

Slide 32 text

Measure Everything If you can’t measure what you’re doing, you have no idea what you’re doing. We measure everything inside of Twitter (affectionately called birdbrain) and make it accessible to everyone.

Slide 33

Slide 33 text

Pay it Forward Support open source organizations and projects important to your business, it’s the right and smart thing to do. This can be financially or simply staffing projects that are strategic to you.

Slide 34

Slide 34 text

Open Source Craft* Use Open Assume Open Deﬁne Secret Sauce Measure Everything Default to GitHub Default to Permissive Acquire and Open Pay it Forward Note: This fits in a tweet

Slide 35

Slide 35 text

Scaling Scaling an open source program

Slide 36

Slide 36 text

Light / Automated Process Automated Process via JIRA Workflow Inspiration from Eclipse.org’s IPZilla

Slide 37

Slide 37 text

Not all process is bad! “process is an embedded reaction to priority stupidity” True. Remember the history of surgery? Ignaz Semmelweiz (hand washing) Surgery checklist: Glad your surgeon washes hands? Even with surgery process, metal objects still are left in people :) See http://many.corante.com/archives/2003/09/17/process_is_an_embedded_reaction_to_prior_stupidity.php See http://en.wikipedia.org/wiki/Ignaz_Semmelweis

Slide 38

Slide 38 text

Automate Quality Checklist Tooling to check for baseline “quality” before we open √ README √ LICENSE √ CONTRIBUTING.MD √ .travis.yml √ Avoid KEYS / sensitive bits √ Avoid GPL License Family Attempts to build and posts results in JIRA. Enforces and teaches good practices.

Slide 39

Slide 39 text

Automate Sourcing for Hiring Let’s scale hiring a bit! Run monthly queries on contributors to our open source projects and projects of interest. Hand it off the Recruiting and the ATS. It’s hard enough to find good talent, why not hire from open source projects of interest.

Slide 40

Slide 40 text

Automate Reporting / Metrics Send out weekly reports based on: Open Sourced Projects (what opened) Top Committers (commits+issues closed) Releases (assumed via git-tag) Contentious Issues (issues with most comments) Top Watched Project (most stars) Top Forked Project (most forks) Track trends. Motivates contribution.

Slide 41

Slide 41 text

Conclusion Deﬁne Your Principles Open Source Almost Everything Measure / Automate Everything

Slide 42

Slide 42 text

Thanks for listening! (especially if you survived last night’s beer fest) @cra zx@twitter.com