Upgrade to Pro — share decks privately, control downloads, hide ads and more …

@TwitterOSS and Innovating in the Open

37b0fbbdf3dc2d989e8082708d50a939?s=47 dlester
November 12, 2014

@TwitterOSS and Innovating in the Open

Presented at Twitter Engineering Effectiveness meet up, November 12, 2014 in Sunnyvale, CA.

37b0fbbdf3dc2d989e8082708d50a939?s=128

dlester

November 12, 2014
Tweet

Transcript

  1. @TwitterOSS and Innovating in the open @davelester, OSS Advocate @

    Twitter Apache Mesos and Aurora PMC Member
  2. Twitter <3 Open Source http://opensource.twitter.com

  3. Front-End

  4. Analytics and Infrastructure

  5. None
  6. “interesting as individual projects may be, taken in isolation, what's

    really accelerating today's pace of change in software is the combinations of these many parts building on and amplifying each other. It's a dynamic that just isn't possible with proprietary software.” - Gordon Haff, Red Hat
  7. “interesting as individual projects may be, taken in isolation, what's

    really accelerating today's pace of change in software is the combinations of these many parts building on and amplifying each other. It's a dynamic that just isn't possible with proprietary software.” - Gordon Haff, Red Hat
  8. Open Programs Office Open Standards Open Source Open Data Community

    Management & Outreach
  9. OSS Advocacy @ Twitter • Making Twitter engineers more effective,

    and using / contributing to open source by removing any roadblocks from that experience • Increasing the number of contributors to flagship Twitter open source projects • We’re engineers embedded within engineering teams • Work both with engineers internally and externally • We’re growing our team!
  10. Case Study: Apache Mesos

  11. What is Mesos? • Top-level software project at the Apache

    Software Foundation • Began as a research project at UC Berkeley, adopted by Twitter early-on • A layer of abstraction between machines in a datacenter and applications that run • cluster manager • resource manager
  12. Key Problems Mesos addresses • Fault tolerance • Resource efficiency

    and utilization
  13. Program against your datacenter like it’s a single pool of

    resources Apache Mesos Node Node Node Node Node Node Node Node Machines Application(s)
  14. Apache Aurora, Twitter’s Open Source Scheduler Apache Mesos Node Node

    Node Node Node Node Node Node Apache Aurora Framework(s) Machines
  15. Apache Aurora • “run 200 of these, forever” • Key

    features Aurora provides: • Deployment and scheduling of jobs • Rich DSL for defining services • Health checking
  16. Mesos Adopters

  17. Project History • 1) Began as research project at UC

    Berkeley AMP Lab • 2) Helped move the project to Apache Incubator to make it independent outside of just Twitter • 3) Grew and matured the project and helped it graduate out of the Apache Incubator • 4) Helped host / organize its first community conference via MesosCon • 5) Along the way, we’ve made our engineers better community members and seen a large # of contributions
  18. An Ecosystem of Frameworks Services Aurora Marathon Kubernetes Singularity Big

    Data Spark Storm Hadoop Batch Chronos Jenkins
  19. Then and Now (17mo later) June 2013 • 4 public

    adopters • 4 Apache releases • 2 OSS frameworks November 2014 • 52 public adopters • 18 Apache releases • 19+ OSS frameworks
  20. Google Summer of Code and Gnome Outreach Program for Women

    • Participated in GSoC Summers of 2013 and 2014. Three interns total • Participated in Gnome Outreach Program for Women Summer 2014, hope to Winter 2014/2015.
  21. Working In the Open • increased speed of innovation •

    more-reliable software • more-visible contributions and impact • broader peer group and sense of community
  22. Thank You Dave Lester @davelester dlester@twitter.com