Future of Collaboration

My pycodeconf talk where I spend some time raising awareness about some issues and then propose some fixes.

Transcript

1. Daniel Greenfeld @pydanny The Future of Collaboration by Daniel Greenfeld
2. Daniel Greenfeld @pydanny Who am I? Daniel Greenfeld (@pydanny) Cartwheel Web / Revsys http://djangopackages.com https://whitespacejobs.org Fiancé of Audrey Roy http://www.flickr.com/photos/pydanny/4442245488
3. Daniel Greenfeld @pydanny Mark Pilgrim is Gone
4. Daniel Greenfeld @pydanny Mark Pilgrim • feedparser • conbtributed to httplib2 • Dive into Python • Dive into HTML5
5. Daniel Greenfeld @pydanny Mark’s Projects? • What is the copyright on his material? • What about his code?
6. Daniel Greenfeld @pydanny httplib2 • Mark wasn’t lead, but outage anyway... • PyPI? • http://code.google.com/p/httplib2 was down • Had to find a cached download •A lot of libraries depend on httplib2
7. Daniel Greenfeld @pydanny PyPI Issues? • Packages are too easily deleted • Dependency checks for that package • Request a project hand-off? • Other projects need to be notified • RSS feed of these things • Needs moderation but PyPI busy
8. Daniel Greenfeld @pydanny Repeating History? • http://pypants(.com|.org|.net) is gone • http://djangolint.com is here today
9. Daniel Greenfeld @pydanny Repeating History? • django-piston • python.org • http://opencomparison.org
10. Daniel Greenfeld @pydanny A Dark Future
11. Daniel Greenfeld @pydanny Critical Packages Breakdown • A number of critical Python Packages vanish • Various build scripts fail • Replace from caches/backups •Domain Knowledge is gone Dark Future
12. Daniel Greenfeld @pydanny Repercussions • Annoyance • Ability for Python to move forward HALTS • Social Issues Dark Future
13. Daniel Greenfeld @pydanny Not the Future
14. Daniel Greenfeld @pydanny • Legacy code tries to install legacy packages • Various build scripts fail • Replace from caches/backups MAYBE! •Domain Knowledge is gone TODAY
15. Daniel Greenfeld @pydanny Library of Alexandria http://en.wikipedia.org/wiki/File:Ancientlibraryalex.jpg
16. Daniel Greenfeld @pydanny Stuff we make today Is legacy within 5 years
17. Daniel Greenfeld @pydanny Trust Issues
18. Daniel Greenfeld @pydanny Lack of trust makes collaboration hard
19. Daniel Greenfeld @pydanny No collaboration causes the NIH plague
20. Daniel Greenfeld @pydanny http://en.wikipedia.org/wiki/File:Zombie-walk-kids.JPG
21. Daniel Greenfeld @pydanny Repercussions TODAY • Annoyance • Ability for Python to move forward HALTS • Social issues
22. Daniel Greenfeld @pydanny Solution? Sponsorships!
23. Daniel Greenfeld @pydanny Sponsorships • Individual, Corporate, Academic, GSOC • Advance an open source project forward!
24. Daniel Greenfeld @pydanny Sponsorships • Python core • Django • Plone • Twisted • Read the Docs (PSF) • WebOb (PSF) Tons more I’m missing!
25. Daniel Greenfeld @pydanny Problems • Application process • Seems focused on short-term development • Interns often produce unusable code • Ongoing maintenance?
26. Daniel Greenfeld @pydanny Critical Packages Breakdown • Legacy code tries to install legacy packages • Various build scripts fail • Replace from caches/backups MAYBE! •Domain Knowledge is gone N O C H AN G ES
27. Daniel Greenfeld @pydanny Repercussions • Annoyance • Ability for Python to move forward HALTS • Social issues N O C H AN G ES
28. Daniel Greenfeld @pydanny Same as before
29. Daniel Greenfeld @pydanny Still have Trust Issues
30. Daniel Greenfeld @pydanny Why should I collaborate with you?
31. Daniel Greenfeld @pydanny Future still looks dark
32. Daniel Greenfeld @pydanny So now what?
33. Daniel Greenfeld @pydanny Solution? Community Managers
34. Daniel Greenfeld @pydanny Community Managers • David Eaves DjangoCon 2011 Keynote • http://bit.ly/djangocon-keynote-2011-1 • Makes sure packages are maintained!
35. Daniel Greenfeld @pydanny Problems • Need core/senior developers • They are already busy • Who assigns authority?
36. Daniel Greenfeld @pydanny Solution Reconsidered PSF Paid Community Manager • Reasonable pay • Works with PyPI team to do package curation • Helps project leads if they need to hand off • Broadcasts so the community knows about it
37. Daniel Greenfeld @pydanny Solution Reconsidered PSF Paid Community Manager • Performance reviews • Term limits • Great resume material • Volunteers have different priorities
38. Daniel Greenfeld @pydanny Increase of Trust
39. Daniel Greenfeld @pydanny This makes collaboration easier
40. Daniel Greenfeld @pydanny The Brighter Future
41. Daniel Greenfeld @pydanny Critical Packages Breakdown • A number of critical Python Packages vanish • Various build scripts fail • Replace from caches/backups •Domain Knowledge is gone Today
42. Daniel Greenfeld @pydanny Critical Packages Breakdown • A number of critical Python Packages vanish • Various build scripts continue •Domain Knowledge is at risk Future
43. Daniel Greenfeld @pydanny Repercussions • Annoyance • Ability for Python to move forward HALTS • Social Issues Today
44. Daniel Greenfeld @pydanny Repercussions • Annoyance • Ability for Python to move forward slows • Mitigated Social Issues Future
45. Daniel Greenfeld @pydanny Precedents • Ubuntu Community Manager • Isn’t there a Fedora Community Manager? • Twilio Community Managers
46. Daniel Greenfeld @pydanny But...
47. Daniel Greenfeld @pydanny I want more
48. Daniel Greenfeld @pydanny More reasons to trust
49. Daniel Greenfeld @pydanny More incentives to collaborate
50. Daniel Greenfeld @pydanny More reasons to give code back
51. Daniel Greenfeld @pydanny Question: How do we keep the projects that do this operating?
52. Daniel Greenfeld @pydanny Call to Action
53. Daniel Greenfeld @pydanny Proposal: PSF Project Incubation
54. Daniel Greenfeld @pydanny The Idea Part I The PSF provides seed money to open source projects that can provide a return of investment on these items: • Tangible benefit to the Python community • Money via a viable business model
55. Daniel Greenfeld @pydanny The Idea Part II • Choose from participants in coding contests • Django Dash & other unborn contests • PSF offers YC-style seed funding • Marketing: Blog posts, PyCon booths, python.org • Lets them go to do their thing What the PSF Board / Community does:
56. Daniel Greenfeld @pydanny The Idea Part III • Deliverable open source code • Something that makes the Python world better • PSF is an investor so dividends come back What the incubatee provides:
57. Daniel Greenfeld @pydanny What this isn’t • Covering < $100 monthly hosting • Throwing money at things without a self- supporting business model • For part-timers - It’s all or nothing
58. Daniel Greenfeld @pydanny If I were rich I would do this right now
59. Daniel Greenfeld @pydanny Example Projects Real World
60. Daniel Greenfeld @pydanny djangolint.com
61. Daniel Greenfeld @pydanny http://djangolint.com • Winner of this year’s Django Dash • I want this for all of Python • Will it be maintained? • Will it go the way of PyPants?
62. Daniel Greenfeld @pydanny http://djangolint.com
63. Daniel Greenfeld @pydanny http://djangolint.com
64. Daniel Greenfeld @pydanny http://djangolint.com • Winner of this year’s Django Dash • I want this for Python • API/Syndication? (Open Comparison) •Will it be maintained? •Will it go the way of PyPants?
65. Daniel Greenfeld @pydanny Pay to analyze privately Business Model (Great for client projects!)
66. Daniel Greenfeld @pydanny Easy linting increases Trust
67. Daniel Greenfeld @pydanny Collaborate with high lint rated projects!
68. Daniel Greenfeld @pydanny readthedocs.org
69. Daniel Greenfeld @pydanny python.rtfd.org
70. Daniel Greenfeld @pydanny http://rtfd.org • Placed in the 2010 Django Dash • python.rtfd.org • plone.rtfd.org • pydanny-event-notes.rtfd.org
71. Daniel Greenfeld @pydanny A measurable benchmark!
72. Daniel Greenfeld @pydanny Documentation increases Trust
73. Daniel Greenfeld @pydanny Documentation increases Trust Trust increases Collaboration
74. Daniel Greenfeld @pydanny Can it do more? • Support other markup languages?
75. Daniel Greenfeld @pydanny Business Model Pay for private doc hosting?
76. Daniel Greenfeld @pydanny depot.io
77. Daniel Greenfeld @pydanny http://depot.io • A place to freeze your Python dependencies • Let’s PyPI focus on what it does best - Indexing • Provides additional security • Fetch packages any way you want • Not yet launched. ☹ Developer paying the bills?
78. Daniel Greenfeld @pydanny http://depot.io • PyPI is still the canonical source • Less need for Paid Community manager • Will probably archive all legacy packages Possible Advantages (Except ones flagged for critical things like security) • OMG CAN WE HAVE THIS PLEASE?
79. Daniel Greenfeld @pydanny Dependability increases Trust Trust increases Collaboration
80. Daniel Greenfeld @pydanny http://depot.io • PSF should incubate this project • Open source it? • + Lowers the bus factor • - Easy launch of copy versions of site • - Does it become harder to earn profit? • + Once launched, SLA is key
81. Daniel Greenfeld @pydanny PyPI
82. Daniel Greenfeld @pydanny http://pypi.python.org/pypi
83. Daniel Greenfeld @pydanny PyPI • Launched before the days of OS X • The canonical source for Python packages • Focusing hard on just Indexing
84. Daniel Greenfeld @pydanny Business Model Pay for a PyPI Appliance?
85. Daniel Greenfeld @pydanny Dependability increases Trust Trust increases Collaboration
86. Daniel Greenfeld @pydanny opencomparison.org
87. Daniel Greenfeld @pydanny http://djangopackages.com/
88. Daniel Greenfeld @pydanny http://pyramid.opencomparison.org/
89. Daniel Greenfeld @pydanny opencomparison.org • Competed in Django Dash 2010 • More sites coming soon! • Plone • Flask • Python! • A year of expansion and code cleanup
90. Daniel Greenfeld @pydanny opencomparison.org Largest sprint group at PyCon 2011!
91. Daniel Greenfeld @pydanny http://bit.ly/django-reg Many good people Most downloads Maintained? Much recent work
92. Daniel Greenfeld @pydanny http://bit.ly/django-reg Many good people Most downloads Maintained? Much recent work Django Social Auth is my current registration preference
93. Daniel Greenfeld @pydanny Metrics increase Trust Trust increases Collaboration
94. Daniel Greenfeld @pydanny • More OAUTH and OpenID Support • Improved search • Language neutral to support other tools • Better caching • Additional RSS feeds • Give us more hard metrics please! I — Open Comparison
95. Daniel Greenfeld @pydanny Business Model ???
96. Daniel Greenfeld @pydanny Business Model • Consulting for Open Comparison • Enterprise Support? • Ideas anyone?
97. Daniel Greenfeld @pydanny Results?
98. Daniel Greenfeld @pydanny Self-sustaining Python ecosystem
99. Daniel Greenfeld @pydanny Critical Packages Breakdown • A number of critical Python Packages vanish • Various build scripts fail • Replace from caches/backups •Domain Knowledge is gone Today
100. Daniel Greenfeld @pydanny Critical Packages Breakdown • A number of critical Python Packages don’t vanish • Various build scripts continue •Domain Knowledge remains Future
101. Daniel Greenfeld @pydanny Repercussions • Annoyance • Ability for Python to move forward HALTS • Social Issues Today
102. Daniel Greenfeld @pydanny Repercussions • Complacency about stability of ecosystem • Ability for Python to move forward continues • What Social Issues? Future
103. Daniel Greenfeld @pydanny The Future?
104. Daniel Greenfeld @pydanny The Ideas • Paid Community Manager • PSF project incubation
105. Daniel Greenfeld @pydanny Shoot them down!
106. Daniel Greenfeld @pydanny Paid Community Manager • If the PSF doesn’t do it, there is already PyPI • Maybe the PSF shouldn’t be involved • Ubuntu didn’t debate the position
107. Daniel Greenfeld @pydanny PSF Project Incubation This is not cheap! At least $15K per incubation
108. Daniel Greenfeld @pydanny PSF Project Incubation This is not cheap! At least $15K per incubation Problem
109. Daniel Greenfeld @pydanny PSF Project Incubation • Already exist, just not with the PSF • How much produced code comes out? • Imagine the energy of a start-up giving back! • Is happening already so PSF ought to profit...
110. Daniel Greenfeld @pydanny Thanks Github! http://octodex.github.com/#collabocats
111. Daniel Greenfeld @pydanny Questions?