Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evolving Open Source - Yehuda Katz

spreeconf
August 23, 2012

Evolving Open Source - Yehuda Katz

spreeconf

August 23, 2012
Tweet

More Decks by spreeconf

Other Decks in Programming

Transcript

  1. SOLVING HARD PROBLEMS I'm going to focus this talk on

    open source projects trying to solve ambitious problems. I've worked on a few.
  2. A WILD REPLACEMENT CHARACTER APPEARS Identify symptoms Understand domain Identify

    root cause Consider solutions Solve root cause Community work http://bit.ly/harder-than-it-looks I've given a talk on the topic of "It's Harder than it Looks" a few times that might be a good companion for this talk.
  3. DOM READINESS. Typically, "solving hard problems" means providing solutions that

    handle common edge-cases. Edge-cases doesn't mean cases only a few people hit. For me, the important edge- cases are parts of standard-looking solutions that aren't obvious.
  4. GOOD VS. GREAT SOLUTIONS. For me, what separates good from

    great solutions is how completely they tackle the problem spaces they claim to solve.
  5. ✓Routing ✓Data Bindings ✓Model Layer ✓View Objects ✓MVWTF FEATURE CHECKBOXES.

    A lot of times people don't notice the difference between partial and complete solutions until the solution starts to break down.
  6. ✓Routing ✓Data Bindings ✓Model Layer ✓View Objects ✓MVWTF FEATURE CHECKBOXES.

    Feature-checkbox-based choices undervalue the *completeness* of a solution. They're implicitly used in the marketing of less complete solutions.
  7. ✓Routing ✓Data Bindings ✓Model Layer ✓View Objects ✓MVWTF FEATURE CHECKBOXES.

    Solutions that handle edge-cases well do it because of a belief in the value of complete solutions, despite the fact that it's impossible to market.
  8. DETOUR I want to take a brief detour to address

    the most serious objection to the thrust of my talk.
  9. Write programs that do one thing and do it well.

    “ "THE UNIX PHILOSOPHY" This amounts to an objection even to modular solutions where each module is built by the same project.
  10. "Write programs that do one thing and do it well"

    This simply moves the complexity to rapidly evolving *public* APIs.
  11. "Write programs that do one thing and do it well"

    This simply moves the complexity to rapidly evolving *public* APIs.
  12. edge-cases "Write programs that do one thing and do it

    well" This simply moves the complexity to rapidly evolving *public* APIs.
  13. "Write programs that do one thing and do it well"

    In many cases, it pushes the integration tax onto the *user*.
  14. "Write programs that do one thing and do it well"

    In many cases, it pushes the integration tax onto the *user*.
  15. edge-cases "Write programs that do one thing and do it

    well" In many cases, it pushes the integration tax onto the *user*.
  16. This is the Unix philosophy: Write programs that do one

    thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a “ THE UNIX PHILOSOPHY
  17. $ git add $(git status | awk 'p==1{print $2}; /git

    add /{p=1}') COMPLEXITY. The Unix philosophy works best when working with simple text formats. In this case, it would be better to have a solution in the git porcelain.
  18. integration costs edge cases Components connect to each other very

    promiscuously. Each link introduces a transaction cost. Typically the burden of this cost is on the end-developer.
  19. integration costs edge cases Components connect to each other very

    promiscuously. Each link introduces a transaction cost. Typically the burden of this cost is on the end-developer.
  20. edge cases integration gets cheaper Components connect to each other

    very promiscuously. Each link introduces a transaction cost. Typically the burden of this cost is on the end-developer.
  21. integration gets cheaper some of the time Components connect to

    each other very promiscuously. Each link introduces a transaction cost. Typically the burden of this cost is on the end-developer.
  22. mongrel rails sessions and cookies parameter parsing HTTP security Request

    and Response objects HTTP mongrel-rails coupling
  23. WHY? For a possibly long time, the costs of integrating

    with third parties is high. You basically have to build the marketplace yourself. It's cheaper to vertically integrate.
  24. STANDARDS THAT AREN'T REALLY STANDARDS. Early on, people pimp "de

    facto standards", because they're a good bludgeon against an integrated solution, even though they don't soften the integration tax much. (Rack for the first year or two)
  25. mongrel / unicorn / thin HTTP rails entry point standards

    session middleware HTTP security standards Controller Action standards standards
  26. OTHER INTEGRATION COSTS. Even when interfaces are standardized, there are

    still integration costs. This is why Rails uses standard interfaces, but still uses its own components except for a few areas.
  27. write who wc unlink tty rmdir rm pr od mv

    mkdir mesg mailx ls ln link find ed du df date cp cmp chown chmod cat uniq time split kill paste tr sleep ps nice file comm cal newgrp tsort touch sed awk at sh AT&T UNIX Unix itself developed many of its tools (and therefore interfaces) in a single place, reduces coordination and integration costs and increasing the speed of evolution.
  28. CRITICISM YIELDS DEFENSIVENESS. Early anti-cruft criticism, because it targets cruft

    that is aligned with real problems, creates an internal defensiveness to criticisms of cruft.
  29. THIS BECOMES RECEIVED WISDOM. "Removing cruft for the sake of

    it will probably break something. Don't touch anything unless you're adding features! We knew what we were doing!"
  30. REMOVING CRUFT. good case remove cruft test fails aha! revert

    worse case reinforces received wisdom remove cruft
  31. REMOVING CRUFT. good case remove cruft test fails aha! revert

    worse case reinforces received wisdom remove cruft ship release
  32. REMOVING CRUFT. good case remove cruft test fails aha! revert

    worse case reinforces received wisdom remove cruft ship release break users
  33. REMOVING CRUFT. good case remove cruft test fails aha! revert

    worse case reinforces received wisdom remove cruft ship release break users oh shit! revert
  34. PROJECTS DEVELOP IMMUNE SYSTEMS. If someone wants to remove cruft

    for the sake of it, people reject the changes out of hand.
  35. FORK IN THE ROAD. By keeping track of the reasons

    for our cruft (and analyzing after the fact), we could determine the value to the codebase of dropping oldIE support.
  36. LIFECYCLE. cruft defensiveness cruft = solution a gap appears received

    wisdom consolidation splash of cold water the most destructive part of the cycle is once the assumptions lose validity, the fact that an alternate solution does not solve every edge-case invalidates the choices they are making that use ecosystem changes to remove cruft.
  37. IT'S OK TO BE "THE DOM LIBRARY FOR IE6 USERS"

    Projects built five years ago are in a good position to provide solutions for edge-cases that were prominent five years ago.
  38. NOT JUST ABOUT SUPPORT MATRIX. Today, many people are moving

    towards making the browser "just another client". Tools built around server-side HTML rendering face similar choices.