Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Oral tradition in software engineering: Passing the craft across generations

Bryan Cantrill
October 06, 2016
33

Oral tradition in software engineering: Passing the craft across generations

My presentation at Monktoberfest 2016. Video: https://www.youtube.com/watch?v=4PaWFYm0kEw

Bryan Cantrill

October 06, 2016
Tweet

Transcript

  1. Oral tradition in software engineering:
    Passing the craft across generations
    CTO
    [email protected]
    Bryan Cantrill
    @bcantrill

    View full-size slide

  2. Oral tradition
    • Long before humans developed systems of writing, we were
    leaving legacies of information via oral tradition
    • Oral tradition has been used to:
    • Preserve and explain cultural values
    • Convey skills and wisdom
    • Entertain as well as enlighten
    • Oral tradition continues to play an important role in post-
    literate societies — including in software engineering

    View full-size slide

  3. Oral tradition in software engineering
    • In the beginning, software was subservient to hardware
    • Software rarely enjoyed the levels of design documentation
    found in computer design
    • Like true oral tradition, it relied primarily upon oral
    retransmission — and much of this is lost today
    • Fred Brooks’ Mythical Man Month captures the tribal wisdom
    of this antediluvian era of software

    View full-size slide

  4. Oral literature
    • When oral tradition is distilled into clear narrative and reliably
    repeated, it becomes oral literature
    • For us in software engineering, the early Internet allowed a
    way to reliably share stories via USENET
    • USENET is filled with incredible and interesting narrative, but
    most survives only in archive
    • But one story from USENET does survive — and it has
    become the epic of software engineering…

    View full-size slide

  5. Our epic, as penned by Ed Nather
    A recent article devoted to the macho side of programming
    made the bald and unvarnished statement:
    Real Programmers write in FORTRAN.
    Maybe they do now,
    in this decadent era of
    Lite beer, hand calculators, and “user-friendly” software
    but back in the Good Old Days,
    when the term “software” sounded funny
    and Real Computers were made out of drums and vacuum tubes,
    Real Programmers wrote in machine code.
    Not FORTRAN. Not RATFOR. Not, even, assembly language.
    Machine Code.
    Raw, unadorned, inscrutable hexadecimal numbers.
    Directly.

    View full-size slide

  6. …and our epic hero, Mel Kaye
    Lest a whole new generation of programmers
    grow up in ignorance of this glorious past,
    I feel duty-bound to describe,
    as best I can through the generation gap,
    how a Real Programmer wrote code.
    I'll call him Mel,
    because that was his name.

    View full-size slide

  7. Our Odyssey
    • The Story of Mel was recorded in the early 1980s, but dates
    from the earliest days of software in the 1960s
    • Another set of stories recorded in the late 1970s at Data
    General has become software engineering lore…
    • Tracy Kidder’s Soul of a New Machine has wisdom that has
    long survived the team and company it describes
    • If Mel is our Illiad, Soul is our Odyssey

    View full-size slide

  8. The 1980s and 1990s
    • In the deeply proprietary era of the 1980s and 1990s, oral
    tradition was locked on proprietary islands — tribes privately
    passed on their own traditions and practices
    • Collections of oral tradition like sgi.badattitude were the stuff
    of legend — but they too have been lost
    • The canonical folklore from this era may be John Browne’s
    beautiful The Bug Count Also Rises, the winner of an internal
    Microsoft Hemingway parody contest (!)

    View full-size slide

  9. Dawn of a new era: November 7, 2005

    View full-size slide

  10. Oral tradition post-YouTube
    • With the rise of Internet video, story — orally delivered and
    aurally received — could travel unencumbered
    • Stories could be suddenly retold simply by sharing a link —
    with all of the force and authority of the original
    • Oral tradition can spread so quickly, we explain it in terms of a
    pandemic: stories that become viral
    • We in software were particularly lucky in that Google Tech
    Talks managed to capture much oral history and tradition
    before participants recognized what it meant to be recorded

    View full-size slide

  11. For me personally, a watershed

    View full-size slide

  12. For me personally, a watershed (ahem!)
    DTrace Review

    View full-size slide

  13. Oral tradition post-YouTube
    • Post-YouTube, our oral tradition is suffering from overload —
    we have an embarrassment of riches!
    • The true oral tradition has shifted from the content to the
    meta-content, e.g. this evergreen topic on HN:

    View full-size slide

  14. Oral tradition closer to home
    • Every source base is awash in narrative: of origin, of struggle,
    of conflict, of rewrite, of success and of failure
    • These narratives ultimately convey not only the history but
    also the values of a source base
    • In the proprietary era, these were purely oral tradition —
    something to regale new hires with over lunch
    • Every source base is loaded with these; several that are
    relevant to mine… (UNIX Unix 6th Edition 7th Edition SunOS
    System III SVR4 Solaris OpenSolaris illumos/SmartOS!)

    View full-size slide

  15. I: The untimely death of Joe Ossanna
    • In 1973, Joe Ossanna at Bell Labs wrote troff in PDP-11
    assembler for the Graphic Systems CAT typesetter
    • In 1975, Ossanna ported it to C — with the intent of a later
    clean rewrite to make the macro language less alien
    • In 1977, Ossanna died suddenly, of either a heart attack, a
    car accident or (most graphically) being struck by a car
    • The code he left behind — which remains in our source base
    — is inscrutable; Brian Kernighan failed to rewrite it!
    • Code survives us

    View full-size slide

  16. II: “You are not expected to understand this”
    • Perhaps the most (in)famous comment in Sixth Edition:



    /*

    * If the new process paused because it was

    * swapped out, set the stack level to the last call

    * to savu(u_ssav). This means that the return

    * which is executed immediately after the call to aretu

    * actually returns from the last routine which did

    * the savu.

    *

    * You are not expected to understand this.

    */

    if(rp->p_flag&SSWAP) {

    rp->p_flag =& ~SSWAP;

    aretu(u.u_ssav);

    }
    • That which we can’t explain, we don’t understand!

    View full-size slide

  17. III: Logical XOR
    • I have always been enraged disappointed by C’s lack of
    logical XOR, having had a hard time holding my tongue, e.g.:



    /*

    * Logical XOR, where are you?

    */

    ASSERT((vstate->dtvs_nglobals == 0) ^ (vstate->dtvs_globals != NULL));
    • I became involved in a heated argument with Roger Faulkner,
    who insisted that Dennis didn’t add logical XOR because “it
    can’t short-circuit” — which is obviously stupid
    • Important difference between Roger and me: Roger had
    worked at Bell Labs with Dennis…

    View full-size slide

  18. III: “Isn’t that much worse to write” (!)
    From [email protected] Sun Aug 6 23:33:54 2000

    To: [email protected]
    Subject: Re: history question


    Hi Roger,


    back from some vacation; here's a belated response.

    ^^ was omitted because it is of little use, particularly

    because the "shortcut evaluation" guaranteed for || and &&

    doesn't help--both sides have to be evaluated no

    matter what. When you do need the effect,

    (A!=0) ^ (B!=0) isn't that much worse to write.


    Dennis

    View full-size slide

  19. IV: “You think this is cheesy?”
    • A favorite comment in the operating system is one that Jeff
    Bonwick and I wrote together in a moment of crisis
    • We had (finally!) figured a narrow path by which we could
    solve a devastating problem that seemed insurmountable
    • The details are highly technical, but documented thoroughly
    in code comments — the context around it in an abstruse
    chapter I wrote in Beautiful Code…

    View full-size slide

  20. IV: “You think this is cheesy?”
    /*
    * The turnstile hash table is partitioned into two halves: the lower half
    * is used for upimutextab[] locks, the upper half for everything else.
    * The reason for the distinction is that SOBJ_USER_PI locks present a
    * unique problem: the upimutextab[] lock passed to turnstile_block()
    * cannot be dropped until the calling thread has blocked on its
    * SOBJ_USER_PI lock and willed its priority down the blocking chain.
    * At that point, the caller's t_lockp will be one of the turnstile locks.
    * If mutex_exit() discovers that the upimutextab[] lock has waiters, it
    * must wake them, which forces a lock ordering on us: the turnstile lock
    * for the upimutextab[] lock will be acquired in mutex_vector_exit(),
    * which will eventually call into turnstile_pi_waive(), which will then
    * acquire the caller's thread lock, which in this case is the turnstile
    * lock for the SOBJ_USER_PI lock. In general, when two turnstile locks
    * must be held at the same time, the lock order must be the address order.
    * Therefore, to prevent deadlock in turnstile_pi_waive(), we must ensure
    * that upimutextab[] locks *always* hash to lower addresses than any
    * other locks. You think this is cheesy? Let's see you do better.
    */

    View full-size slide

  21. Code comments as oral tradition?
    • The best comments seem to have an aspect of oral tradition
    to them: they not only serve to explain what is going on, but
    provide a narrative as to why
    • Comments live with the code but do not otherwise affect its
    correctness — allowing for broad scope and artistic license
    • Relevant, correct software systems will survive long past their
    creators; comments are the art within the timeless machine
    • Comments are how we speak to future generations!

    View full-size slide

  22. Oral tradition in software engineering
    • Software engineering has changed radically in the last
    decade: it’s now open, transparent and distributed
    • This has forced a revolution in oral tradition, which has
    increasingly become recorded, yielding oral literature
    • Oral literature allows us to expose and preserve the narrative
    behind the artifact — to explain the why of the thing
    • Software engineering teams should be encouraged to record
    their oral tradition: it attracts technologists who share values!
    • Our oral tradition is part of our legacy to future generations!

    View full-size slide