Slide 1

Slide 1 text

Open-source as a project model for internal work opentalk.klamp.in Welcome!

Slide 2

Slide 2 text

Front-end Developer. Mostly.

Slide 3

Slide 3 text

Currently work at InVision

Slide 4

Slide 4 text

This talk stems from the work I was part of at USAA and Rackspace, two very large companies. Not a representative of either company

Slide 5

Slide 5 text

While at USAA, I’d have frequent conversations about infrastructure work with co-workers

Slide 6

Slide 6 text

In our role at the time, we were tasked with developing and supporting an infrastructure framework for over 300 developers. His focus was on the server-side, mine on the front-end. Supporting a growing team is tough

Slide 7

Slide 7 text

Supporting a growing team is tough After a certain number of users, you won't be able to keep up with the constant flow of "drive-by" questions and e-mails. You must balance time spent supporting your fellow teams with time spent developing new features.

Slide 8

Slide 8 text

Teams and project will always be understaffed and there will always be more to do than people to do it

Slide 9

Slide 9 text

Most projects are understaffed You can’t waste your users time (both consuming and contributing). Consumers of your project will likely be very busy with their own work and will have little time for contributing back.

Slide 10

Slide 10 text

While we can easily support those around us

Slide 11

Slide 11 text

Even co-located teams face physical constraints

Slide 12

Slide 12 text

Even co-located teams face physical constraints You can’t be everywhere at once, or have everyone next to you at once. At a certain size, you're essentially working as a distributed team due to space constraints.

Slide 13

Slide 13 text

While mulling on these issues, we became interested in how open-source projects run their show. jQuery has hundreds of thousands of users and contributors from all over the world who mostly work for free.

Slide 14

Slide 14 text

Open-source projects have to be good at scaling How do they support their ever growing user base while staying adaptable? A projects success can also be its demise

Slide 15

Slide 15 text

Open-source projects have to rely mostly on volunteers How do they get people to volunteer their time? OSS doesn’t have a big budget

Slide 16

Slide 16 text

Open-source projects have to have distributed teams Most open-source projects don't have dedicated, co-located teams. How do they cope?

Slide 17

Slide 17 text

Yet they manage. Quite well in fact. While there are a million OS projects that fail, many are outstanding successes, because they overcome the problems they face and are stronger for it.

Slide 18

Slide 18 text

How to foster an Open-source culture at work? So can we foster a more open, maintainable and friendly project by taking an "open-source" approach at work?

Slide 19

Slide 19 text

Reduce Friction to Contribute How do you get contributions from people who are likely already overworked?

Slide 20

Slide 20 text

Practice empathetic code management How can you continue to evolve your product without causing endless headaches for those who have helped you get to where you are?

Slide 21

Slide 21 text

Embrace the human community at work Just because it's work, doesn't mean it's not human. How do you build a community that’s supportive and encouraging and avoid the politics?

Slide 22

Slide 22 text

Reduce the friction to contribute Everyone is busy. No one has time to waste. Open-source projects face this trouble every day. They don’t pay most of their contributors. They have to rely on their contributors taking time out of their day. Let’s do the same at work.

Slide 23

Slide 23 text

"Better to have people think you're a fool than to open your mouth and remove all doubt." Maybe have been lincoln that said this. May have been Mark Twain

Slide 24

Slide 24 text

I know it was said by wilson. w. wilson though on Home Improvement though. Idea is, assume the developers are uncomfortable/intimidated when trying to contribute code.

Slide 25

Slide 25 text

The #1 friction is fear Fear of looking foolish Fear of breaking something Those who are fearless suffer from the Dunning–Kruger effect

Slide 26

Slide 26 text

Use automation to prevent dumb mistakes

Slide 27

Slide 27 text

It’s nicer to hear from a bot that you’re wrong than a human. As a human, you’re less strict than bots, which hurts in the long run.

Slide 28

Slide 28 text

"pre-commit": [ "grunt lint" ], "pre-push": [ "grunt build", "grunt test" ] makes PRs much easier b/c you’re not having to be nitpick

Slide 29

Slide 29 text

Automate testing Provides newcomers with a set of requirements to test their code changes against Allows much more confidence for newcomers that their changes haven't broken anything Unit, Midway, Visual Makes your job easier, as you can have a better trust that changes won’t break stuff.

Slide 30

Slide 30 text

Let developers mess up Spend time trying to prevent it, but also spend time planning for recovery from mistakes. Expect failure. Plan for it.

Slide 31

Slide 31 text

Write docs, but don't expect them to be read It’s tough to read all the documentation out there and not everyone has time. Docs should be more useful to the main dev team, as a quick way of pointing people in the right direction. Saves you the trouble of writing the same email explaining the same concept again and again. Be friendly with your RTFM.

Slide 32

Slide 32 text

An example is worth 1000 technical words

Slide 33

Slide 33 text

$ ack .tag trying to find a class

Slide 34

Slide 34 text

staging

returned invalid results

Slide 35

Slide 35 text

staging

because it was treating it as a regex

Slide 36

Slide 36 text

ack [options] PATTERN [FILE...] WHY ARE YOU YELLING AT ME? What is pattern? File…? What about searching folders?

Slide 37

Slide 37 text

Options -a, --all Operate on all files, regardless of type (but still skip directories like blib, CVS , etc.) -A NUM , --after-context= NUM Print NUM lines of trailing context after matching lines. -B NUM , --before-context= NUM Print NUM lines of leading context before matching lines. -C [ NUM ], --context[= NUM ] Print NUM lines (default 2) of context around matching lines. -c, --count Suppress normal output; instead print a count of matching lines for each input file. If -l is in effect, it will only show the number of lines for each file that has lines matching. Without -l, some line counts may be zeroes. If combined with -h (--no-filename) ack outputs only one total count. --color, --nocolor --color highlights the matching text. --nocolor supresses the color. This is on by default unless the output is redirected. On Windows, this option is off by default unless the Win32::Console::ANSI module is installed or the "ACK_PAGER_COLOR" environment variable is used. --color-filename=color Sets the color to be used for filenames. --color-match=color Sets the color to be used for matches. --color-lineno=color Sets the color to be used for line numbers. --column Show the column number of the first match. This is helpful for editors that can place your cursor at a given position. --env, --noenv --noenv disables all environment processing. No .ackrc is read and all environment variables are ignored. By default, ack considers .ackrc and settings in the environment. --flush --flush flushes output immediately. This is off by default unless ack is running interactively (when output goes to a pipe or file). -f Only print the files that would be searched, without actually doing any searching. PATTERN must not be specified, or it will be taken as a path to search. --follow, --nofollow Follow or don't follow symlinks, other than whatever starting files or directories were specified on the command line. This is off by default. -G REGEX Only paths matching REGEX are included in the search. The entire path and filename are matched against REGEX , and REGEX is a Perl regular expression, not a shell glob. The options -i, -w, -v, and -Q do not apply to this REGEX . -g REGEX Print files where the relative path + filename matches REGEX . This option is a convenience shortcut for -f -G REGEX . The options -i, -w, -v, and -Q do not apply to this REGEX . --group, --nogroup --group groups matches by file name with. This is the default when used interactively. --nogroup prints one result per line, like grep. This is the default when output is redirected. -H, --with-filename Print the filename for each match. -h, --no-filename Suppress the prefixing of filenames on output when multiple files are searched. --help Print a short help statement. -i, --ignore-case Ignore case in the search strings. This applies only to the PATTERN , not to the regexes given for the -g and -G options. --[no]ignore-dir= DIRNAME Ignore directory (as CVS , .svn, etc are ignored). May be used multiple times to ignore multiple directories. For example, mason users may wish to include --ignore-dir=data. The --noignore-dir option allows users to search directories which would normally be ignored (perhaps to research the contents of .svn/props directories). The DIRNAME must always be a simple directory name. Nested directories like foo/bar are NOT supported. You would need to specify --ignore-dir=foo and then no files from any foo directory are taken into account by ack unless given explicitly on the command line. --line= NUM Only print line NUM of each file. Multiple lines can be given with multiple --line options or as a comma separated list (--line=3,5,7). --line=4-7 also works. The lines are always output in ascending order, no matter the order given on the command line. -l, --files-with-matches Only print the filenames of matching files, instead of the matching text. -L, --files-without-matches Only print the filenames of files that do NOT match. This is equivalent to specifying -l and -v. --match REGEX Specify the REGEX explicitly. This is helpful if you don't want to put the regex as your first argument, e.g. when executing multiple searches over the same set of files. # search for foo and bar in given files ack file1 t/file* --match foo ack file1 t/file* --match bar -m= NUM , --max-count= NUM Stop reading a file after NUM matches. --man Print this manual page. -n, --no-recurse No descending into subdirectories. -o Show only the part of each line matching PATTERN (turns off text highlighting) --output=expr Output the evaluation of expr for each line (turns off text highlighting) --pager=program Direct ack's output through program. This can also be specified via the "ACK_PAGER" and "ACK_PAGER_COLOR" environment variables. Using --pager does not suppress grouping and coloring like piping output on the command-line does. --passthru Prints all lines, whether or not they match the expression. Highlighting will still work, though, so it can be used to highlight matches while still seeing the entire file, as in: # Watch a log file, and highlight a certain IP address $ tail -f ~/access.log | ack --passthru 123.45.67.89 --print0 Only works in conjunction with -f, -g, -l or -c (filename output). The filenames are output separated with a null byte instead of the usual newline. This is helpful when dealing with filenames that contain whitespace, e.g. # remove all files of type html ack -f --html --print0 | xargs -0 rm -f -Q, --literal Quote all metacharacters in PATTERN , it is treated as a literal. This applies only to the PATTERN , not to the regexes given for the -g and -G options. -r, -R, --recurse Recurse into sub-directories. This is the default and just here for compatibility with grep. You can also use it for turning --no-recurse off. --smart-case, --no-smart-case Ignore case in the search strings if PATTERN contains no uppercase characters. This is similar to "smartcase" in vim. This option is off by default. -i always overrides this option. This applies only to the PATTERN , not to the regexes given for the -g and -G options. --sort-files Sorts the found files lexically. Use this if you want your file listings to be deterministic between runs of ack. --show-types Outputs the filetypes that ack associates with each file. Works with -f and -g options. --thpppt Display the all-important Bill The Cat logo. Note that the exact spelling of --thpppppt is not important. It's checked against a regular expression. --type=TYPE, --type=noTYPE Specify the types of files to include or exclude from a search. TYPE is a filetype, like perl or xml. --type=perl can also be specified as --perl, and --type=noperl can be done as --noperl. If a file is of both type "foo" and "bar", specifying --foo and --nobar will exclude the file, because an exclusion takes precedence over an inclusion. Type specifications can be repeated and are ORed together. See ack --help=types for a list of valid types. --type-add TYPE =.EXTENSION[,.EXT2[,...]] Files with the given EXTENSION (s) are recognized as being of (the existing) type TYPE . See also "Defining your own types". --type-set TYPE =.EXTENSION[,.EXT2[,...]] Files with the given EXTENSION (s) are recognized as being of type TYPE . This replaces an existing definition for type TYPE . See also "Defining your own types". -u, --unrestricted All files and directories (including blib/, core.*, ...) are searched, nothing is skipped. When both -u and --ignore-dir are used, the --ignore-dir option has no effect. -v, --invert-match Invert match: select non-matching lines This applies only to the PATTERN , not to the regexes given for the -g and -G options. --version Display version and copyright information. -w, --word-regexp Force PATTERN to match only whole words. The PATTERN is wrapped with "\b" metacharacters. This applies only to the PATTERN , not to the regexes given for the -g and -G options. -1 Stops after reporting first match of any kind. This is different from --max-count=1 or -m1, where only one match per file is shown. Also, -1 works with -f and -g, where -m does not. Sure, I could have read through the man page and viewed all the options.

Slide 38

Slide 38 text

$ ack simpleTerm $ ack “\.escape” $ ack -w onlywords show me examples instead. Lots and lots of examples

Slide 39

Slide 39 text

Build a demo site. Twitter bootstrap, etc http://git-man-page-generator.lokaltog.net/

Slide 40

Slide 40 text

Good variable naming & organization is a TL;DR for your code Focus on readable code over clever/small code

Slide 41

Slide 41 text

Code comments https://twitter.com/nzkoz/status/538892801941848064 Credit goes to Michael Koziarski (@nzkoz) Keep them up-front and center, so as to make it obvious when they're wrong. Don't hide docs in a wiki where people have to search to find them Apply DRY rules to docs as well, otherwise. JSDoc has this problem

Slide 42

Slide 42 text

Know that many people want to help, but won't Time is limited, making a contribution to a big framework takes a lot of time that people with project deadlines don't have Don't fret too much about it. Be opening and encouraging, but not pushy Don't try gimmicks to get contributions. Contests only serve to undermine efforts -> Pumpkin contest, 4 entries out of ~200 people despite amazing prizes. People didn’t trust they could win, so they didn’t risk the effort.

Slide 43

Slide 43 text

Empathetic Code Management

Slide 44

Slide 44 text

better != together Contrary to what Jack Johnson sings, it’s not always better when we’re together may work when small, but won’t scale up. Longer release cycles regression testing everything Responsibility becomes vague. Who fixes the bugs? What if team A breaks team B’s code with update?

Slide 45

Slide 45 text

App includes set of common features - site template, - re-usable components, - common styles "infrastructure code” Common to live together with the functional codebase

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

Starks have been working 3 weeks on this new feature. A feature that requires some common code to behave a specific way. They’re a day away from releasing…

Slide 51

Slide 51 text

Unfortunately, yesterday, lannisters released update w/modification to code that Team stark was expecting to work a specific way. Except it no longer does

Slide 52

Slide 52 text

Now team in scramble mode They hadn’t told the lannisters about their work (why would they need to know)?

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

Lannister already merged and shipped the code.

Slide 55

Slide 55 text

Team is relying on those changes for the code that they just shipped. Sooo….

Slide 56

Slide 56 text

At this point, people start bracing for a long weekend of refactoring code.

Slide 57

Slide 57 text

The king is just “work the weekend m’kay” and we’ll do a “tiger team” to discuss this on 8am monday morning

Slide 58

Slide 58 text

The Case for a Separate Infrastructure This could be avoided. Issue was two-fold No one responsible for infrastructure code and communication. infrastructure code being “pushed” out to teams

Slide 59

Slide 59 text

Separate team Separate codebase Need separate infrastructure team, even if one person, to manage changes. Need a separate codebase to manage upgrades

Slide 60

Slide 60 text

Most of the time, the common codebase gets pushed out to all the apps on update. You want to get features out there, right?

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

But it’s a bad idea, and the open source world has figured that out. You will break prod with this model. Again and again and again.

Slide 63

Slide 63 text

Pull your updates Allow the apps the room to "pull" in the code when their ready. This isn't an uncommon model. Almost all open-source projects out there follow it.

Slide 64

Slide 64 text

Teams go to the common repo when they’re ready to upgrade. So app a can be upgraded separate from app b

Slide 65

Slide 65 text

Then when app b has finished their dev cycle, they can upgrade on their own time. Package managers like NPM follow this model (unless you're using the latest flag, which again, you probably shouldn't be)

Slide 66

Slide 66 text

Lower risk of breaking production with a release Teams able to fully regression test code, at their own pace (no weekends) Test changes with small pilot groups If something breaks due to the release, each team has option to delay upgrade until later Ability to rollback an update because infrastructure is versioned

Slide 67

Slide 67 text

Faster iteration due to smaller testing efforts Able to release code quicker. Shortened release cycles, each version change will be more bite-sized, allowing teams an easier upgrade. Doesn't mean code isn't tested. infrastructure team should have regression tests they run. Also doesn't mean app teams on their own, or that the infrastructure team is free to make any change they want. Teams should be there for each other (empathy)

Slide 68

Slide 68 text

Slower propagation of features Drawbacks: takes effort for a team to pull down code, compared to it automatically being pushed to them trade-off of the model and needs to be understood and accepted as part of the deal. Easy to blame pull model, therefore important to look deeper at the problem before abandonment

Slide 69

Slide 69 text

Visual Inconsistencies hardest pill to swallow, effects app end-user If your infrastructure code includes visual design, splintering versions means having an inconsistent experience across apps which are supposed to look similar.

Slide 70

Slide 70 text

Plan Ahead by Bundling Visual Changes If you know you have a series of visual updates that need consistency across the site, go through the effort to plan a combined release of these changes. Along with this, get in touch with the app teams and see if you can do a coordinated release for all of the apps. It does mean letting go of the benefits described above, but hopefully this isn't a common occurance.

Slide 71

Slide 71 text

Passively communicate Give information for passively, to avoid spamming inboxes and to save time

Slide 72

Slide 72 text

https://twitter.com/sosowski/status/517005067334205441 Many ways to do versioning, including the microsoft way. Other’s include dates. But what if we can send information for free with every update?

Slide 73

Slide 73 text

Semantic Versioning

Slide 74

Slide 74 text

Major.Minor.Patch MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards-compatible manner, and PATCH version when you make backwards-compatible bug fixes.

Slide 75

Slide 75 text

0.10.35 -> 0.10.36 Trust, but verify

Slide 76

Slide 76 text

0.10.36 -> 0.12.0 Expect a little bit of work

Slide 77

Slide 77 text

http://semver.org/ In-depth thoughts on how to manage versioning

Slide 78

Slide 78 text

Node Stability Index Allows you to introduce new components with a disclaimer. Add index with component documentation.

Slide 79

Slide 79 text

0 - Deprecated 1 - Experimental 2 - Unstable 3 - Stable 4 - API Frozen 5 - Locked Can have an experimental component live next to a stable one Allows more flexibility

Slide 80

Slide 80 text

Update them regularly Update regularly, otherwise it’s meaningless and not worth it. Mgmt might also not like it b/c it reveals a certain amount of hesitation around code :/

Slide 81

Slide 81 text

keepachangelog.com Change log is a file which contains a curated, chronologically ordered list of notable changes for each version of an open source project. Make it easier for users and contributors to see precisely what notable changes have been made between each release (or version) of the project. Highlight breaking changes Auto-generate changelogs from commit messages. Side benefit is encouraging useful commit messages. Angular.js has an interesting approach to formatting their messages. They follow this model:

Slide 82

Slide 82 text

typeOfChange(component): Description of change Angular follows as commit message format. Where changeType can be ‘feat’, ’style’, ‘tests’, 'fix',

Slide 83

Slide 83 text

feat(ruler): add inches as well as centimeters

Slide 84

Slide 84 text

fix(pen): use blue ink instead of red ink BREAKING CHANGE: Pen now uses blue ink instead of red. To migrate, change your code from the following: `pen.draw('blue')` To: `pen.draw('red')` Can include breaking change details

Slide 85

Slide 85 text

Generates a markdown file like this

Slide 86

Slide 86 text

lmgtfy.com/?q= changelog+generator github.com/skywinder/github-changelog-generator github.com/ajoslin/conventional-changelog

Slide 87

Slide 87 text

Embrace the human community at work

Slide 88

Slide 88 text

Don't get good at answering e-mails & attending meetings These behaviors build silos by keeping communication hidden. Unless publicly recorded, meetings and emails are “closed- source”. Meetings & emails don’t scale.

Slide 89

Slide 89 text

Silos encourage politics & discourage outside voices How is someone supposed to help when they don’t know the details? Communicate in the open, via github, chat rooms, message boards

Slide 90

Slide 90 text

Have an open, public chat room Slack/Hipchat/gitter.im Chat history that’s searchable Easier for quick discussion. Team can help out, instead of an individual. People can lurk.

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

Publicize project status Work as if you’ll never meet your users face-to-face in real life

Slide 93

Slide 93 text

workingon.co, waffle.io, trello.com

Slide 94

Slide 94 text

Keep an ever-growing backlog Don't fear an endless backlog. It's okay to have too much work to do. It’s better to track the work than forget it.

Slide 95

Slide 95 text

Fit in time to write Blog posts. Weekly ‘write ups’. A-ha’s. Keep a history of why decisions were made, you'll forget why a month later otherwise when someone asks.

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

Set timelines for Code Reviews People are impatient and that’s okay. If they take the time to reach out to you, set expectations. If you know you don't have time to review a PR right away, make a quick comment saying so. Everyone understands people get busy Going 24 hours before reviewing code is not unreasonable to ask for small code reviews. Bigger PRs require more time, and likely more cycles of feedback. Make sure both sides understand this. Same for emails

Slide 98

Slide 98 text

Reduce variability, yet embrace it Reducing is good for things like code syntax/style, release cycles, code standards. Embracing is good for things like experimental work, new libraries/techniques/languages

Slide 99

Slide 99 text

Define community guidelines sass-lang.com/community-guidelines The best OSS projects are driven by their community. Go to a conference for Drupal or Angular or React or Sass and people will be most excited by the community around them. Foster that with guidelines that clearly define acceptable behavior.

Slide 100

Slide 100 text

You want thinkers, not doers contributors are volunteers when it comes to their thoughts. You can't force good behavior. You have to foster it.

Slide 101

Slide 101 text

1) Reduce the friction to contribute 2) Practice empathetic code management 3) Embrace the Human Community at Work The problems that OSS projects face mirror the same problems workplaces face and revolve more around human interaction than the code itself. By following the lead of what the open-source world has done, you can ensure a more open, maintainable and friendly project, even if the work you do is closed-source.

Slide 102

Slide 102 text

Thanks! open.klamp.in @klamping @gmail. com

Slide 103

Slide 103 text

Thanks! open.klamp.in @klamping @gmail. com