Plumbin' Pipelines with Gulp.js - 3hr Workshop

With JavaScript being no toy language anymore our demands on a high quality development environment have risen tremendously. One part of a good development environment are build tools, and Gulp.js -- being the JavaScript streaming build system -- is but one of the many choices a developer has nowadays. For many however, it's considered to be the best! In this three hour workshop, we will take a good look into Gulp and its possibilities. After our in-depth 3 hour course you will be able to do the following:

- Know how Gulp's API and the ecosystem of its plugins work
- Create parallel and sequential execution chains to be in total control of your build flow
- Know how 3rd party Node modules evolve around Gulp and how they can be integrated
- Create incremental builds to speed up your build time
- Know how streams work in Node.js
- And use stream arrays and merge streams to plumb together sophisticated pipelines doing all the work for you

Join us and become a build plumber!

More information
http://github.com/frontend-tooling
http://bit.ly/gulp-tool-book
http://fettblog.eu

Transcript

1. Plumbin' Pipelines
   with Gulp.js
   

   View Slide

2. @Dynatrace
   @Ruxit
   

   View Slide

3. @ddprrt
   fettblog.eu
   

   View Slide

4. HTML
   CSS
   JavaScript
   

   View Slide

5. Sass CoffeeScript LESS
   P o s t C S S H A M L J a d e
   U g l i f y E S 6 R e a c t J S
   B r o w s e r i f y A n g u l a r J S
   E m b e r C S S M i n J S L i n t
   ESHint ImageOptim Mocha
   Jasmine TypeScript
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. 1530 lines of code
    original Ant tasks used:
    concat — copy — delete — mkdir
    

    View Slide

11. View Slide

12. Let's talk a
    short bit about
    the JS build tool
    revolution
    

    View Slide

13. Grunt started a boom
    

    View Slide

14. Gruntfiles get long
    

    View Slide

15. Grunt tasks get slow
    

    View Slide

16. lots of reads
    and writes
    

    View Slide

17. And then came Gulp
    

    View Slide

18. Disclaimer
    

    View Slide

19. I (occasionally) contribute
    to Gulp
    

    View Slide

20. I'm writing a book on Gulp
    http://bit.ly/gulp-tool-book
    
    39% off with 39baumgar

    coupon code!
    

    View Slide

21. I don't know Java anymore
    Java is to JavaScript what

    Alf is to Gandalf
    

    View Slide

22. Let's go!
    

    View Slide

23. The ecosystem
    

    View Slide

24. View Slide

25. Runtime environment
    JavaScript
    

    View Slide

26. Package System for Node
    Host of all things Gulp plugins
    

    View Slide

27. CLI Gulpfile
    PluginA
    PluginA
    

    View Slide

28. CLI Gulpfile
    PluginA
    PluginA
    starts
    

    View Slide

29. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts
    

    View Slide

30. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts
    uses
    

    View Slide

31. npm install -g gulp-cli
    npm init
    npm install --save-dev gulp
    touch gulpfile.js
    

    View Slide

32. npm install -g gulpjs/gulp-cli#4.0
    npm init
    npm install --save-dev gulpjs/gulp#4.0
    touch gulpfile.js
    

    View Slide

33. The basics:
    Streams
    

    View Slide

34. gulp.src(…) gulp.dest(…)
    Reads files Writes files
    

    View Slide

35. gulp.src(…) gulp.dest(…)
    

    View Slide

36. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    

    View Slide

37. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    .pipe(concat())
    

    View Slide

38. to the command line!
    

    View Slide

39. Gulp API
    • gulp.task creates a new task
    
    • It requires to return either a Stream, a Promise or an Observable
    
    • gulp.src “globs” files and returns a stream of virtual file objects
    
    • each file can be piped through a process (jshint, uglify, less, etc.)
    
    • gulp.dest saves the file back to the file system
    

    View Slide

40. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    

    View Slide

41. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    defines a new task
    

    View Slide

42. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    with a defined name
    

    View Slide

43. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    we load a certain file
    (or files)
    Starting here, we have virtual
    files in-memory instead of
    real files we
    

    View Slide

44. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    and pipe it through a
    series of operations
    

    View Slide

45. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    before we save it
    again on the "real" file system
    

    View Slide

46. Tasks
    

    View Slide

47. scripts styles lint
    

    View Slide

48. scripts
    styles
    lint
    gulp.parallel
    

    View Slide

49. scripts
    styles
    lint
    gulp.parallel
    gulp.series
    

    View Slide

50. to the command line!
    

    View Slide

51. Gulp API
    • The second parameter of gulp.task is always a function.
    
    • gulp.series is a task function that runs tasks in sequential order.
    
    • gulp.parallel is a task function that starts every task concurrently
    
    • Both task functions accept task names and other functions as
    parameters.
    
    • They can be combined infinitly
    

    View Slide

52. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    

    View Slide

53. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in series
    

    View Slide

54. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in parallel
    

    View Slide

55. A development
    environment
    

    View Slide

56. gulp.watch
    scripts
    *.js
    

    View Slide

57. Bowser-Sync!
    r
    

    View Slide

58. to the command line!
    

    View Slide

59. Gulp API
    • gulp.watch creates a file watcher and listens to changes
    
    • changes include ‘change’, ‘add’, ‘unlink’ and others
    
    • BrowserSync is a development tool that can be fully integrated in
    Gulp.
    
    • Watchers trigger a browserSync.reload call
    

    View Slide

60. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    

    View Slide

61. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    watches this Glob pattern
    

    View Slide

62. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    starts this task on
    change, unlink, add
    

    View Slide

63. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    });
    

    View Slide

64. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    }); BrowserSync set up to start
    a dev server, serving dist and
    app statically
    

    View Slide

65. gulp.watch('dist/**/*', bSync.reload);
    And a watcher that triggers a
    reload
    

    View Slide

66. Incremental
    builds
    

    View Slide

67. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    

    View Slide

68. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    

    View Slide

69. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    Too much is going on!
    Each change: Uglify all
    the files?
    

    View Slide

70. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    

    View Slide

71. View Slide

72. filter files
    that have changed
    

    View Slide

73. filter files
    that have changed
    do performance
    heavy operations
    

    View Slide

74. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    

    View Slide

75. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    and continue
    with the other ops
    

    View Slide

76. to the command line!
    

    View Slide

77. Gulp Plugins
    • gulp-cached and gulp-remember can be used to create file caches
    
    • The plugin filters non-changed files and ads them back to the
    stream once we are done with performance-heavy tasks
    
    • Additionally to that, we can use gulp.lastRun in Gulp 4 to filter files
    during globbing
    
    • gulp-newer allows us to do incremental copies/builds on a per-file
    basis
    

    View Slide

78. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    

    View Slide

79. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    we use the cache to
    check if files have
    changed
    

    View Slide

80. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    once we are done,
    we remember all
    the other files
    we stored in
    the cache
    

    View Slide

81. Part II
    

    View Slide

82. A short
    source map interlude
    

    View Slide

83. Browserify
    (for Babel/React)
    

    View Slide

84. so similar … yet so
    different?
    Gulp
    Streams
    Browserify 

    Streams
    

    View Slide

85. Why not both??
    

    View Slide

86. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    

    View Slide

87. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    b.bundle emits a
    stream.
    But no vinyl file
    objects
    

    View Slide

88. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-source-stream
    wraps the original
    stream into
    a vinyl file object
    

    View Slide

89. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-buffer converts
    the stream contents
    to a buffer for
    plugins who need
    such
    

    View Slide

90. Stream arrays
    and merge Streams
    

    View Slide

91. A static
    site generator
    

    View Slide

92. What does it do?
    • Generates static HTML sites
    
    • From a templating engine
    
    • Can parse Markdown
    
    • Can parse HTML
    
    • Can create permalinks
    
    • For different types (posts, pages)
    

    View Slide

93. The stack
    • We use kramdown to convert Markdown to HTML
    
    • We use nujucks for our templating engine
    
    • We rename posts to feature blog permalink
    
    • We rename pages to resemble pretty URLs
    

    View Slide

94. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    

    View Slide

95. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    

    View Slide

96. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!
    

    View Slide

97. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    

    View Slide

98. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!
    

    View Slide

99. What if we could
    reuse parts of the
    stream?
    

    View Slide

100. to the command line!
     

     View Slide

101. to the command line!
     

     View Slide

102. gulp.task('default', function(cb) {
     var streams = elements.map(function(el) {
     return merge(
     gulp.src(el.dir + '/**.md').pipe(markdown()),
     gulp.src(el.dir + '/**.html')
     ).pipe(rename(el.renamefn));
     });
     return merge(streams).pipe(data(options))
     .pipe(wrap(layoutStr))
     .pipe(swig())
     .pipe(gulp.dest('build'));
     });
     

     View Slide

103. gulp.task('default', function(cb) {
     var streams = elements.map(function(el) {
     return merge(
     gulp.src(el.dir + '/**.md').pipe(markdown()),
     gulp.src(el.dir + '/**.html')
     ).pipe(rename(el.renamefn));
     });
     return merge(streams).pipe(data(options))
     .pipe(wrap(layoutStr))
     .pipe(swig())
     .pipe(gulp.dest('build'));
     });
     we combine
     multiple sources
     to one stream
     

     View Slide

104. gulp.task('default', function(cb) {
     var streams = elements.map(function(el) {
     return merge(
     gulp.src(el.dir + '/**.md').pipe(markdown()),
     gulp.src(el.dir + '/**.html')
     ).pipe(rename(el.renamefn));
     });
     return merge(streams).pipe(data(options))
     .pipe(wrap(layoutStr))
     .pipe(swig())
     .pipe(gulp.dest('build'));
     });
     with Array.map
     and merge we can
     create stream
     arrays
     

     View Slide

105. Material
     

     View Slide

106. Workshop files
     https://github.com/frontend-tooling
     
     https://github.com/frontend-tooling/sample-project-gulp
     
     https://github.com/frontend-tooling/static-site-generator
     
     http://fettblog.eu
     
     http://speakerdeck.com/ddprrt
     
     

     View Slide

107. Reading Material
     http://bit.ly/gulp-tool-book
     
     39% off with 39baumgar

     coupon code!
     

     View Slide

108. View Slide

109. @ddprrt
     

     View Slide