Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Plumbin' Pipelines with Gulp.js - 3hr Workshop

Plumbin' Pipelines with Gulp.js - 3hr Workshop

With JavaScript being no toy language anymore our demands on a high quality development environment have risen tremendously. One part of a good development environment are build tools, and Gulp.js -- being the JavaScript streaming build system -- is but one of the many choices a developer has nowadays. For many however, it's considered to be the best! In this three hour workshop, we will take a good look into Gulp and its possibilities. After our in-depth 3 hour course you will be able to do the following:

- Know how Gulp's API and the ecosystem of its plugins work
- Create parallel and sequential execution chains to be in total control of your build flow
- Know how 3rd party Node modules evolve around Gulp and how they can be integrated
- Create incremental builds to speed up your build time
- Know how streams work in Node.js
- And use stream arrays and merge streams to plumb together sophisticated pipelines doing all the work for you

Join us and become a build plumber!

More information
http://github.com/frontend-tooling
http://bit.ly/gulp-tool-book
http://fettblog.eu

Stefan Baumgartner

November 10, 2015
Tweet

More Decks by Stefan Baumgartner

Other Decks in Programming

Transcript

  1. Plumbin' Pipelines
    with Gulp.js

    View full-size slide

  2. @Dynatrace
    @Ruxit

    View full-size slide

  3. @ddprrt
    fettblog.eu

    View full-size slide

  4. HTML
    CSS
    JavaScript

    View full-size slide

  5. Sass CoffeeScript LESS
    P o s t C S S H A M L J a d e
    U g l i f y E S 6 R e a c t J S
    B r o w s e r i f y A n g u l a r J S
    E m b e r C S S M i n J S L i n t
    ESHint ImageOptim Mocha
    Jasmine TypeScript

    View full-size slide

  6. 1530 lines of code
    original Ant tasks used:
    concat — copy — delete — mkdir

    View full-size slide

  7. Let's talk a
    short bit about
    the JS build tool
    revolution

    View full-size slide

  8. Grunt started a boom

    View full-size slide

  9. Gruntfiles get long

    View full-size slide

  10. Grunt tasks get slow

    View full-size slide

  11. lots of reads
    and writes

    View full-size slide

  12. And then came Gulp

    View full-size slide

  13. I (occasionally) contribute
    to Gulp

    View full-size slide

  14. I'm writing a book on Gulp
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View full-size slide

  15. I don't know Java anymore
    Java is to JavaScript what

    Alf is to Gandalf

    View full-size slide

  16. The ecosystem

    View full-size slide

  17. Runtime environment
    JavaScript

    View full-size slide

  18. Package System for Node
    Host of all things Gulp plugins

    View full-size slide

  19. CLI Gulpfile
    PluginA
    PluginA

    View full-size slide

  20. CLI Gulpfile
    PluginA
    PluginA
    starts

    View full-size slide

  21. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts

    View full-size slide

  22. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts
    uses

    View full-size slide

  23. npm install -g gulp-cli
    npm init
    npm install --save-dev gulp
    touch gulpfile.js

    View full-size slide

  24. npm install -g gulpjs/gulp-cli#4.0
    npm init
    npm install --save-dev gulpjs/gulp#4.0
    touch gulpfile.js

    View full-size slide

  25. The basics:
    Streams

    View full-size slide

  26. gulp.src(…) gulp.dest(…)
    Reads files Writes files

    View full-size slide

  27. gulp.src(…) gulp.dest(…)

    View full-size slide

  28. gulp.src(…) .pipe(uglify()) gulp.dest(…)

    View full-size slide

  29. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    .pipe(concat())

    View full-size slide

  30. to the command line!

    View full-size slide

  31. Gulp API
    • gulp.task creates a new task

    • It requires to return either a Stream, a Promise or an Observable

    • gulp.src “globs” files and returns a stream of virtual file objects

    • each file can be piped through a process (jshint, uglify, less, etc.)

    • gulp.dest saves the file back to the file system

    View full-size slide

  32. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });

    View full-size slide

  33. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    defines a new task

    View full-size slide

  34. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    with a defined name

    View full-size slide

  35. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    we load a certain file
    (or files)
    Starting here, we have virtual
    files in-memory instead of
    real files we

    View full-size slide

  36. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    and pipe it through a
    series of operations

    View full-size slide

  37. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    before we save it
    again on the "real" file system

    View full-size slide

  38. scripts styles lint

    View full-size slide

  39. scripts
    styles
    lint
    gulp.parallel

    View full-size slide

  40. scripts
    styles
    lint
    gulp.parallel
    gulp.series

    View full-size slide

  41. to the command line!

    View full-size slide

  42. Gulp API
    • The second parameter of gulp.task is always a function.

    • gulp.series is a task function that runs tasks in sequential order.

    • gulp.parallel is a task function that starts every task concurrently

    • Both task functions accept task names and other functions as
    parameters.

    • They can be combined infinitly

    View full-size slide

  43. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );

    View full-size slide

  44. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in series

    View full-size slide

  45. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in parallel

    View full-size slide

  46. A development
    environment

    View full-size slide

  47. gulp.watch
    scripts
    *.js

    View full-size slide

  48. Bowser-Sync!
    r

    View full-size slide

  49. to the command line!

    View full-size slide

  50. Gulp API
    • gulp.watch creates a file watcher and listens to changes

    • changes include ‘change’, ‘add’, ‘unlink’ and others

    • BrowserSync is a development tool that can be fully integrated in
    Gulp.

    • Watchers trigger a browserSync.reload call

    View full-size slide

  51. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }

    View full-size slide

  52. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    watches this Glob pattern

    View full-size slide

  53. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    starts this task on
    change, unlink, add

    View full-size slide

  54. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    });

    View full-size slide

  55. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    }); BrowserSync set up to start
    a dev server, serving dist and
    app statically

    View full-size slide

  56. gulp.watch('dist/**/*', bSync.reload);
    And a watcher that triggers a
    reload

    View full-size slide

  57. Incremental
    builds

    View full-size slide

  58. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  59. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  60. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    Too much is going on!
    Each change: Uglify all
    the files?

    View full-size slide

  61. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  62. filter files
    that have changed

    View full-size slide

  63. filter files
    that have changed
    do performance
    heavy operations

    View full-size slide

  64. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files

    View full-size slide

  65. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    and continue
    with the other ops

    View full-size slide

  66. to the command line!

    View full-size slide

  67. Gulp Plugins
    • gulp-cached and gulp-remember can be used to create file caches

    • The plugin filters non-changed files and ads them back to the
    stream once we are done with performance-heavy tasks

    • Additionally to that, we can use gulp.lastRun in Gulp 4 to filter files
    during globbing

    • gulp-newer allows us to do incremental copies/builds on a per-file
    basis

    View full-size slide

  68. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });

    View full-size slide

  69. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    we use the cache to
    check if files have
    changed

    View full-size slide

  70. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    once we are done,
    we remember all
    the other files
    we stored in
    the cache

    View full-size slide

  71. A short
    source map interlude

    View full-size slide

  72. Browserify
    (for Babel/React)

    View full-size slide

  73. so similar … yet so
    different?
    Gulp
    Streams
    Browserify 

    Streams

    View full-size slide

  74. Why not both??

    View full-size slide

  75. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }

    View full-size slide

  76. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    b.bundle emits a
    stream.
    But no vinyl file
    objects

    View full-size slide

  77. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-source-stream
    wraps the original
    stream into
    a vinyl file object

    View full-size slide

  78. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-buffer converts
    the stream contents
    to a buffer for
    plugins who need
    such

    View full-size slide

  79. Stream arrays
    and merge Streams

    View full-size slide

  80. A static
    site generator

    View full-size slide

  81. What does it do?
    • Generates static HTML sites

    • From a templating engine

    • Can parse Markdown

    • Can parse HTML

    • Can create permalinks

    • For different types (posts, pages)

    View full-size slide

  82. The stack
    • We use kramdown to convert Markdown to HTML

    • We use nujucks for our templating engine

    • We rename posts to feature blog permalink

    • We rename pages to resemble pretty URLs

    View full-size slide

  83. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View full-size slide

  84. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View full-size slide

  85. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!

    View full-size slide

  86. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View full-size slide

  87. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!

    View full-size slide

  88. What if we could
    reuse parts of the
    stream?

    View full-size slide

  89. to the command line!

    View full-size slide

  90. to the command line!

    View full-size slide

  91. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });

    View full-size slide

  92. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });
    we combine
    multiple sources
    to one stream

    View full-size slide

  93. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });
    with Array.map
    and merge we can
    create stream
    arrays

    View full-size slide

  94. Workshop files
    https://github.com/frontend-tooling

    https://github.com/frontend-tooling/sample-project-gulp

    https://github.com/frontend-tooling/static-site-generator

    http://fettblog.eu

    http://speakerdeck.com/ddprrt

    View full-size slide

  95. Reading Material
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View full-size slide