Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Plumbin' Pipelines with Gulp.js - 3hr Workshop

Plumbin' Pipelines with Gulp.js - 3hr Workshop

With JavaScript being no toy language anymore our demands on a high quality development environment have risen tremendously. One part of a good development environment are build tools, and Gulp.js -- being the JavaScript streaming build system -- is but one of the many choices a developer has nowadays. For many however, it's considered to be the best! In this three hour workshop, we will take a good look into Gulp and its possibilities. After our in-depth 3 hour course you will be able to do the following:

- Know how Gulp's API and the ecosystem of its plugins work
- Create parallel and sequential execution chains to be in total control of your build flow
- Know how 3rd party Node modules evolve around Gulp and how they can be integrated
- Create incremental builds to speed up your build time
- Know how streams work in Node.js
- And use stream arrays and merge streams to plumb together sophisticated pipelines doing all the work for you

Join us and become a build plumber!

More information
http://github.com/frontend-tooling
http://bit.ly/gulp-tool-book
http://fettblog.eu

Stefan Baumgartner

November 10, 2015
Tweet

More Decks by Stefan Baumgartner

Other Decks in Programming

Transcript

  1. Plumbin' Pipelines
    with Gulp.js

    View Slide

  2. @Dynatrace
    @Ruxit

    View Slide

  3. @ddprrt
    fettblog.eu

    View Slide

  4. HTML
    CSS
    JavaScript

    View Slide

  5. Sass CoffeeScript LESS
    P o s t C S S H A M L J a d e
    U g l i f y E S 6 R e a c t J S
    B r o w s e r i f y A n g u l a r J S
    E m b e r C S S M i n J S L i n t
    ESHint ImageOptim Mocha
    Jasmine TypeScript

    View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. 1530 lines of code
    original Ant tasks used:
    concat — copy — delete — mkdir

    View Slide

  11. View Slide

  12. Let's talk a
    short bit about
    the JS build tool
    revolution

    View Slide

  13. Grunt started a boom

    View Slide

  14. Gruntfiles get long

    View Slide

  15. Grunt tasks get slow

    View Slide

  16. lots of reads
    and writes

    View Slide

  17. And then came Gulp

    View Slide

  18. Disclaimer

    View Slide

  19. I (occasionally) contribute
    to Gulp

    View Slide

  20. I'm writing a book on Gulp
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View Slide

  21. I don't know Java anymore
    Java is to JavaScript what

    Alf is to Gandalf

    View Slide

  22. Let's go!

    View Slide

  23. The ecosystem

    View Slide

  24. View Slide

  25. Runtime environment
    JavaScript

    View Slide

  26. Package System for Node
    Host of all things Gulp plugins

    View Slide

  27. CLI Gulpfile
    PluginA
    PluginA

    View Slide

  28. CLI Gulpfile
    PluginA
    PluginA
    starts

    View Slide

  29. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts

    View Slide

  30. CLI Gulpfile
    PluginA
    PluginA
    loads
    starts
    uses

    View Slide

  31. npm install -g gulp-cli
    npm init
    npm install --save-dev gulp
    touch gulpfile.js

    View Slide

  32. npm install -g gulpjs/gulp-cli#4.0
    npm init
    npm install --save-dev gulpjs/gulp#4.0
    touch gulpfile.js

    View Slide

  33. The basics:
    Streams

    View Slide

  34. gulp.src(…) gulp.dest(…)
    Reads files Writes files

    View Slide

  35. gulp.src(…) gulp.dest(…)

    View Slide

  36. gulp.src(…) .pipe(uglify()) gulp.dest(…)

    View Slide

  37. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    .pipe(concat())

    View Slide

  38. to the command line!

    View Slide

  39. Gulp API
    • gulp.task creates a new task

    • It requires to return either a Stream, a Promise or an Observable

    • gulp.src “globs” files and returns a stream of virtual file objects

    • each file can be piped through a process (jshint, uglify, less, etc.)

    • gulp.dest saves the file back to the file system

    View Slide

  40. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });

    View Slide

  41. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    defines a new task

    View Slide

  42. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    with a defined name

    View Slide

  43. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    we load a certain file
    (or files)
    Starting here, we have virtual
    files in-memory instead of
    real files we

    View Slide

  44. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    and pipe it through a
    series of operations

    View Slide

  45. gulp.task('styles', function() {
    return gulp.src('app/styles/main.less')
    .pipe(less())
    .pipe(minifyCSS())
    .pipe(prefix())
    .pipe(gulp.dest('dist/styles'));
    });
    before we save it
    again on the "real" file system

    View Slide

  46. Tasks

    View Slide

  47. scripts styles lint

    View Slide

  48. scripts
    styles
    lint
    gulp.parallel

    View Slide

  49. scripts
    styles
    lint
    gulp.parallel
    gulp.series

    View Slide

  50. to the command line!

    View Slide

  51. Gulp API
    • The second parameter of gulp.task is always a function.

    • gulp.series is a task function that runs tasks in sequential order.

    • gulp.parallel is a task function that starts every task concurrently

    • Both task functions accept task names and other functions as
    parameters.

    • They can be combined infinitly

    View Slide

  52. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );

    View Slide

  53. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in series

    View Slide

  54. gulp.task('default',
    gulp.series('clean',
    gulp.parallel('styles', 'scripts'),
    ‘server'
    )
    );
    runs in parallel

    View Slide

  55. A development
    environment

    View Slide

  56. gulp.watch
    scripts
    *.js

    View Slide

  57. Bowser-Sync!
    r

    View Slide

  58. to the command line!

    View Slide

  59. Gulp API
    • gulp.watch creates a file watcher and listens to changes

    • changes include ‘change’, ‘add’, ‘unlink’ and others

    • BrowserSync is a development tool that can be fully integrated in
    Gulp.

    • Watchers trigger a browserSync.reload call

    View Slide

  60. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }

    View Slide

  61. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    watches this Glob pattern

    View Slide

  62. function watcher(done) {
    gulp.watch('styles/**/*.less', gulp.parallel(‘styles’));
    done();
    }
    starts this task on
    change, unlink, add

    View Slide

  63. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    });

    View Slide

  64. gulp.task('server', function(done) {
    bSync({
    server: {
    baseDir: ['dist', 'app']
    }
    })
    done();
    }); BrowserSync set up to start
    a dev server, serving dist and
    app statically

    View Slide

  65. gulp.watch('dist/**/*', bSync.reload);
    And a watcher that triggers a
    reload

    View Slide

  66. Incremental
    builds

    View Slide

  67. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  68. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  69. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    Too much is going on!
    Each change: Uglify all
    the files?

    View Slide

  70. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  71. View Slide

  72. filter files
    that have changed

    View Slide

  73. filter files
    that have changed
    do performance
    heavy operations

    View Slide

  74. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files

    View Slide

  75. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    and continue
    with the other ops

    View Slide

  76. to the command line!

    View Slide

  77. Gulp Plugins
    • gulp-cached and gulp-remember can be used to create file caches

    • The plugin filters non-changed files and ads them back to the
    stream once we are done with performance-heavy tasks

    • Additionally to that, we can use gulp.lastRun in Gulp 4 to filter files
    during globbing

    • gulp-newer allows us to do incremental copies/builds on a per-file
    basis

    View Slide

  78. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });

    View Slide

  79. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    we use the cache to
    check if files have
    changed

    View Slide

  80. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    once we are done,
    we remember all
    the other files
    we stored in
    the cache

    View Slide

  81. Part II

    View Slide

  82. A short
    source map interlude

    View Slide

  83. Browserify
    (for Babel/React)

    View Slide

  84. so similar … yet so
    different?
    Gulp
    Streams
    Browserify 

    Streams

    View Slide

  85. Why not both??

    View Slide

  86. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }

    View Slide

  87. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    b.bundle emits a
    stream.
    But no vinyl file
    objects

    View Slide

  88. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-source-stream
    wraps the original
    stream into
    a vinyl file object

    View Slide

  89. var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('js'));
    }
    vinyl-buffer converts
    the stream contents
    to a buffer for
    plugins who need
    such

    View Slide

  90. Stream arrays
    and merge Streams

    View Slide

  91. A static
    site generator

    View Slide

  92. What does it do?
    • Generates static HTML sites

    • From a templating engine

    • Can parse Markdown

    • Can parse HTML

    • Can create permalinks

    • For different types (posts, pages)

    View Slide

  93. The stack
    • We use kramdown to convert Markdown to HTML

    • We use nujucks for our templating engine

    • We rename posts to feature blog permalink

    • We rename pages to resemble pretty URLs

    View Slide

  94. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View Slide

  95. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View Slide

  96. In pipes
    gulp.src(‘posts/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘posts/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!

    View Slide

  97. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())

    View Slide

  98. In pipes
    gulp.src(‘pages/*.md')
    .pipe(kramdown())
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    gulp.src(‘pages/*.html')
    .pipe(wrap())
    .pipe(nunjucks())
    .pipe(rename())
    .pipe(gulp.dest())
    the same!

    View Slide

  99. What if we could
    reuse parts of the
    stream?

    View Slide

  100. to the command line!

    View Slide

  101. to the command line!

    View Slide

  102. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });

    View Slide

  103. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });
    we combine
    multiple sources
    to one stream

    View Slide

  104. gulp.task('default', function(cb) {
    var streams = elements.map(function(el) {
    return merge(
    gulp.src(el.dir + '/**.md').pipe(markdown()),
    gulp.src(el.dir + '/**.html')
    ).pipe(rename(el.renamefn));
    });
    return merge(streams).pipe(data(options))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));
    });
    with Array.map
    and merge we can
    create stream
    arrays

    View Slide

  105. Material

    View Slide

  106. Workshop files
    https://github.com/frontend-tooling

    https://github.com/frontend-tooling/sample-project-gulp

    https://github.com/frontend-tooling/static-site-generator

    http://fettblog.eu

    http://speakerdeck.com/ddprrt

    View Slide

  107. Reading Material
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View Slide

  108. View Slide

  109. @ddprrt

    View Slide