Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced JavaScript build pipelines using Gulp.js

Advanced JavaScript build pipelines using Gulp.js

It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.

But are build tools really the answer to everything?

In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.

Stefan Baumgartner

November 18, 2015
Tweet

More Decks by Stefan Baumgartner

Other Decks in Programming

Transcript

  1. Advanced JavaScript
    Build Pipelines
    with Gulp.js

    View full-size slide

  2. Let's talk a
    short bit about
    the JS build tool
    revolution

    View full-size slide

  3. HTML
    CSS
    JavaScript

    View full-size slide

  4. Sass CoffeeScript LESS
    P o s t C S S H A M L J a d e
    U g l i f y E S 6 R e a c t J S
    B r o w s e r i f y A n g u l a r J S
    E m b e r C S S M i n J S L i n t
    ESHint ImageOptim Mocha
    Jasmine TypeScript

    View full-size slide

  5. 1530 lines of code
    original Ant tasks used:
    concat — copy — delete — mkdir

    View full-size slide

  6. Maybe Java isn't the right
    tool?
    Java is to JavaScript what

    Alf is to Gandalf

    View full-size slide

  7. Grunt started a boom

    View full-size slide

  8. Gruntfiles get long

    View full-size slide

  9. Grunt tasks get slow:
    lots of reads
    and writes

    View full-size slide

  10. One unmanageable horde
    of (sometimes)
    low quality plug-ins

    View full-size slide

  11. And then came Gulp

    View full-size slide

  12. React is in love with
    Webpack

    View full-size slide

  13. Ember.js digs Broccoli.js

    View full-size slide

  14. Ember.js digs Broccoli.js

    View full-size slide

  15. So… every community seems
    to have its preference
    Stick with your communities
    preference!

    View full-size slide

  16. For all the rest?
    Gulp might be a good,
    very good option

    View full-size slide

  17. I (occasionally) contribute
    to Gulp

    View full-size slide

  18. I'm writing a book on Gulp
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View full-size slide

  19. gulp.src(…) gulp.dest(…)
    Reads files Writes files

    View full-size slide

  20. gulp.src(…) .pipe(uglify()) gulp.dest(…)

    View full-size slide

  21. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    .pipe(concat())

    View full-size slide

  22. task
    manager
    streaming
    file
    system
    file watcher

    View full-size slide

  23. gulp.task('scripts', function() {
    return gulp.src(‘src/**/*.js')
    .pipe(uglify())
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist'));
    });
    gulp.task('default', function(done) {
    gulp.watch(‘src/**/*.js', gulp.parallel(‘scripts'));
    done();
    });

    View full-size slide

  24. undertaker.task('scripts', function() {
    return vinyl.src(‘src/**/*.js')
    .pipe(uglify())
    .pipe(concat('main.min.js'))
    .pipe(vinyl.dest('dist'));
    });
    undertaker.task('default', function(done) {
    chokidar.watch(‘src/**/*.js', undertaker.parallel(‘scripts'));
    done();
    });

    View full-size slide

  25. so similar … yet so
    different?
    Gulp
    Streams
    Browserify 

    Streams

    View full-size slide

  26. var source = require(‘vinyl-source-stream’);
    var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(gulp.dest('js'));
    }
    gulp.task(‘bundle’, bundle);

    View full-size slide

  27. This also goes the
    other way around…

    View full-size slide

  28. var app = express();
    var router = express.Router();
    router.get('/*', function(req, res) {
    var stream = request('http://host.to.forward.to' + req.originalUrl);
    stream.pipe(res);
    stream
    .pipe(source('.' + req.originalUrl))
    .pipe(gulp.dest('./cachedir'));
    });
    app.use(express.static('_site'));
    app.use(express.static('cachedir'));
    app.use(router);

    View full-size slide

  29. var app = express();
    var router = express.Router();
    router.get('/*', function(req, res) {
    var stream = request('http://host.to.forward.to' + req.originalUrl);
    stream.pipe(res);
    stream
    .pipe(source('.' + req.originalUrl))
    .pipe(gulp.dest('./cachedir'));
    });
    app.use(express.static('_site'));
    app.use(express.static('cachedir'));
    app.use(router);

    View full-size slide

  30. So start thinking
    streams …

    View full-size slide

  31. Multiple input formats
    return gulp.src(‘src/**/*.coffee)
    .pipe(coffee())
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))

    View full-size slide

  32. Multiple input formats
    return gulp.src(‘src/**/*.js)
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))

    View full-size slide

  33. Multiple input formats
    return gulp.src(‘src/**/*.js)
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))
    the same!

    View full-size slide

  34. Multiple input formats
    And actually, you just want
    one bundle in the end

    View full-size slide

  35. What if we could
    reuse parts of the
    stream?

    View full-size slide

  36. return gulp.src(‘app1/src/**/*.coffee’)
    .pipe(coffee())
    .pipe(uglify()
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest('build'));

    View full-size slide

  37. var merge = require(‘merge2’);
    return merge(gulp.src(‘app1/src/**/*.coffee’)
    .pipe(coffee()),
    gulp.src(‘app1/src/**/*.js’))
    .pipe(uglify()
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest('build'));

    View full-size slide

  38. var merge = require(‘merge2’);
    return gulp.src(‘app1/src/**/*.coffee’)
    .pipe(markdown()),
    .pipe(gulp.src(‘app1/src/**/*.js’), {passthrough: true})
    .pipe(rename(blogFn))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));

    View full-size slide

  39. Multiple bundles
    And now we need this
    over and over again
    for all our applications…

    View full-size slide

  40. var elems = [
    { dir: ‘app1’, bundleName: ‘app1.min.js’ },
    { dir: ‘app2’, bundleName: ‘app2.min.js’ }
    ];

    View full-size slide

  41. var streams = elems.map(function(el) {
    return gulp.src(el.dir + ‘src/**/*.coffee’)
    .pipe(coffee()),
    .pipe(gulp.src(el.dir + ‘src/**/*.js’), {passthrough: true})
    .pipe(uglify())
    .pipe(concat(el.bundleName))
    });
    return merge(streams)
    .pipe(gulp.dest('build'));

    View full-size slide

  42. Incremental
    builds

    View full-size slide

  43. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  44. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  45. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    Too much is going on!
    Each change: Uglify all
    the files?

    View full-size slide

  46. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View full-size slide

  47. filter files
    that have changed

    View full-size slide

  48. filter files
    that have changed
    do performance
    heavy operations

    View full-size slide

  49. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files

    View full-size slide

  50. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    and continue
    with the other ops

    View full-size slide

  51. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });

    View full-size slide

  52. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    we use the cache to
    check if files have
    changed

    View full-size slide

  53. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    once we are done,
    we remember all
    the other files
    we stored in
    the cache

    View full-size slide

  54. gulp.task('lint', function () {
    return gulp.src(‘src/**/*.js’)
    .pipe(jshint())
    .pipe(jshint.reporter(‘default’))
    .pipe(jshint.reporter(‘fail’));
    });

    View full-size slide

  55. gulp.task('lint', function () {
    return gulp.src(‘src/**/*.js’, { since: gulp.lastRun(‘lint’) })
    .pipe(jshint())
    .pipe(jshint.reporter(‘default’))
    .pipe(jshint.reporter(‘fail’));
    });

    View full-size slide

  56. gulp.task('images', function () {
    return gulp.src(‘src/images/**/*.*’)
    .pipe(imagemin())
    .pipe(gulp.dest(‘dist’));
    });

    View full-size slide

  57. gulp.task('images', function () {
    return gulp.src(‘src/images/**/*.*’)
    .pipe(newer(‘dist’))
    .pipe(imagemin())
    .pipe(gulp.dest(‘dist’));
    });

    View full-size slide

  58. When architecting
    Gulp build pipelines…

    View full-size slide

  59. Separate the process
    from the content

    View full-size slide

  60. Think about how the
    basic pipeline would
    look like

    View full-size slide

  61. Think about how
    your data looks like

    View full-size slide

  62. And then posh your pipes up
    with stream tools that
    change your data accordingly

    View full-size slide