Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced JavaScript build pipelines using Gulp.js

Advanced JavaScript build pipelines using Gulp.js

It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.

But are build tools really the answer to everything?

In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.

Stefan Baumgartner

November 18, 2015
Tweet

More Decks by Stefan Baumgartner

Other Decks in Programming

Transcript

  1. Advanced JavaScript
    Build Pipelines
    with Gulp.js

    View Slide

  2. Let's talk a
    short bit about
    the JS build tool
    revolution

    View Slide

  3. HTML
    CSS
    JavaScript

    View Slide

  4. Sass CoffeeScript LESS
    P o s t C S S H A M L J a d e
    U g l i f y E S 6 R e a c t J S
    B r o w s e r i f y A n g u l a r J S
    E m b e r C S S M i n J S L i n t
    ESHint ImageOptim Mocha
    Jasmine TypeScript

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. 1530 lines of code
    original Ant tasks used:
    concat — copy — delete — mkdir

    View Slide

  10. View Slide

  11. Maybe Java isn't the right
    tool?
    Java is to JavaScript what

    Alf is to Gandalf

    View Slide

  12. Grunt started a boom

    View Slide

  13. Gruntfiles get long

    View Slide

  14. Grunt tasks get slow:
    lots of reads
    and writes

    View Slide

  15. One unmanageable horde
    of (sometimes)
    low quality plug-ins

    View Slide

  16. And then came Gulp

    View Slide

  17. React is in love with
    Webpack

    View Slide

  18. Ember.js digs Broccoli.js

    View Slide

  19. Ember.js digs Broccoli.js

    View Slide

  20. View Slide

  21. So… every community seems
    to have its preference
    Stick with your communities
    preference!

    View Slide

  22. For all the rest?
    Gulp might be a good,
    very good option

    View Slide

  23. Disclaimer

    View Slide

  24. I (occasionally) contribute
    to Gulp

    View Slide

  25. I'm writing a book on Gulp
    http://bit.ly/gulp-tool-book

    39% off with 39baumgar

    coupon code!

    View Slide

  26. gulp.src(…) gulp.dest(…)
    Reads files Writes files

    View Slide

  27. gulp.src(…) .pipe(uglify()) gulp.dest(…)

    View Slide

  28. gulp.src(…) .pipe(uglify()) gulp.dest(…)
    .pipe(concat())

    View Slide

  29. task
    manager
    streaming
    file
    system
    file watcher

    View Slide

  30. gulp.task('scripts', function() {
    return gulp.src(‘src/**/*.js')
    .pipe(uglify())
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist'));
    });
    gulp.task('default', function(done) {
    gulp.watch(‘src/**/*.js', gulp.parallel(‘scripts'));
    done();
    });

    View Slide

  31. undertaker.task('scripts', function() {
    return vinyl.src(‘src/**/*.js')
    .pipe(uglify())
    .pipe(concat('main.min.js'))
    .pipe(vinyl.dest('dist'));
    });
    undertaker.task('default', function(done) {
    chokidar.watch(‘src/**/*.js', undertaker.parallel(‘scripts'));
    done();
    });

    View Slide

  32. so similar … yet so
    different?
    Gulp
    Streams
    Browserify 

    Streams

    View Slide

  33. var source = require(‘vinyl-source-stream’);
    var b = browserify({
    entries: ['_js/main.js']
    });
    var bundle = function() {
    return b.bundle()
    .pipe(source(‘main.js’))
    .pipe(gulp.dest('js'));
    }
    gulp.task(‘bundle’, bundle);

    View Slide

  34. This also goes the
    other way around…

    View Slide

  35. var app = express();
    var router = express.Router();
    router.get('/*', function(req, res) {
    var stream = request('http://host.to.forward.to' + req.originalUrl);
    stream.pipe(res);
    stream
    .pipe(source('.' + req.originalUrl))
    .pipe(gulp.dest('./cachedir'));
    });
    app.use(express.static('_site'));
    app.use(express.static('cachedir'));
    app.use(router);

    View Slide

  36. var app = express();
    var router = express.Router();
    router.get('/*', function(req, res) {
    var stream = request('http://host.to.forward.to' + req.originalUrl);
    stream.pipe(res);
    stream
    .pipe(source('.' + req.originalUrl))
    .pipe(gulp.dest('./cachedir'));
    });
    app.use(express.static('_site'));
    app.use(express.static('cachedir'));
    app.use(router);

    View Slide

  37. So start thinking
    streams …

    View Slide

  38. Multiple input formats
    return gulp.src(‘src/**/*.coffee)
    .pipe(coffee())
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))

    View Slide

  39. Multiple input formats
    return gulp.src(‘src/**/*.js)
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))

    View Slide

  40. Multiple input formats
    return gulp.src(‘src/**/*.js)
    .pipe(uglify())
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest(‘build’))
    the same!

    View Slide

  41. Multiple input formats
    And actually, you just want
    one bundle in the end

    View Slide

  42. What if we could
    reuse parts of the
    stream?

    View Slide

  43. return gulp.src(‘app1/src/**/*.coffee’)
    .pipe(coffee())
    .pipe(uglify()
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest('build'));

    View Slide

  44. var merge = require(‘merge2’);
    return merge(gulp.src(‘app1/src/**/*.coffee’)
    .pipe(coffee()),
    gulp.src(‘app1/src/**/*.js’))
    .pipe(uglify()
    .pipe(concat(‘main.js’))
    .pipe(gulp.dest('build'));

    View Slide

  45. var merge = require(‘merge2’);
    return gulp.src(‘app1/src/**/*.coffee’)
    .pipe(markdown()),
    .pipe(gulp.src(‘app1/src/**/*.js’), {passthrough: true})
    .pipe(rename(blogFn))
    .pipe(wrap(layoutStr))
    .pipe(swig())
    .pipe(gulp.dest('build'));

    View Slide

  46. Multiple bundles
    And now we need this
    over and over again
    for all our applications…

    View Slide

  47. var elems = [
    { dir: ‘app1’, bundleName: ‘app1.min.js’ },
    { dir: ‘app2’, bundleName: ‘app2.min.js’ }
    ];

    View Slide

  48. var streams = elems.map(function(el) {
    return gulp.src(el.dir + ‘src/**/*.coffee’)
    .pipe(coffee()),
    .pipe(gulp.src(el.dir + ‘src/**/*.js’), {passthrough: true})
    .pipe(uglify())
    .pipe(concat(el.bundleName))
    });
    return merge(streams)
    .pipe(gulp.dest('build'));

    View Slide

  49. Incremental
    builds

    View Slide

  50. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  51. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  52. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())
    Too much is going on!
    Each change: Uglify all
    the files?

    View Slide

  53. Some tasks take long
    gulp.src(‘scripts/*.js’)
    .pipe(uglify())
    .pipe(gulp.dest())
    .pipe(concat())

    View Slide

  54. View Slide

  55. filter files
    that have changed

    View Slide

  56. filter files
    that have changed
    do performance
    heavy operations

    View Slide

  57. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files

    View Slide

  58. filter files
    that have changed
    do performance
    heavy operations
    remember the
    old files
    and continue
    with the other ops

    View Slide

  59. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });

    View Slide

  60. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    we use the cache to
    check if files have
    changed

    View Slide

  61. gulp.task('scripts', function () {
    return gulp.src('app/scripts/**/*.js')
    .pipe(cached('ugly'))
    .pipe(uglify())
    .pipe(remember('ugly'))
    .pipe(concat('main.min.js'))
    .pipe(gulp.dest('dist/scripts'));
    });
    once we are done,
    we remember all
    the other files
    we stored in
    the cache

    View Slide

  62. gulp.task('lint', function () {
    return gulp.src(‘src/**/*.js’)
    .pipe(jshint())
    .pipe(jshint.reporter(‘default’))
    .pipe(jshint.reporter(‘fail’));
    });

    View Slide

  63. gulp.task('lint', function () {
    return gulp.src(‘src/**/*.js’, { since: gulp.lastRun(‘lint’) })
    .pipe(jshint())
    .pipe(jshint.reporter(‘default’))
    .pipe(jshint.reporter(‘fail’));
    });

    View Slide

  64. gulp.task('images', function () {
    return gulp.src(‘src/images/**/*.*’)
    .pipe(imagemin())
    .pipe(gulp.dest(‘dist’));
    });

    View Slide

  65. gulp.task('images', function () {
    return gulp.src(‘src/images/**/*.*’)
    .pipe(newer(‘dist’))
    .pipe(imagemin())
    .pipe(gulp.dest(‘dist’));
    });

    View Slide

  66. When architecting
    Gulp build pipelines…

    View Slide

  67. Separate the process
    from the content

    View Slide

  68. Think about how the
    basic pipeline would
    look like

    View Slide

  69. Think about how
    your data looks like

    View Slide

  70. And then posh your pipes up
    with stream tools that
    change your data accordingly

    View Slide

  71. View Slide

  72. @ddprrt

    View Slide