Advanced JavaScript build pipelines using Gulp.js

Advanced JavaScript build pipelines using Gulp.js

It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.

But are build tools really the answer to everything?

In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.

187d92c9284160ad908885ab096f5209?s=128

Stefan Baumgartner

November 18, 2015
Tweet

Transcript

  1. Advanced JavaScript Build Pipelines with Gulp.js

  2. Let's talk a short bit about the JS build tool

    revolution
  3. HTML CSS JavaScript

  4. Sass CoffeeScript LESS P o s t C S S

    H A M L J a d e U g l i f y E S 6 R e a c t J S B r o w s e r i f y A n g u l a r J S E m b e r C S S M i n J S L i n t ESHint ImageOptim Mocha Jasmine TypeScript
  5. None
  6. None
  7. None
  8. None
  9. 1530 lines of code original Ant tasks used: concat —

    copy — delete — mkdir
  10. None
  11. Maybe Java isn't the right tool? Java is to JavaScript

    what
 Alf is to Gandalf
  12. Grunt started a boom

  13. Gruntfiles get long

  14. Grunt tasks get slow: lots of reads and writes

  15. One unmanageable horde of (sometimes) low quality plug-ins

  16. And then came Gulp

  17. React is in love with Webpack

  18. Ember.js digs Broccoli.js

  19. Ember.js digs Broccoli.js

  20. None
  21. So… every community seems to have its preference Stick with

    your communities preference!
  22. For all the rest? Gulp might be a good, very

    good option
  23. Disclaimer

  24. I (occasionally) contribute to Gulp

  25. I'm writing a book on Gulp http://bit.ly/gulp-tool-book 39% off with

    39baumgar
 coupon code!
  26. gulp.src(…) gulp.dest(…) Reads files Writes files

  27. gulp.src(…) .pipe(uglify()) gulp.dest(…)

  28. gulp.src(…) .pipe(uglify()) gulp.dest(…) .pipe(concat())

  29. task manager streaming file system file watcher

  30. gulp.task('scripts', function() { return gulp.src(‘src/**/*.js') .pipe(uglify()) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist')); }); gulp.task('default',

    function(done) { gulp.watch(‘src/**/*.js', gulp.parallel(‘scripts')); done(); });
  31. undertaker.task('scripts', function() { return vinyl.src(‘src/**/*.js') .pipe(uglify()) .pipe(concat('main.min.js')) .pipe(vinyl.dest('dist')); }); undertaker.task('default',

    function(done) { chokidar.watch(‘src/**/*.js', undertaker.parallel(‘scripts')); done(); });
  32. so similar … yet so different? Gulp Streams Browserify 


    Streams
  33. var source = require(‘vinyl-source-stream’); var b = browserify({ entries: ['_js/main.js']

    }); var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(gulp.dest('js')); } gulp.task(‘bundle’, bundle);
  34. This also goes the other way around…

  35. var app = express(); var router = express.Router(); router.get('/*', function(req,

    res) { var stream = request('http://host.to.forward.to' + req.originalUrl); stream.pipe(res); stream .pipe(source('.' + req.originalUrl)) .pipe(gulp.dest('./cachedir')); }); app.use(express.static('_site')); app.use(express.static('cachedir')); app.use(router);
  36. var app = express(); var router = express.Router(); router.get('/*', function(req,

    res) { var stream = request('http://host.to.forward.to' + req.originalUrl); stream.pipe(res); stream .pipe(source('.' + req.originalUrl)) .pipe(gulp.dest('./cachedir')); }); app.use(express.static('_site')); app.use(express.static('cachedir')); app.use(router);
  37. So start thinking streams …

  38. Multiple input formats return gulp.src(‘src/**/*.coffee) .pipe(coffee()) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’))

  39. Multiple input formats return gulp.src(‘src/**/*.js) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’))

  40. Multiple input formats return gulp.src(‘src/**/*.js) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’)) the same!

  41. Multiple input formats And actually, you just want one bundle

    in the end
  42. What if we could reuse parts of the stream?

  43. return gulp.src(‘app1/src/**/*.coffee’) .pipe(coffee()) .pipe(uglify() .pipe(concat(‘main.js’)) .pipe(gulp.dest('build'));

  44. var merge = require(‘merge2’); return merge(gulp.src(‘app1/src/**/*.coffee’) .pipe(coffee()), gulp.src(‘app1/src/**/*.js’)) .pipe(uglify() .pipe(concat(‘main.js’))

    .pipe(gulp.dest('build'));
  45. var merge = require(‘merge2’); return gulp.src(‘app1/src/**/*.coffee’) .pipe(markdown()), .pipe(gulp.src(‘app1/src/**/*.js’), {passthrough: true})

    .pipe(rename(blogFn)) .pipe(wrap(layoutStr)) .pipe(swig()) .pipe(gulp.dest('build'));
  46. Multiple bundles And now we need this over and over

    again for all our applications…
  47. var elems = [ { dir: ‘app1’, bundleName: ‘app1.min.js’ },

    { dir: ‘app2’, bundleName: ‘app2.min.js’ } ];
  48. var streams = elems.map(function(el) { return gulp.src(el.dir + ‘src/**/*.coffee’) .pipe(coffee()),

    .pipe(gulp.src(el.dir + ‘src/**/*.js’), {passthrough: true}) .pipe(uglify()) .pipe(concat(el.bundleName)) }); return merge(streams) .pipe(gulp.dest('build'));
  49. Incremental builds

  50. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())

  51. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())

  52. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat()) Too much

    is going on! Each change: Uglify all the files?
  53. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())

  54. None
  55. filter files that have changed

  56. filter files that have changed do performance heavy operations

  57. filter files that have changed do performance heavy operations remember

    the old files
  58. filter files that have changed do performance heavy operations remember

    the old files and continue with the other ops
  59. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js'))

    .pipe(gulp.dest('dist/scripts')); });
  60. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js'))

    .pipe(gulp.dest('dist/scripts')); }); we use the cache to check if files have changed
  61. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js'))

    .pipe(gulp.dest('dist/scripts')); }); once we are done, we remember all the other files we stored in the cache
  62. gulp.task('lint', function () { return gulp.src(‘src/**/*.js’) .pipe(jshint()) .pipe(jshint.reporter(‘default’)) .pipe(jshint.reporter(‘fail’)); });

  63. gulp.task('lint', function () { return gulp.src(‘src/**/*.js’, { since: gulp.lastRun(‘lint’) })

    .pipe(jshint()) .pipe(jshint.reporter(‘default’)) .pipe(jshint.reporter(‘fail’)); });
  64. gulp.task('images', function () { return gulp.src(‘src/images/**/*.*’) .pipe(imagemin()) .pipe(gulp.dest(‘dist’)); });

  65. gulp.task('images', function () { return gulp.src(‘src/images/**/*.*’) .pipe(newer(‘dist’)) .pipe(imagemin()) .pipe(gulp.dest(‘dist’)); });

  66. When architecting Gulp build pipelines…

  67. Separate the process from the content

  68. Think about how the basic pipeline would look like

  69. Think about how your data looks like

  70. And then posh your pipes up with stream tools that

    change your data accordingly
  71. None
  72. @ddprrt