E4E Automating Your Site's Front-end Performance

Kitt Hodsden • @kitt • http://ki.tt/e4e2015 1
Automate Your Site's
Front-End Performance
Kitt Hodsden / @kitt
slides here

View Slide

Me.

View Slide

You.

View Slide


View Slide

Performance is
everybody’s problem.

View Slide

“What does FAST even mean?”

View Slide

Determine key metrics

View Slide

Establish a baseline

View Slide

Make a change

View Slide

Make a change
Measure effects

View Slide


View Slide

40% will abandon a
website that takes more
than 3 seconds to load.

View Slide

https://blog.kissmetrics.com/loading-time/
Not quite

View Slide

Performance Budget

View Slide

http://timkadlec.com/2013/01/setting-a-performance-budget/
http://timkadlec.com/2014/11/performance-budget-metrics/
http://danielmall.com/articles/how-to-make-a-performance-budget/

View Slide


View Slide

https://speakerdeck.com/lara/designing-for-performance?slide=69

View Slide


View Slide

How fast is our site?

View Slide

Kitt Hodsden • @kitt • http://ki.tt/e4e2015

View Slide

No.

View Slide

The simplest way?

View Slide

Your browser

View Slide

Firefox

View Slide

Safari

View Slide

IE

View Slide

Chrome

View Slide

https://developer.chrome.com/devtools/docs/network

View Slide


View Slide

https://lafikl.github.io/perfBar/
36

View Slide


37

View Slide

https://lafikl.github.io/perfBar/

View Slide

But… but…
39

View Slide

My network!
40

View Slide

My browsers!
41

View Slide

https://speakerdeck.com/lara/designing-for-performance?slide=69

View Slide

Automation!
43

View Slide


View Slide

http://nodejs.org/

View Slide

npm
46
Node Package Manager for installing node packages

View Slide


View Slide

npm install -g grunt-cli
48

View Slide

49
package.json
Gruntfile.js
The two files grunt uses

View Slide

npm init
50

View Slide

51
$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sane defaults.
See `npm help json` for definitive documentation on these fields
and exactly what they do.
Use `npm install --save` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
name: (dev) performance
version: (1.0.0)
description: Automating front-end performance metrics and improvements
entry point: (index.js)
test command:
git repository:
keywords:
author:
license: (ISC)
About to write to .../e4e/dev/package.json:
...

View Slide

52
$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sane defaults.
See `npm help json` for definitive documentation on these fields
and exactly what they do.
Use `npm install --save` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
name: (dev) performance
version: (1.0.0)
description: Automating front-end performance metrics and improvements
entry point: (index.js)
test command:
git repository:
keywords:
author:
license: (ISC)
About to write to .../e4e/dev/package.json:
...

View Slide

53
{
"name": "performance",
"version": "1.0.0",
"description": "Automating front-end performance”,
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC"
}
This is what a package.json file looks like

View Slide

Just download them.
https://github.com/search?q=Gruntfile.js
https://github.com/kitt/grunt-perf-template
54

View Slide

55
{
"engines": {
"node": ">= 4.1.0"
},
"devDependencies": {
"grunt": "~0.4.5",
"grunt-contrib-jshint": "~0.11.3",
"grunt-contrib-watch": "~0.6.1",
}
}

View Slide

56
{
"engines": {
"node": ">= 4.1.0"
},
"devDependencies": {
"grunt": "~0.4.5",
"grunt-contrib-jshint": "~0.11.3",
"grunt-contrib-watch": "~0.6.1",
}
}

View Slide

57
$ npm install
With a package.json file, you can install the needed packages easily.

View Slide

58
$ npm install
No -g here!
With a package.json file, you can install the needed packages easily.

View Slide

59
$ ls
Gemfile README.txt fonts node_modules tpl
Gemfile.lock config.rb img package.json widgets
Gruntfile.js css js scss

View Slide

Check dependencies into the repo?
http://addyosmani.com/blog/checking-in-front-end-dependencies/
http://redotheweb.com/2013/09/12/should-you-commit-dependencies.html
60
Total sidebar on checking dependencies into the repo or not

View Slide

module.exports = function(grunt) {
// Project configuration.
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
uglify: {
options: {
banner: '/*! <%= pkg.name %> <%="yyyy-mm-dd") %> */\n'
},
build: {
src: 'src/<%= pkg.name %>.js',
dest: 'build/<%= pkg.name %>.min.js'
}
}
});
// Load the plugin that provides the "uglify" task.
grunt.loadNpmTasks('grunt-contrib-uglify');
// Default task(s).
grunt.registerTask('default', ['uglify']);
};

View Slide

62
grunt.initConfig({
uglify: {
options: {
},
build: {
}
}
});
// Default task(s).
};
A basic Gruntfile.js file

View Slide

63
grunt.initConfig({
uglify: {
options: {
},
build: {
}
}
});
// Default task(s).
};

View Slide

64
grunt.initConfig({
uglify: {
options: {
},
build: {
}
}
});
// Default task(s).
};

View Slide

65
grunt.loadNpmTasks('grunt-aws-s3');
grunt.loadNpmTasks('grunt-cafe-mocha');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-copy');
grunt.loadNpmTasks('grunt-contrib-csslint');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-contrib-jade');
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-contrib-imagemin');
grunt.loadNpmTasks('grunt-contrib-sass');
grunt.loadNpmTasks('grunt-env');
grunt.loadNpmTasks('grunt-favicons');
grunt.loadNpmTasks('grunt-html-validation');
grunt.loadNpmTasks('grunt-mkdir');

View Slide

66
$ npm install matchdep --save-dev

View Slide

67
$ npm install matchdep --save-dev

View Slide

68
...
// load all the grunt modules instead of one each line
require("matchdep").filterDev("grunt-*").forEach(grunt.loadNpmTasks);
grunt.registerTask('checkjs', ['jshint']);
grunt.registerTask('watchjs', ['jshint', 'watch']);
grunt.registerTask('gruntjs', ['jshint:gruntfile', 'watch']);

View Slide

https://www.flickr.com/photos/dandechiaro/4151566643

View Slide

npm install grunt-devperf --save-dev
70

View Slide

https://github.com/macbre/phantomas

View Slide

72
devperf: {
options: {
urls: [
'http://localhost:8000'
]
}
}
…
grunt.registerTask('pe', ['devperf']);

View Slide


View Slide

74
…
devperf: {
options: {
urls: [
'http://fromthemiddle.io/'
],
numberOfRuns: 5,
openResults: true,
resultsFolder: './devperf',
phantomasOptions: {
'film-strip': true
}
}
}
…
grunt.registerTask('pe', ['devperf']);

View Slide


View Slide

https://www.flickr.com/photos/ndrwfgg/8072089187

View Slide

http://www.devobjective.com/

View Slide

Remove HTML
Comments
84
https://github.com/gruntjs/grunt-contrib-htmlmin

View Slide

npm install grunt-contrib-htmlmin —save-dev
85
https://github.com/gruntjs/grunt-contrib-htmlmin

View Slide

86
…
htmlmin: {
dist: {
options: {
removeComments: true,
collapseWhitespace: true
},
files: {
'dist/index.html': 'src/index.html',
'dist/contact.html': 'src/contact.html'
}
},
dev: {
files: {
'dist/index.html': 'src/index.html',
'dist/contact.html': 'src/contact.html'
}
}
}
…
grunt.registerTask(‘comments-be-gone‘, ['htmlmin']);

View Slide

Make a change
Measure effects

View Slide

Concatenate and
Minify CSS files
89

View Slide

npm install grunt-contrib-cssmin --save-dev
90
Minimize CSS files

View Slide

Remove duplicate CSS?
91

View Slide

npm install grunt-csscss --save-dev
92
Report duplicate CSS in files

View Slide

Remove unused CSS?
93

View Slide

npm install grunt-uncss --save-dev
94
Remove unused CSS

View Slide

npm install grunt-phantomcss —save-dev
https://github.com/chrisgladd/grunt-phantomcss
https://github.com/Huddle/PhantomCSS
95
Make sure you haven’t removed things you need

View Slide

Make sure you haven’t removed things you need

View Slide

grunt-cssstats configuration https://github.com/cssstats/grunt-cssstats
phantomcss: {
desktop: {
options: {
screenshots: 'test/visual/desktop/',
results: 'results/visual/desktop',
viewportSize: [1024, 768]
},
src: [
'test/visual/**.js'
]
},
mobile: {
options: {
screenshots: 'test/visual/mobile/',
results: 'results/visual/mobile',
},
src: [
'test/visual/**.js'
]
}
}

View Slide

phantomcss: {
desktop: {
options: {
screenshots: 'test/visual/desktop/',
results: 'results/visual/desktop',
},
src: [
'test/visual/**.js'
]
},
mobile: {
options: {
screenshots: 'test/visual/mobile/',
results: 'results/visual/mobile',
},
src: [
'test/visual/**.js'
]
}
}

View Slide

casper.start(‘https://kitt.hodsden.org/')
.then(function() {
phantomcss.screenshot('#region-branding', 'region-branding');
})
.then(function now_check_the_screenshots() {
phantomCSS.compareAll();
});

View Slide

npm install grunt-cssstats --save-dev
100
https://github.com/cssstats/grunt-cssstats
Look at the site’s CSS statistics

View Slide

https://github.com/cssstats/cssstats

View Slide


View Slide

cssstats: {
dev: {
options: {},
files: {
'stats/css-stats.json': ['css/example.css']
},
}
}
…
grunt.registerTask('stats', ['cssstats']);

View Slide

cssstats: {
dev: {
options: { safe: false }, // Barf on invalid CSS
files: {
'stats/cssstats-dev.json': ['app/styles/main.css']
}
},
prod: {
files: {
'stats/cssstats.json': ['dist/styles.css']
}
}
}

View Slide

https://github.com/cssstats/cssstats
110

View Slide

Aggregate and
Minify JavaScript?
111

View Slide

npm install grunt-contrib-uglify --save-dev
112
Minimize javascript ﬁles

View Slide

Make a change
Measure effects

View Slide

Too many small images?
115
Making things easier with sprites!

View Slide

npm install grunt-montage --save-dev
116
Making things easier with sprites!

View Slide

117
grunt.initConfig({
montage: {
iconaroo: {
files: {
"images/sprites": [
"images/icons/*.png"
]
},
options: {
size: 32,
outputImage: "sprite-icons.png",
outputStylesheet: "sprite-icons.css"
}
}
}
});
Sample grunt-montage configuration

View Slide

npm install grunt-contrib-imagemin --save-dev
118
Make pages load faster by reducing image sizes

View Slide

119
imagemin: {
prod: {
files: [{
expand: true,
cwd: 'imgs-src/',
src: ['**/*.{png,jpg,gif}'],
dest: 'imgs/'
}]
}
}

View Slide

npm install grunt-svgmin --save-dev
120
Reduce SVG sizes, too

View Slide

121
grunt.initConfig({
svgmin: { // Task
options: { // Configuration that will be passed directly to SVGO
plugins: [
{ removeViewBox: false },
{ removeUselessStrokeAndFill: false }
]
},
dist: { // Target
files: [{ // Dictionary of files
expand: true, // Enable dynamic expansion.
cwd: 'img/src', // Src matches are relative to this path.
src: ['**/*.svg'], // Actual pattern(s) to match.
dest: 'img/', // Destination path prefix.
ext: '.min.svg' // Dest filepaths will have this extension.
// ie: optimise img/src/branding/logo.svg and store it in img/branding/logo.min.svg
}]
}
});
grunt.loadNpmTasks('grunt-svgmin');
grunt.registerTask('default', ['svgmin']);
Sample grunt-svgmin configuration

View Slide


View Slide

npm install grunt-responsive-images --save-dev
123
Responsive images by resizing
https://github.com/andismith/grunt-responsive-images

View Slide

124
grunt.initConfig({
responsive_images: {
myTask: {
options: {
sizes: [{
width: 320,
height: 240
},{
name: 'large',
width: 640
},{
name: "large",
width: 1024,
suffix: "_x2",
quality: 60
}]
},
files: [{
expand: true,
src: ['assets/**.{jpg,gif,png}'],
cwd: 'test/',
dest: 'resize/{%= width %}/'
}]
}
},
});
Sample grunt-responsive-images configuration

View Slide

npm install grunt-grunticon --save-dev
125
SVG with PNG fallbacks
https://github.com/ﬁlamentgroup/grunticon

View Slide

?
126

View Slide

webfonts!
127

View Slide

Him: “Hey, I have two megs of webfonts…”
Me: “…”
Him: “Is this okay?”
128

View Slide

No.
129

View Slide

npm install grunt-ziti —save-dev
https://github.com/caiguanhao/grunt-ziti
130
Reduce font file sizes

View Slide

131
grunt.initConfig({
ziti: {
subset_only: {
options: {
font: {
chars: '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz',
},
convert: false
},
files: {
‘assets/fonts/my.ttf': [ ‘src/fonts/original.ttf' ]
}
}
}
});
grunt.loadNpmTasks('grunt-ziti');
grunt.registerTask('fontsubset', ['ziti']);
Sample grunt-ziti configuration

View Slide

Wait… again…
132

View Slide

Dependent on
my network?
133

View Slide

Yep.
134

View Slide

How about not?
135

View Slide


View Slide

http://www.webpagetest.org/getkey.php
Request an API key.

View Slide


View Slide

http://www.webpagetest.org/getkey.php
Limits of public use
The API key is provisioned for up
to 200 page loads per day …
sufficient for most low-volume use
cases and for building a proof-of-
concept for larger testing

View Slide

Need more?

View Slide

https://sites.google.com/a/webpagetest.org/docs/private-instances
Set up your own WebPageTest server.

View Slide

npm install grunt-perfbudget --save-dev
143
https://github.com/tkadlec/grunt-perfbudget

View Slide

144
perfbudget: {
default: {
options: {
url: ‘http://example.io/',
key: 'PUT_YOUR_API_KEY_HERE'
}
}
}
…
grunt.registerTask('budg', ['perfbudget']);

View Slide

145
perfbudget: {
default: {
options: {
url: 'http://fromthemiddle.io/',
key: 'thisismykey',
wptInstance: 'ec2.us-west-2.amazonaws.com',
location: 'us-west-2_wptdriver',
pollResults: 10,
connectivity: '3G',
runs: 1,
timeout: 240
}
}
}

View Slide

146
perfbudget: {
default: {
options: {
key: 'thisismykey',
pollResults: 10,
connectivity: '3G',
runs: 1,
timeout: 240
}
}
}

View Slide

147
perfbudget: {
default: {
options: {
key: 'thisismykey',
pollResults: 10,
connectivity: '3G',
runs: 1,
timeout: 240
}
}
}

View Slide


View Slide

149
perfbudget: {
default: {
options: {
…
connectivity: '3G',
timeout: 240,
budget: {
render: '3000',
SpeedIndex: '7500',
visualComplete: '6000'
}
}
}
}

View Slide


View Slide

Now what?
152

View Slide

Integrate into CI builds
153

View Slide


View Slide

Tracking over time
155

View Slide

Tracking over time
156
http://www.wptmonitor.org/home

View Slide

npm install grunt-wpt --save-dev
157
https://github.com/sideroad/grunt-wpt
https://github.com/sideroad/grunt-wpt-page

View Slide

158
wpt: {
options: {
server: 'ec2.us-west-2.amazonaws.com',
dest: 'wpt/'
},
ftm: {
options: {
server: 'http://ec2.us-west-2.amazonaws.com',
url: ['http://fromthemiddle.io/']
},
dest: 'wpt/',
}
}
…
grunt.registerTask('wptrun', ['wpt']);

View Slide


View Slide

https://github.com/trulia/webpagetest-charts-ui
https://github.com/trulia/webpagetest-charts-api

View Slide

npm install grunt-shell --save-dev
161
Installing the grunt shell package to get to everything else

View Slide

function log(err, stdout, stderr, cb) {
console.log(stdout);
cb();
}
grunt.initConfig({
shell: {
dirListing: {
command: 'ls',
options: {
callback: log
}
}
}
});

View Slide

Oh boy!

View Slide

https://github.com/nature/webpagetest-mapper
webpagetest-mapper

View Slide

https://github.com/marcelduran/webpagetest-api
Dependent on
webpagetest-api

View Slide

npm install webpagetest-api -g
npm install webpagetest-mapper -g
166

View Slide

$ cat tests.json
[
{
"name": "From the Middle",
"url": "http://fromthemiddle.io/",
"type": "home"
},
{
"name": "Kitt Hodsden",
"url": "https://kitt.hodsden.org/",
"type": "away"
}
]

View Slide


View Slide

https://github.com/trulia/webpagetest-charts-ui
https://github.com/trulia/webpagetest-charts-api

View Slide

Make a change
Measure effects

View Slide

Make a change
Measure effects
Announce effects

View Slide


View Slide

sitespeed.io

View Slide

http://run.sitespeed.io/

View Slide


View Slide

Automation!

View Slide

https://speakerdeck.com/soulislove/monitoring-
web-performance-using-open-source-tools-san-
francisco-and-silicon-valley-web-performance-group
Peter Hedenskog / @soulislove

View Slide

https://speakerdeck.com/soulislove/monitoring-
web-performance-using-open-source-tools-san-
francisco-and-silicon-valley-web-performance-group
snk.ms/sswpt

View Slide

https://docs.docker.com/installation/

View Slide

Start docker (osx)

View Slide

https://www.sitespeed.io/documentation/performance-dashboard/
docker pull sitespeedio/graphite
docker pull grafana/grafana
docker pull sitespeedio/sitespeed.io

View Slide

start graphite

View Slide

https://www.sitespeed.io/documentation/performance-dashboard/
$ sudo mkdir -p /data/graphite/storage/whisper
$ sudo htpasswd -c /path/to/.htpasswd YOUR_USERNAME
$ sudo docker run -d \
--name graphite \
-p 8080:80 \
-p 2003:2003 \
--restart="always" \
-v /data/graphite/storage/whisper:/opt/graphite/storage/whisper \
-v /path/to/.htpasswd:/etc/nginx/.htpasswd \
sitespeedio/graphite

View Slide

start grafana

View Slide

$ sudo mkdir -p /data/grafana
$ sudo docker run -d -p 3000:3000 \
-v /data/grafana:/var/lib/grafana \
-e "GF_SECURITY_ADMIN_USER=myuser" \
-e "GF_SECURITY_ADMIN_PASSWORD=MY_SUPER_STRONG_PASSWORD" \
--name grafana \
grafana/grafana

View Slide


View Slide

$ sudo mkdir -p /data/grafana
$ sudo docker run -d -p 3000:3000 \
-v /data/grafana:/var/lib/grafana \
-e "GF_SECURITY_ADMIN_USER=myuser" \
-e "GF_SECURITY_ADMIN_PASSWORD=MY_SUPER_STRONG_PASSWORD" \
--name grafana \
grafana/grafana

View Slide

seed a single metric run

View Slide

# ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:62:d5:83:18
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:62ff:fed5:8318/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:419578 errors:0 dropped:0 overruns:0 frame:0
TX packets:399294 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:66734551 (66.7 MB) TX bytes:834474594 (834.4 MB)

View Slide

$ docker run --privileged --rm -v /sitespeed.io:/sitespeed.io
sitespeedio/sitespeed.io sitespeed.io -u http://fromthemiddle.io -b
firefox -n 1 --graphiteHost 172.17.42.1 --graphiteNamespace
sitespeed.io

View Slide

sitespeed.io

View Slide


View Slide

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
0,15,30,45 * * * * docker run --privileged --rm -v /sitespeed.io:/
sitespeed.io sitespeedio/sitespeed.io sitespeed.io -f urls.txt -b
firefox -n 11 --connection cable -r /tmp/ --graphiteHost
YOUR_GRAPHITE_HOST --seleniumServer http://127.0.0.1:4444/wd/hub >>
/tmp/sitespeed-run.txt 2>&1
# YOUR_GRAPHITE_HOST = likely your IP address if you docker’d
# -f urls.txt = list of URLs to analyze
# -n 11 = depth to go, use -n for single page

View Slide

Phew!
214

View Slide

http://tiny.run:3000/
myuser / MY_SUPER_STRONG_PASSWORD
hi / hihihi
215

View Slide

Worth looking at:
https://github.com/andyshora/grunt-yslow
https://github.com/jrcryer/grunt-pagespeed
https://github.com/addyosmani/psi/
http://perf-tooling.today/tools
216

View Slide


View Slide

Questions?
218

View Slide

Thanks!
Kitt Hodsden
http://ki.tt/
@kitt

View Slide

Wait a second…

View Slide


View Slide

Set up your own WebPageTest server.

View Slide

https://aws.amazon.com/
Login In to Amazon AWS

View Slide

Click on EC2

View Slide

Click on Launch Instance

View Slide

Click on Community AMIs

View Slide


View Slide

$ ssh -i downloaded.pem [email protected]

View Slide

$ vi ~/.ssh/authorized_keys
dd # delete the first line
[esc] # trigger the vi command sequence
:wq # save the file and quit

View Slide

sudo vi /var/www/webpagetest/www/settings/settings.ini
# update publishTo with ec2 hostname
# update ec2 = 0
# update ec2_locations = 0
# add EC2.us-west-2.securityGroup line from sample
# add EC2.us-west-2.subnetId line from sample
#

View Slide

sudo vi /var/www/webpage/test/www/settings/keys.ini
[server]
secret=addarandomstringhere
key=addakeyhere
[addakeyhere]
description=Web UI
;[email protected]
limit=0
[thisisyourapikey]
description=API Key
;[email protected]
limit=0

View Slide

$ cd /var/www/webpage/test/www/settings/
$ sudo cp locations.ini.EC2-sample locations.ini
# delete extra locations
# update key values to your thisisyourkey value

View Slide

$ sudo service nginx restart

View Slide

Set up testing instances

View Slide

EC2 server key in settings.ini
(more complicated to set up, better automation management)

View Slide

OR

View Slide

Manual Setup

View Slide

Click on Launch Instance

View Slide


View Slide

wpt_server=ec2.us-west-2.amazonaws.com wpt_key=YOURKEY wpt_location=us-west-2

View Slide

E4E Automating Your Site's Front-end Performance

E4E Automating Your Site's Front-end Performance

Kitt Hodsden

More Decks by Kitt Hodsden

Other Decks in Technology

Featured

Transcript