The modern SEO's toolkit

I evangelise that every online marketer should be thinking about going beyond "software" and into hacking together your own basic tools. Illustrated with examples and details.

Transcript

1. WILL CRITCHLOW The modern SEO’s toolkit
2. WILL CRITCHLOW - @willcritchlow Marketing. With computers
3. WILL CRITCHLOW - @willcritchlow Our ability to use APIs, scrapers, multiple tools is going to be super critical. Avinash Kaushik, interview on SEOmoz
4. WILL CRITCHLOW - @willcritchlow I could talk about all the big tools, but you know about them (and if you don’t, they’re easy to find)
5. WILL CRITCHLOW - @willcritchlow This talk is about the toolbox, not all-powerful tools [though I love that the picture on the right is entitled “really big lathe”] http://www.flickr.com/photos/dottiemae/5187337181/ http://www.flickr.com/photos/endless__mike/2757408845/
6. WILL CRITCHLOW - @willcritchlow Expect the output of every program to become the input to another, as yet unknown, program Doug McIlroy, Basics of the Unix Philosophy
7. WILL CRITCHLOW - @willcritchlow They refer to “pipes” (|) because you connect programs like this: `cat $file | grep 404` http://www.flickr.com/photos/danndalf/3534506071/
8. WILL CRITCHLOW - @willcritchlow I run a company whose product is written in code, and I don’t yet speak the language. I sometimes feel like a newspaper publisher who has to take his editor’s word for it that the articles are good. Roy Bahat, President at IGN Entertainment
9. WILL CRITCHLOW - @willcritchlow I am not trying to turn you into a developer (that would make Bill concerned). Not being an author isn’t an excuse for not learning English
10. What would you use? WILL CRITCHLOW - @willcritchlow
11. WILL CRITCHLOW - @willcritchlow You need to find all the googlebot visits in a certain time-period in a 22Gb log file
12. WILL CRITCHLOW - @willcritchlow I’d be turning to bash, grep and sed
13. WILL CRITCHLOW - @willcritchlow I’d be turning to bash, grep and sed Does getting started with this sound too hard?
14. WILL CRITCHLOW - @willcritchlow Too hard? Really? I call BS – here’s how to get up and running from scratch in 45 minutes
15. WILL CRITCHLOW - @willcritchlow I cut it down to 15 minutes – some sequences may be shortened
16. WILL CRITCHLOW - @willcritchlow 45 minutes too long? Can you spare 2?
17. WILL CRITCHLOW - @willcritchlow grep input.log -e '404 .*Googlebot/2.1' > output.log e.g. find googlebot 404 errors
18. WILL CRITCHLOW - @willcritchlow Pro-tips: man <command> / <command> --help More information about grep and sed. I also use curl -I, whois, host
19. WILL CRITCHLOW - @willcritchlow You need to explain to a client or a dev team the changes you want to make to a site
20. WILL CRITCHLOW - @willcritchlow Get comfortable with a prototyping / screenshot annotation tool. This is Balsamiq
21. WILL CRITCHLOW - @willcritchlow I’m currently really liking Skitch (mac only currently, but recently acquired by Evernote and apparently coming to Windows)
22. WILL CRITCHLOW - @willcritchlow Sometimes screencasts or quick videos are the easiest way to get the point across. It’s never been easier. I like screenr for screencasts (warning: public).
23. WILL CRITCHLOW - @willcritchlow I’ve been liking G+ for video, though Skype group calling is also good
24. WILL CRITCHLOW - @willcritchlow Start using a ticket tracking tool / kanban board / project management tool. My current favourite is Trello
25. WILL CRITCHLOW - @willcritchlow You need to get persuasive information for your boss on how people perceive your site
26. WILL CRITCHLOW - @willcritchlow See the Whiteboard Friday: Replicating Google’s Panda Questionnaire
27. WILL CRITCHLOW - @willcritchlow I really like Smartsheet for interfacing easily with Amazon Mechanical Turk
28. WILL CRITCHLOW - @willcritchlow You need to classify a bunch of URLs. But you need to discover the classifications as you go along
29. WILL CRITCHLOW - @willcritchlow Rich Baxter wrote a great how-to post on this
30. WILL CRITCHLOW - @willcritchlow I’d be turning to Excel for this. Pro-tip: F9 to evaluate a part of a formula on the fly. Don’t forget to ESC out afterwards F9
31. WILL CRITCHLOW - @willcritchlow You need to check a site to find pages with the old GA code on them
32. WILL CRITCHLOW - @willcritchlow I had to do this just the other day, and I used Screaming Frog (you’ll need the paid version to search the HTML of the pages)
33. WILL CRITCHLOW - @willcritchlow You need to debug when Google Analytics events are being fired
34. WILL CRITCHLOW - @willcritchlow I like the tamper data extension for Firefox
35. WILL CRITCHLOW - @willcritchlow Chrome has an extension for turning on ga_debug.js. SEER did a great video explaining how to use it
36. WILL CRITCHLOW - @willcritchlow You should also get to grips with “inspect element”
37. Let’s talk about debugging WILL CRITCHLOW - @willcritchlow
38. WILL CRITCHLOW - @willcritchlow The most common question I’m asked: Why (TF) doesn’t it work? 1. reproduce 2. simplify 3. isolate 4. document (++verbosity) 5. hack 6. sleep, shower
39. WILL CRITCHLOW - @willcritchlow You need to gather data about the proportions of people tweeting different phrases
40. WILL CRITCHLOW - @willcritchlow Twitter streaming API (garden hose) (http://www.flickr.com/photos/pinksherbet/3916146579/)
41. WILL CRITCHLOW - @willcritchlow JSON looks like this (this is the Hunch API). Tom would access this with Google Docs
42. WILL CRITCHLOW - @willcritchlow This is the entire python code we used to access the Hunch API on our recent hack day. If you’re following along at home you will need to pip install simplejson
43. WILL CRITCHLOW - @willcritchlow You need to whip up a quick website to display the contents of your database to others in your team
44. WILL CRITCHLOW - @willcritchlow Look how easy it is to get Python and Django up and running
45. WILL CRITCHLOW - @willcritchlow Once you are up and running to this point, articles like this one and tutorials like this one become useful.
46. WILL CRITCHLOW - @willcritchlow You need to grab all the email addresses of the commenters on a WordPress site
47. WILL CRITCHLOW - @willcritchlow Grabbing email addresses of commenters: https://gist.github.com/1301218. Learn SQL to speak to databases directly
48. WILL CRITCHLOW - @willcritchlow You need to make a bunch of circles in sizes proportional to input data
49. WILL CRITCHLOW - @willcritchlow Sometimes you need specialised tools. We find data visualisation tools useful for briefing designers
50. WILL CRITCHLOW - @willcritchlow You want to make a page scroll sideways when someone scrolls their mouse wheel
51. WILL CRITCHLOW - @willcritchlow I don’t know why you want to do that
52. WILL CRITCHLOW - @willcritchlow But if you did... This is the entire jQuery code you’d need to make a page scroll sideways. We truly stand on the shoulders of giants
53. WILL CRITCHLOW - @willcritchlow Speaking of the shoulders of giants. If you want to make a button wobble... Part of a larger trend of great CSS and JavaScript frameworks
54. Oh, want to know what HTML5 actually is? WILL CRITCHLOW - @willcritchlow
55. WILL CRITCHLOW - @willcritchlow HTML5 almost never manifests itself visibly on the page. The main things you need to know: semantic markup and native multimedia (oh, and canvas).
56. WILL CRITCHLOW - @willcritchlow CSS3 is a lot of the magic
57. WILL CRITCHLOW - @willcritchlow The rest is JavaScript
58. WILL CRITCHLOW - @willcritchlow You need to grab a screenshot of your site (or your competitor’s?) every day to correlate against analytics data
59. WILL CRITCHLOW - @willcritchlow Check out PhantomJS (via @ipullrank)
60. Rapid fire round WILL CRITCHLOW - @willcritchlow
61. WILL CRITCHLOW - @willcritchlow Possibly the most powerful tool I have learnt recently is git. Github has amazing hidden depths and tools for everything you can imagine
62. WILL CRITCHLOW - @willcritchlow I recommend reading @holman’s stuff on how github uses github. While you’re at it, check out speaker deck.
63. WILL CRITCHLOW - @willcritchlow General stuff 1. Set yourself up with upstream = DistilledLtd, origin = your branch and any other colleagues’ remotes as you need them 2. Regularly pull upstream master into your local master branch Developing something 1. git checkout master 2. git pull upstream master 3. git checkout -b featurebranch 4. [do your doing] 5. commit regularly, but as close to one commit per ticket as you can (you can rebase commits for extra special bonus points) add “re #ticketnum” into your commit message 6. git push origin featurebranch [do this regularly - it’s your best backup] When you’re ready for something to be tested / deployed: 1. Go to github.com/username/projectname//tree/featurebranch and create a pull request to upstream master 2. Add the URL of the pull request to the ticket, mark it to test and assign to the tester The Distilled git(hub) process (advanced) To test a featurebranch: 1. git checkout master 2. git pull upstream master 3. git checkout -b testing 4. git pull personfork featurebranch 1. if there are merge issues, fix ‘em (outside the scope of this checklist) and instead of pushing the big green button below, you’d have to push into upstream master 5. do your testing 1. if rejecting, update ticket as appropriate, git checkout master, git branch -d testing and start again with the next feature 2. if accepting, merge changes on github pull request using the big green button, git checkout master, git pull upstream master git branch -d testing and continue, update ticket to deploy and assign to the deployer Rebase (squash commits) 1. You can squash multiple commits into a single commit. This only works if the commits haven’t been pushed to a remote repository. git log (to see how many commits you need to squash) 2. git rebase -i HEAD~4 (where 4 is the number of commits you want to squash) 3. In the editor, pick commit message to keep, squash other messages http://gitready.com/advanced/2009/02/10/squashing-commits-with- rebase.html
64. WILL CRITCHLOW - @willcritchlow There is some cool stuff in the recent GA updates
65. WILL CRITCHLOW - @willcritchlow wow, we're going 800 people per hour, or 1500 people per hour. it was a real mind shift. maybe i'll do a post. @dannysullivan 29 Sep
66. WILL CRITCHLOW - @willcritchlow Get ranking data from codex.getstat.com
67. WILL CRITCHLOW - @willcritchlow Do you think there’s any chance that blekko will roll out webgrep to run any query you like?
68. WILL CRITCHLOW - @willcritchlow DataCouch is a github for data – fork, pull request (see later)
69. WILL CRITCHLOW - @willcritchlow Did you know you can now drop images onto Google Search (via Dave Naylor: http://dis.tl/oMMmzi)
70. WILL CRITCHLOW - @willcritchlow ThinkUp runs on your server and gives you social media analytics
71. WILL CRITCHLOW - @willcritchlow I came across Chandoo via Todd Malicoat. It’s packed with Excel resources and downloads
72. WILL CRITCHLOW - @willcritchlow Chrome has a scraper extension (via @iamoldskool)
73. WILL CRITCHLOW - @willcritchlow Topsy is one of the few real-time search engines left – now with added G+
74. WILL CRITCHLOW - @willcritchlow Uptime monitors (disclosure: we invested in Server Density). I’d love to see an SEO uptime monitor monitoring 404s, accidental noindex etc and alerting.
75. WILL CRITCHLOW - @willcritchlow A handy post on integrating the GA API into WordPress (for top posts etc). Via @smashingmag
76. WILL CRITCHLOW - @willcritchlow Know more about your visitors with Pzyche from Jason Duke
77. WILL CRITCHLOW - @willcritchlow Text APIs (hat-tip @ipullrank)
78. WILL CRITCHLOW - @willcritchlow Speaking of whom – check out his tools
79. Just for fun WILL CRITCHLOW - @willcritchlow
80. WILL CRITCHLOW - @willcritchlow I haven’t come up with a use for the face.com API apart from putting moustaches on photos, but I’m sure you can do something ingenious
81. WILL CRITCHLOW - @willcritchlow apigee: tools, dashboards and analytics for public APIs (via @andyjamesdavies)
82. WILL CRITCHLOW - @willcritchlow Another geeky tool to install on your new Ubuntu virtual machine. Built from buzzwords (MongoDB, node, etc.). From Square
83. Will Critchlow If you’re enjoying yourself, please tweet when NYC is awake – there is still time for them to buy tickets for next week: http://www.distilled.net/events/ Founder – Distilled will.critchlow@distilled.net @willcritchlow WILL CRITCHLOW - @willcritchlow