Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automation With or Without APIs

Avatar for Scott Scott
October 18, 2014

Automation With or Without APIs

Everyone wants to automate their life a little more. Some services make it easy but others make it difficult if not impossible to interact with. However, where there's a will, there's a way (with a little bit of code of course). Want a notification on your android watch when the weather is nice enough to open your windows? There's an API for that!

What about automatically downloading your bank statement in PDF every month and placing it in your Dropbox. Umm, maybe??? Want to buy your favorite custom pizza with a tweet? Nope, probably not. We will discuss potential areas for life hacking using service aggregators and technology like proxies and automated browsers. Don't know how to program? Don't worry. Even YOU can begin your journey today without writing a single line of code.

Avatar for Scott

Scott

October 18, 2014

Other Decks in Technology

Transcript

  1. YES

  2. Web Scraping aaS Kimono - Freemium Hosted Portia + Scrapy

    - Open Source (python) Self or paid hosting from ScrapingHub
  3. PhantomJS
 Nested Callbacks phantom.create(function(ph){          ph.createPage(function(page){  

                   page.open('http://www.espn.com/',  function(status){                          console.log('Opened  site?  %s',  status);                  });          });   });
  4. var  casper  =  require(‘casper').create();   var  links  =  [];  

        function  getLinks()  {          var  links  =  document.querySelectorAll('h3.r  a');          return  Array.prototype.map.call(links,  function(e)  {                  return  e.getAttribute('href');          });   }       casper.start('http://google.com/',  function()  {          //  search  for  'casperjs'  from  google  form          this.fill('form[action="/search"]',  {  q:  'casperjs'  },  true);   });       casper.then(function()  {          //  aggregate  results  for  the  'casperjs'  search          links  =  this.evaluate(getLinks);          //  now  search  for  'phantomjs'  by  filling  the  form  again          this.fill('form[action="/search"]',  {  q:  'phantomjs'  },  true);   });       casper.then(function()  {          //  aggregate  results  for  the  'phantomjs'  search          links  =  links.concat(this.evaluate(getLinks));   });       casper.run(function()  {          //  echo  results  in  some  pretty  fashion          this.echo(links.length  +  '  links  found:');          this.echo('  -­‐  '  +  links.join('\n  -­‐  ')).exit();   });  
  5. Resources • ifttt.com • tasker.dinglisch.net • imacros.net • kimonolabs.com •

    scrapinghub.com • github.com/scrapinghub/portia • charlesproxy.com • telerik.com/fiddler • getpostman.com • luckymarmot.com/paw • phantomjs.org • casperjs.org • nightmarejs.org