Harry is a developer team lead at Shopify, who's worked on Javascript frameworks, site reliability engineering, and big data warehousing. https://speakerdeck.com/airhorns 2015-04-09 11:46:53 -0400 How does Shopify change their production systems - both Hadoop and web tier - without breaking them? This discusses a spectrum of techniques: ring deploys, Docker based isolation, staging environments, logging strategies, and online verifiers to test changes and determine whether a solution is valid before exposing it to the world. How does Shopify change their production systems - both Hadoop and web tier - without breaking them? This discusses a spectrum of techniques: ring deploys, Docker based isolation, staging environments, logging strategies, and online verifiers to test changes and determine whether a solution is valid before exposing it to the world. Tue, 23 Feb 2016 00:00:00 -0500 https://speakerdeck.com/airhorns/shipping-with-confidence https://speakerdeck.com/airhorns/shipping-with-confidence While many companies have focused on powering products with their big data, Shopify has instead focused on building a well modelled data warehouse to power better decisions across it's rather wild business first. This seems like a terribly uncool thing to do these days: it doesn't involve much machine learning or advanced math, but it has been wildly successful at Shopify, and laid a fantastic foundation for data scientists to start doing the fancy stuff. Shopify has built a highly structured, clean, and useful warehouse using new ideas like Apache Spark and old ideas like dimensional modelling to achieve technical and organizational scalability. This talk will be about how we model our data, why we think it is the right thing to do, and the hard-won lessons learned in deploying Spark and Hadoop for such a task. While many companies have focused on powering products with their big data, Shopify has instead focused on building a well modelled data warehouse to power better decisions across it's rather wild business first. This seems like a terribly uncool thing to do these days: it doesn't involve much machine learning or advanced math, but it has been wildly successful at Shopify, and laid a fantastic foundation for data scientists to start doing the fancy stuff. Shopify has built a highly structured, clean, and useful warehouse using new ideas like Apache Spark and old ideas like dimensional modelling to achieve technical and organizational scalability. This talk will be about how we model our data, why we think it is the right thing to do, and the hard-won lessons learned in deploying Spark and Hadoop for such a task. Thu, 09 Apr 2015 00:00:00 -0400 https://speakerdeck.com/airhorns/big-data-warehousing-or-how-to-make-impact-with-data https://speakerdeck.com/airhorns/big-data-warehousing-or-how-to-make-impact-with-data