Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Rails to build Growth Hacks Fast

Using Rails to build Growth Hacks Fast

What is Growth Hacking and how it works
Back-end Technology Freedom
Front-end strategy
Handlebars for templates
Pushing data to track your user
Using AB Test tool

Rubens Stulzer

September 18, 2015

Transcript

192 slides
  1. EVERY GROWTH TEAM SHOULD • Have only one goal, and

    it should be clean and simple • Validate hypothesis, preferably raised through data
  2. EVERY GROWTH TEAM SHOULD • Have only one goal, and

    it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often
  3. EVERY GROWTH TEAM SHOULD • Have only one goal, and

    it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure
  4. EVERY GROWTH TEAM SHOULD • Have only one goal, and

    it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure • Read the numbers and understand them
  5. FOCUS • Grow Monthly Active Users on VivaReal mobile apps

    • Two important things to do it • Acquire more users through VivaReal Website
  6. FOCUS • Grow Monthly Active Users on VivaReal mobile apps

    • Two important things to do it • Acquire more users through VivaReal Website • Activate more users on VivaReal Apps
  7. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps
  8. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store
  9. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it
  10. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it
  11. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property
  12. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead
  13. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition
  14. FUNNEL • There is a huge funnel to bring website

    users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition Activation
  15. EXPERIMENT VALIDATION • We don’t know nothing, never, we should

    experiment it always • The easiest way to validate a hypothesis is through an A/B Test
  16. THE GOOD • Easy to implement the experiment code •

    It is simple to pick a winner through experiments dashboard
  17. THE GOOD • Easy to implement the experiment code •

    It is simple to pick a winner through experiments dashboard • Don’t need to be an expert on it to get things running
  18. THE BAD • No easy way to segment your users

    • No easy way to configure traffic allocation for a given experiment
  19. THE UGLY <% ab_test(:login_button, "/images/button1.jpg", "/images/button2.jpg") do |button_file| %> <%=

    image_tag(button_file, alt: "Login!") %> <% end %> <% the_test = ab_test('split-extras', 'splitted', 'not-splitted') %> <div class="row"> <div class="col-md-12"> <div class=“plans-options-duration plans-options-box-<%= 'center' if the_test == 'splitted' %>"> <!-- ... bunch of code --> </div> </div> </div>
  20. THE UGLY def index the_test = ab_test('new-home', 'new', 'old') if

    the_test == 'new' render ‘new_index', layout: 'new_application' else render 'old_index', layout: 'application' end end
  21. THE UGLY • Doesn’t add good value to the codebase

    • Someone will probably pick one variation, and you will forget that ugly piece of code there
  22. STACK • Java, Java everywhere • Huge PostgreSQL • Two

    different websites for Desktop and Mobile, both on the same project
  23. STACK • Java, Java everywhere • Huge PostgreSQL • Two

    different websites for Desktop and Mobile, both on the same project • RequireJS
  24. STACK • Java, Java everywhere • Huge PostgreSQL • Two

    different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website
  25. STACK • Java, Java everywhere • Huge PostgreSQL • Two

    different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website • LESS for Desktop website
  26. GIT AND DEPLOY • Since we use git for our

    projects, we work with pull requests
  27. GIT AND DEPLOY • Since we use git for our

    projects, we work with pull requests • We use Amazon Elastic Beanstalk
  28. GIT AND DEPLOY • Since we use git for our

    projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build
  29. GIT AND DEPLOY • Since we use git for our

    projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version
  30. GIT AND DEPLOY • Since we use git for our

    projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version • Unfortunately takes about half hour to finish deployment process
  31. URGENCY • Growth Team works under urgency • We need

    to validate hypothesis ASAP • With the current workflow it would be very slow
  32. STRATEGY • Growth Team should act like a third party

    application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work
  33. STRATEGY • Growth Team should act like a third party

    application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool
  34. STRATEGY • Growth Team should act like a third party

    application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool • A Ruby on Rails should app give me a JSON API to work with
  35. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  36. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  37. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  38. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  39. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  40. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  41. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  42. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  43. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head

    = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
  44. STRATEGY • No more bullshit code inside production codebase •

    No more approvals from other teams • Deploy in seconds
  45. STRATEGY • No more bullshit code inside production codebase •

    No more approvals from other teams • Deploy in seconds • Freedom to do anything inside the website
  46. STRATEGY • Handlebars for template • SASS for styling •

    Simple JavaScript for experiments • Rails as JSON API
  47. STRATEGY • Handlebars for template • SASS for styling •

    Simple JavaScript for experiments • Rails as JSON API • Grunt tasks for development and deploy
  48. (function() { var isPropertyDetailPage = window.location.pathname.match(/\/imovel\/|\/imoveis-lancamento\//), self = this; if

    (!isPropertyDetailPage) { var variationName = optimizely.variationNamesMap['2658391651'], template; if (variationName === 'Original') { return; } else if (variationName === 'Variation #1') { template = self.GrowthApp.Templates['experiment-1-phone-boxes']({ title: 'Baixe GRÁTIS o app do VivaReal!', subTitle: 'Enviaremos o link para seu celular :)', endPointURL: self.GrowthApp.urlOrigin('api/phone-boxes'), chevronURL: self.GrowthApp.urlOrigin('images/experiment-1/chevron.svg'), CTA: 'Enviar SMS' }); } $('body').append(template); $('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); });
  49. $('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); }); $('#experiment-input').on('focus', function()

    { optimizely.push(['trackEvent', 'focusOnSMSField']); }); $('.js-form').on('submit', function(event) { event.preventDefault(); var variationURL = $(this).data('url'), url = self.GrowthApp.Utils.urlOrigin(variationURL) + '.json', mobileNumber = $('#experiment-input').val(), leadData = {experiment_lead: {mobile_number: mobileNumber}}; $.post(url, leadData, function(data) { var message = data.message; $('.js-form p').html(message); $('.js-form input.sms-box__text-field').val(''); optimizely.push(['trackEvent', 'SMSSent']); }).fail(function(data) { alert('Ops, algo deu errado, tente novamente mais tarde!'); }); }); } })();
  50. <div class="sms-box {{boxSize}}"> <h5 class="sms-box__title js-olark-heading"> {{title}} <img src="{{chevronURL}}" class="sms-box__chevron">

    </h5> <form class="sms-box__form js-olark-form" accept-charset="utf-8" data-url="{{endPointURL}}"> <p class="sms-box__sub-title">{{subTitle}}</p> <input type="text" name="mobile_number" id="olark-experiment-1-input" placeholder="Digite seu celular" class="sms-box__text-field" required="required"> <input type="submit" value="{{CTA}}" class="sms-box__submit"> </form> </div><!-- sms-box -->
  51. class API::PhoneBoxesController < API::BaseController def create lead = PhoneLead.new(phone_lead_params) if

    lead.save render json: {message: phone_lead.success} else render json: {errors: phone_lead.errors.full_messages}, status: :unprocessable_entity end end private def phone_lead_params params.require(:phone_lead).permit(:mobile_number) end end
  52. class PhoneLead < ActiveRecord::Base after_create :send_sms validates_presence_of :mobile_number def send_sms

    client = Twilio::REST::Client.new normalized_number = "+55#{self.mobile_number.gsub(/[(),-,_]/, '')}" client.account.messages.create({ to: normalized_number, from: TWILIO_PHONE_NUMBERS.sample, body: 'Aqui esta o link para baixar gratis o app VivaReal: http://bit.ly/1-AppsVivaReal' }) end end
  53. RAILS • RSpec to test a bunch of crazy experiments

    • Simple JSON API • Easy to code ruby scripts and run them using cronjob to create reports
  54. RSPEC • The best time to send a SMS to

    users download our apps • 7 days of week
  55. RSPEC • The best time to send a SMS to

    users download our apps • 7 days of week • 3 times a day
  56. RSPEC • The best time to send a SMS to

    users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown)
  57. RSPEC • The best time to send a SMS to

    users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown) • 63 freaking links to check
  58. RSPEC • Why check those 63 links? • For attribution

    and tracking • We need to be 100% sure that those app installations came from the SMS we sent, from which time and day, and if those users became MAUs
  59. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do

    [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
  60. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do

    [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
  61. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do

    [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
  62. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do

    [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
  63. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do

    [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
  64. RSpec.describe API::SmartBannerDesktopController, type: :controller do describe 'create', :vcr do context

    'valid' do before :each do post :create, format: :json, non_app_user: {mobile_number: '(11) 99999-9900', experiment_id: 50} end it 'should respond 200' do expect(response.status).to eq 200 end it 'should respond success message' do expect(json['message']).to eq 'SMS Enviado com sucesso!' end it 'stores lead number on database' do expect(NonAppUser.last.mobile_number).to eq '+5511999999900' end end context 'invalid phone' do before :each do post :create, format: :json, non_app_user: {mobile_number: '', experiment_id: nil} end it 'should respond unprocessable entity' do expect(response.status).to eq 422 end it 'should respond rejection message' do expect(json['errors']).to eq ['Número do celular não pode ficar em branco'] end end end end
  65. AUTOMATED REPORTS • Every single day, a lib run in

    order to: • Get data from our marketing analytics platform
  66. AUTOMATED REPORTS • Every single day, a lib run in

    order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet
  67. AUTOMATED REPORTS • Every single day, a lib run in

    order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet • So, we can keep an eye on all numbers and funnels on a daily basis
  68. TRACK EVERYTHING • Never trust on your AB test platform

    alone • Track all your events on your AB test platform, plus track it on your analytics tool
  69. TRACK EVERYTHING • Never trust on your AB test platform

    alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that)
  70. TRACK EVERYTHING • Never trust on your AB test platform

    alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that) • Try to build your own user tracking and analytics platform, as we did in VivaReal
  71. TRACKING SERIOUS BUSINESS • Your AB test platform will probably

    bring you false positive results • Deliver all tracking results to your Data Scientist
  72. TRACKING SERIOUS BUSINESS • Your AB test platform will probably

    bring you false positive results • Deliver all tracking results to your Data Scientist • Re-run you experiments twice at least
  73. CEREBRO • If is possible, try to have your own

    platform • You have all data raw (non processed data)
  74. CEREBRO • If is possible, try to have your own

    platform • You have all data raw (non processed data) • More accurate results
  75. CEREBRO • If is possible, try to have your own

    platform • You have all data raw (non processed data) • More accurate results • More info: http://engenharia.vivareal.com.br/a-look- inside-cerebro-our-user-tracking-and-analytics-platform/
  76. DAILY REPORTS • Look your numbers on a daily basis

    • Act based on them • Be sure, to know why your numbers is going down or up
  77. DAILY REPORTS • Look your numbers on a daily basis

    • Act based on them • Be sure, to know why your numbers is going down or up • Know why is the most important thing, and just tracking your events carefully, you can really know what is going on

More Decks by Rubens Stulzer

Other Decks in Programming