Share
Embed
Using Rails to build Growth Hacks Fast
What is Growth Hacking and how it works
Back-end Technology Freedom
Front-end strategy
Handlebars for templates
Pushing data to track your user
Using AB Test tool
Rubens Stulzer
September 18, 2015
Transcript
192 slides
-
Ultimately Growth Hacking is the efforts of product, engineering and
marketing where the focus is grow -
EVERY GROWTH TEAM SHOULD • Have only one goal, and
it should be clean and simple -
EVERY GROWTH TEAM SHOULD • Have only one goal, and
it should be clean and simple • Validate hypothesis, preferably raised through data -
EVERY GROWTH TEAM SHOULD • Have only one goal, and
it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often -
EVERY GROWTH TEAM SHOULD • Have only one goal, and
it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure -
EVERY GROWTH TEAM SHOULD • Have only one goal, and
it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure • Read the numbers and understand them -
FOCUS • Grow Monthly Active Users on VivaReal mobile apps
• Two important things to do it -
FOCUS • Grow Monthly Active Users on VivaReal mobile apps
• Two important things to do it • Acquire more users through VivaReal Website -
FOCUS • Grow Monthly Active Users on VivaReal mobile apps
• Two important things to do it • Acquire more users through VivaReal Website • Activate more users on VivaReal Apps -
FUNNEL • There is a huge funnel to bring website
users to apps -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition -
FUNNEL • There is a huge funnel to bring website
users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition Activation -
EXPERIMENT VALIDATION • We don’t know nothing, never, we should
experiment it always -
EXPERIMENT VALIDATION • We don’t know nothing, never, we should
experiment it always • The easiest way to validate a hypothesis is through an A/B Test -
THE GOOD • Easy to implement the experiment code •
It is simple to pick a winner through experiments dashboard -
THE GOOD • Easy to implement the experiment code •
It is simple to pick a winner through experiments dashboard • Don’t need to be an expert on it to get things running -
THE BAD • No easy way to segment your users
• No easy way to configure traffic allocation for a given experiment -
THE UGLY <% ab_test(:login_button, "/images/button1.jpg", "/images/button2.jpg") do |button_file| %> <%=
image_tag(button_file, alt: "Login!") %> <% end %> <% the_test = ab_test('split-extras', 'splitted', 'not-splitted') %> <div class="row"> <div class="col-md-12"> <div class=“plans-options-duration plans-options-box-<%= 'center' if the_test == 'splitted' %>"> <!-- ... bunch of code --> </div> </div> </div> -
THE UGLY def index the_test = ab_test('new-home', 'new', 'old') if
the_test == 'new' render ‘new_index', layout: 'new_application' else render 'old_index', layout: 'application' end end -
THE UGLY • Doesn’t add good value to the codebase
• Someone will probably pick one variation, and you will forget that ugly piece of code there -
STACK • Java, Java everywhere • Huge PostgreSQL • Two
different websites for Desktop and Mobile, both on the same project -
STACK • Java, Java everywhere • Huge PostgreSQL • Two
different websites for Desktop and Mobile, both on the same project • RequireJS -
STACK • Java, Java everywhere • Huge PostgreSQL • Two
different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website -
STACK • Java, Java everywhere • Huge PostgreSQL • Two
different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website • LESS for Desktop website -
GIT AND DEPLOY • Since we use git for our
projects, we work with pull requests -
GIT AND DEPLOY • Since we use git for our
projects, we work with pull requests • We use Amazon Elastic Beanstalk -
GIT AND DEPLOY • Since we use git for our
projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build -
GIT AND DEPLOY • Since we use git for our
projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version -
GIT AND DEPLOY • Since we use git for our
projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version • Unfortunately takes about half hour to finish deployment process -
URGENCY • Growth Team works under urgency • We need
to validate hypothesis ASAP -
URGENCY • Growth Team works under urgency • We need
to validate hypothesis ASAP • With the current workflow it would be very slow -
STRATEGY • Growth Team should act like a third party
application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work -
STRATEGY • Growth Team should act like a third party
application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool -
STRATEGY • Growth Team should act like a third party
application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool • A Ruby on Rails should app give me a JSON API to work with -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
(function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head
= document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })(); -
STRATEGY • No more bullshit code inside production codebase •
No more approvals from other teams -
STRATEGY • No more bullshit code inside production codebase •
No more approvals from other teams • Deploy in seconds -
STRATEGY • No more bullshit code inside production codebase •
No more approvals from other teams • Deploy in seconds • Freedom to do anything inside the website -
STRATEGY • Handlebars for template • SASS for styling •
Simple JavaScript for experiments -
STRATEGY • Handlebars for template • SASS for styling •
Simple JavaScript for experiments • Rails as JSON API -
STRATEGY • Handlebars for template • SASS for styling •
Simple JavaScript for experiments • Rails as JSON API • Grunt tasks for development and deploy -
(function() { var isPropertyDetailPage = window.location.pathname.match(/\/imovel\/|\/imoveis-lancamento\//), self = this; if
(!isPropertyDetailPage) { var variationName = optimizely.variationNamesMap['2658391651'], template; if (variationName === 'Original') { return; } else if (variationName === 'Variation #1') { template = self.GrowthApp.Templates['experiment-1-phone-boxes']({ title: 'Baixe GRÁTIS o app do VivaReal!', subTitle: 'Enviaremos o link para seu celular :)', endPointURL: self.GrowthApp.urlOrigin('api/phone-boxes'), chevronURL: self.GrowthApp.urlOrigin('images/experiment-1/chevron.svg'), CTA: 'Enviar SMS' }); } $('body').append(template); $('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); }); -
$('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); }); $('#experiment-input').on('focus', function()
{ optimizely.push(['trackEvent', 'focusOnSMSField']); }); $('.js-form').on('submit', function(event) { event.preventDefault(); var variationURL = $(this).data('url'), url = self.GrowthApp.Utils.urlOrigin(variationURL) + '.json', mobileNumber = $('#experiment-input').val(), leadData = {experiment_lead: {mobile_number: mobileNumber}}; $.post(url, leadData, function(data) { var message = data.message; $('.js-form p').html(message); $('.js-form input.sms-box__text-field').val(''); optimizely.push(['trackEvent', 'SMSSent']); }).fail(function(data) { alert('Ops, algo deu errado, tente novamente mais tarde!'); }); }); } })(); -
<div class="sms-box {{boxSize}}"> <h5 class="sms-box__title js-olark-heading"> {{title}} <img src="{{chevronURL}}" class="sms-box__chevron">
</h5> <form class="sms-box__form js-olark-form" accept-charset="utf-8" data-url="{{endPointURL}}"> <p class="sms-box__sub-title">{{subTitle}}</p> <input type="text" name="mobile_number" id="olark-experiment-1-input" placeholder="Digite seu celular" class="sms-box__text-field" required="required"> <input type="submit" value="{{CTA}}" class="sms-box__submit"> </form> </div><!-- sms-box --> -
class API::PhoneBoxesController < API::BaseController def create lead = PhoneLead.new(phone_lead_params) if
lead.save render json: {message: phone_lead.success} else render json: {errors: phone_lead.errors.full_messages}, status: :unprocessable_entity end end private def phone_lead_params params.require(:phone_lead).permit(:mobile_number) end end -
class PhoneLead < ActiveRecord::Base after_create :send_sms validates_presence_of :mobile_number def send_sms
client = Twilio::REST::Client.new normalized_number = "+55#{self.mobile_number.gsub(/[(),-,_]/, '')}" client.account.messages.create({ to: normalized_number, from: TWILIO_PHONE_NUMBERS.sample, body: 'Aqui esta o link para baixar gratis o app VivaReal: http://bit.ly/1-AppsVivaReal' }) end end -
RAILS • RSpec to test a bunch of crazy experiments
• Simple JSON API -
RAILS • RSpec to test a bunch of crazy experiments
• Simple JSON API • Easy to code ruby scripts and run them using cronjob to create reports -
RSPEC • The best time to send a SMS to
users download our apps -
RSPEC • The best time to send a SMS to
users download our apps • 7 days of week -
RSPEC • The best time to send a SMS to
users download our apps • 7 days of week • 3 times a day -
RSPEC • The best time to send a SMS to
users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown) -
RSPEC • The best time to send a SMS to
users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown) • 63 freaking links to check -
RSPEC • Why check those 63 links? • For attribution
and tracking -
RSPEC • Why check those 63 links? • For attribution
and tracking • We need to be 100% sure that those app installations came from the SMS we sent, from which time and day, and if those users became MAUs -
RSpec.describe ToStoreController, type: :controller do describe 'each week day' do
[:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end -
RSpec.describe ToStoreController, type: :controller do describe 'each week day' do
[:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end -
RSpec.describe ToStoreController, type: :controller do describe 'each week day' do
[:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end -
RSpec.describe ToStoreController, type: :controller do describe 'each week day' do
[:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end -
RSpec.describe ToStoreController, type: :controller do describe 'each week day' do
[:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end -
~/growth-app master= ∴ ............................................................... Finished in 0.28754 seconds 63 examples,
0 failures ./bin/rspec spec/controllers/to_store_controller_spec.rb -
RSpec.describe API::SmartBannerDesktopController, type: :controller do describe 'create', :vcr do context
'valid' do before :each do post :create, format: :json, non_app_user: {mobile_number: '(11) 99999-9900', experiment_id: 50} end it 'should respond 200' do expect(response.status).to eq 200 end it 'should respond success message' do expect(json['message']).to eq 'SMS Enviado com sucesso!' end it 'stores lead number on database' do expect(NonAppUser.last.mobile_number).to eq '+5511999999900' end end context 'invalid phone' do before :each do post :create, format: :json, non_app_user: {mobile_number: '', experiment_id: nil} end it 'should respond unprocessable entity' do expect(response.status).to eq 422 end it 'should respond rejection message' do expect(json['errors']).to eq ['Número do celular não pode ficar em branco'] end end end end -
AUTOMATED REPORTS • Every single day, a lib run in
order to: • Get data from our marketing analytics platform -
AUTOMATED REPORTS • Every single day, a lib run in
order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet -
AUTOMATED REPORTS • Every single day, a lib run in
order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet • So, we can keep an eye on all numbers and funnels on a daily basis -
TRACKING DONE RIGHT Track user events is crucial to grow
your business -
TRACK EVERYTHING • Never trust on your AB test platform
alone • Track all your events on your AB test platform, plus track it on your analytics tool -
TRACK EVERYTHING • Never trust on your AB test platform
alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that) -
TRACK EVERYTHING • Never trust on your AB test platform
alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that) • Try to build your own user tracking and analytics platform, as we did in VivaReal -
TRACKING SERIOUS BUSINESS • Your AB test platform will probably
bring you false positive results -
TRACKING SERIOUS BUSINESS • Your AB test platform will probably
bring you false positive results • Deliver all tracking results to your Data Scientist -
TRACKING SERIOUS BUSINESS • Your AB test platform will probably
bring you false positive results • Deliver all tracking results to your Data Scientist • Re-run you experiments twice at least -
CEREBRO • If is possible, try to have your own
platform • You have all data raw (non processed data) -
CEREBRO • If is possible, try to have your own
platform • You have all data raw (non processed data) • More accurate results -
CEREBRO • If is possible, try to have your own
platform • You have all data raw (non processed data) • More accurate results • More info: http://engenharia.vivareal.com.br/a-look- inside-cerebro-our-user-tracking-and-analytics-platform/ -
DAILY REPORTS • Look your numbers on a daily basis
• Act based on them -
DAILY REPORTS • Look your numbers on a daily basis
• Act based on them • Be sure, to know why your numbers is going down or up -
DAILY REPORTS • Look your numbers on a daily basis
• Act based on them • Be sure, to know why your numbers is going down or up • Know why is the most important thing, and just tracking your events carefully, you can really know what is going on