Using Rails to build Growth Hacks Fast

What is Growth Hacking and how it works
Back-end Technology Freedom
Front-end strategy
Handlebars for templates
Pushing data to track your user
Using AB Test tool

Transcript

2. USING RAILS TO BUILD GROWTH HACKS FAST
3. RUBENS STULZER Full Stack Rails Developer ~ UX Designer @RubensStulzer stulzer
5. GROWTH HACKING A little overview
7. Ultimately Growth Hacking is the efforts of product, engineering and marketing where the focus is grow
9. User Experience
10. User Experience Engineering
11. User Experience Engineering Data Science
12. User Experience Engineering Data Science Marketing
13. User Experience Engineering Data Science Marketing GROWTH
15. EVERY GROWTH TEAM SHOULD
16. EVERY GROWTH TEAM SHOULD • Have only one goal, and it should be clean and simple
17. EVERY GROWTH TEAM SHOULD • Have only one goal, and it should be clean and simple • Validate hypothesis, preferably raised through data
18. EVERY GROWTH TEAM SHOULD • Have only one goal, and it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often
19. EVERY GROWTH TEAM SHOULD • Have only one goal, and it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure
20. EVERY GROWTH TEAM SHOULD • Have only one goal, and it should be clean and simple • Validate hypothesis, preferably raised through data • Be prepared to fail fast, and fail often • Learn every single time you are validating something, specially if it is a failure • Read the numbers and understand them
21. FOCUS FOR GROWTH Acquisition and Activation
23. FOCUS
24. FOCUS • Grow Monthly Active Users on VivaReal mobile apps
25. FOCUS • Grow Monthly Active Users on VivaReal mobile apps • Two important things to do it
26. FOCUS • Grow Monthly Active Users on VivaReal mobile apps • Two important things to do it • Acquire more users through VivaReal Website
27. FOCUS • Grow Monthly Active Users on VivaReal mobile apps • Two important things to do it • Acquire more users through VivaReal Website • Activate more users on VivaReal Apps
29. FUNNEL
30. FUNNEL • There is a huge funnel to bring website users to apps
31. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps
32. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store
33. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it
34. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it
35. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property
36. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead
37. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition
38. FUNNEL • There is a huge funnel to bring website users to apps • They need to know about our apps • They need to go to App Store • Download it • Open it • Search for a property • And than send a lead Acquisition Activation
40. EXPERIMENT VALIDATION
41. EXPERIMENT VALIDATION • We don’t know nothing, never, we should experiment it always
42. EXPERIMENT VALIDATION • We don’t know nothing, never, we should experiment it always • The easiest way to validate a hypothesis is through an A/B Test
43. AB TEST IN RAILS The Good, the Bad and the Ugly
45. THE GOOD
46. THE GOOD • Easy to implement the experiment code
47. THE GOOD • Easy to implement the experiment code • It is simple to pick a winner through experiments dashboard
48. THE GOOD • Easy to implement the experiment code • It is simple to pick a winner through experiments dashboard • Don’t need to be an expert on it to get things running
50. THE BAD
51. THE BAD • No easy way to segment your users
52. THE BAD • No easy way to segment your users • No easy way to configure traffic allocation for a given experiment
54. THE UGLY
55. THE UGLY <% ab_test(:login_button, "/images/button1.jpg", "/images/button2.jpg") do |button_file| %> <%= image_tag(button_file, alt: "Login!") %> <% end %> <% the_test = ab_test('split-extras', 'splitted', 'not-splitted') %> <div class="row"> <div class="col-md-12"> <div class=“plans-options-duration plans-options-box-<%= 'center' if the_test == 'splitted' %>"> <!-- ... bunch of code --> </div> </div> </div>
56. THE UGLY
57. THE UGLY
58. THE UGLY def index the_test = ab_test('new-home', 'new', 'old') if the_test == 'new' render ‘new_index', layout: 'new_application' else render 'old_index', layout: 'application' end end
59. THE UGLY
60. THE UGLY
61. THE UGLY • Doesn’t add good value to the codebase
62. THE UGLY • Doesn’t add good value to the codebase • Someone will probably pick one variation, and you will forget that ugly piece of code there
63. HOW VIVAREAL WORKS Brief overview about our stack
65. STACK
66. STACK • Java, Java everywhere
67. STACK • Java, Java everywhere
68. STACK • Java, Java everywhere
69. STACK • Java, Java everywhere • Huge PostgreSQL
70. STACK • Java, Java everywhere • Huge PostgreSQL • Two different websites for Desktop and Mobile, both on the same project
71. STACK • Java, Java everywhere • Huge PostgreSQL • Two different websites for Desktop and Mobile, both on the same project • RequireJS
72. STACK • Java, Java everywhere • Huge PostgreSQL • Two different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website
73. STACK • Java, Java everywhere • Huge PostgreSQL • Two different websites for Desktop and Mobile, both on the same project • RequireJS • SASS for Mobile website • LESS for Desktop website
75. GIT AND DEPLOY
76. GIT AND DEPLOY • Since we use git for our projects, we work with pull requests
77. GIT AND DEPLOY • Since we use git for our projects, we work with pull requests • We use Amazon Elastic Beanstalk
78. GIT AND DEPLOY • Since we use git for our projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build
79. GIT AND DEPLOY • Since we use git for our projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version
80. GIT AND DEPLOY • Since we use git for our projects, we work with pull requests • We use Amazon Elastic Beanstalk • A huge jenkins to run the build • We make the swap to the new version • Unfortunately takes about half hour to finish deployment process
82. URGENCY
83. URGENCY • Growth Team works under urgency
84. URGENCY • Growth Team works under urgency • We need to validate hypothesis ASAP
85. URGENCY • Growth Team works under urgency • We need to validate hypothesis ASAP • With the current workflow it would be very slow
86. GROWTH TEAM STRATEGY Freedom to do whatever we want
88. STRATEGY
89. STRATEGY • Growth Team should act like a third party application
90. STRATEGY • Growth Team should act like a third party application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work
91. STRATEGY • Growth Team should act like a third party application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool
92. STRATEGY • Growth Team should act like a third party application • One simple javascript should inject a script tag and a stylesheet tag containing the code to make experiments work • Experiments should be controlled through external tool • A Ruby on Rails should app give me a JSON API to work with
94. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
95. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
96. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
97. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
98. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
99. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
100. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
101. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
102. (function() { var scriptElement = document.createElement('script'), body = document.getElementsByTagName('body')[0], head = document.head, link = document.createElement('link'); scriptElement.type = 'text/javascript'; scriptElement.async = true; link.type = 'text/css'; link.rel = 'stylesheet'; scriptElement.src = '//growth.vivareal.com.br/build/lets-grow.js'; link.href = 'http://growth.vivareal.com.br/styles.css'; body.appendChild(scriptElement); head.appendChild(link); })();
104. STRATEGY
105. STRATEGY • No more bullshit code inside production codebase
106. STRATEGY • No more bullshit code inside production codebase • No more approvals from other teams
107. STRATEGY • No more bullshit code inside production codebase • No more approvals from other teams • Deploy in seconds
108. STRATEGY • No more bullshit code inside production codebase • No more approvals from other teams • Deploy in seconds • Freedom to do anything inside the website
109. STRATEGY
110. STRATEGY • Handlebars for template
111. STRATEGY • Handlebars for template • SASS for styling
112. STRATEGY • Handlebars for template • SASS for styling • Simple JavaScript for experiments
113. STRATEGY • Handlebars for template • SASS for styling • Simple JavaScript for experiments • Rails as JSON API
114. STRATEGY • Handlebars for template • SASS for styling • Simple JavaScript for experiments • Rails as JSON API • Grunt tasks for development and deploy
115. (function() { var isPropertyDetailPage = window.location.pathname.match(/\/imovel\/|\/imoveis-lancamento\//), self = this; if (!isPropertyDetailPage) { var variationName = optimizely.variationNamesMap['2658391651'], template; if (variationName === 'Original') { return; } else if (variationName === 'Variation #1') { template = self.GrowthApp.Templates['experiment-1-phone-boxes']({ title: 'Baixe GRÁTIS o app do VivaReal!', subTitle: 'Enviaremos o link para seu celular :)', endPointURL: self.GrowthApp.urlOrigin('api/phone-boxes'), chevronURL: self.GrowthApp.urlOrigin('images/experiment-1/chevron.svg'), CTA: 'Enviar SMS' }); } $('body').append(template); $('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); });
116. $('.js-heading').on('click', function() { $('.js-form').slideToggle(); $('.sms-box__chevron').toggleClass('sms-box__chevron--is-active'); optimizely.push(['trackEvent', 'triggerSMSBox']); }); $('#experiment-input').on('focus', function() { optimizely.push(['trackEvent', 'focusOnSMSField']); }); $('.js-form').on('submit', function(event) { event.preventDefault(); var variationURL = $(this).data('url'), url = self.GrowthApp.Utils.urlOrigin(variationURL) + '.json', mobileNumber = $('#experiment-input').val(), leadData = {experiment_lead: {mobile_number: mobileNumber}}; $.post(url, leadData, function(data) { var message = data.message; $('.js-form p').html(message); $('.js-form input.sms-box__text-field').val(''); optimizely.push(['trackEvent', 'SMSSent']); }).fail(function(data) { alert('Ops, algo deu errado, tente novamente mais tarde!'); }); }); } })();
118. <div class="sms-box {{boxSize}}"> <h5 class="sms-box__title js-olark-heading"> {{title}} <img src="{{chevronURL}}" class="sms-box__chevron"> </h5> <form class="sms-box__form js-olark-form" accept-charset="utf-8" data-url="{{endPointURL}}"> <p class="sms-box__sub-title">{{subTitle}}</p> <input type="text" name="mobile_number" id="olark-experiment-1-input" placeholder="Digite seu celular" class="sms-box__text-field" required="required"> <input type="submit" value="{{CTA}}" class="sms-box__submit"> </form> </div><!-- sms-box -->
120. class API::PhoneBoxesController < API::BaseController def create lead = PhoneLead.new(phone_lead_params) if lead.save render json: {message: phone_lead.success} else render json: {errors: phone_lead.errors.full_messages}, status: :unprocessable_entity end end private def phone_lead_params params.require(:phone_lead).permit(:mobile_number) end end
122. class PhoneLead < ActiveRecord::Base after_create :send_sms validates_presence_of :mobile_number def send_sms client = Twilio::REST::Client.new normalized_number = "+55#{self.mobile_number.gsub(/[(),-,_]/, '')}" client.account.messages.create({ to: normalized_number, from: TWILIO_PHONE_NUMBERS.sample, body: 'Aqui esta o link para baixar gratis o app VivaReal: http://bit.ly/1-AppsVivaReal' }) end end
123. RAILS Why Rails is so important to us
125. RAILS
126. RAILS • RSpec to test a bunch of crazy experiments
127. RAILS • RSpec to test a bunch of crazy experiments • Simple JSON API
128. RAILS • RSpec to test a bunch of crazy experiments • Simple JSON API • Easy to code ruby scripts and run them using cronjob to create reports
130. RSPEC
131. RSPEC • The best time to send a SMS to users download our apps
132. RSPEC • The best time to send a SMS to users download our apps • 7 days of week
133. RSPEC • The best time to send a SMS to users download our apps • 7 days of week • 3 times a day
134. RSPEC • The best time to send a SMS to users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown)
135. RSPEC • The best time to send a SMS to users download our apps • 7 days of week • 3 times a day • 3 links for each time (iOS, Android, Unknown) • 63 freaking links to check
136. RSPEC
137. RSPEC
138. RSPEC • Why check those 63 links?
139. RSPEC • Why check those 63 links? • For attribution and tracking
140. RSPEC • Why check those 63 links? • For attribution and tracking • We need to be 100% sure that those app installations came from the SMS we sent, from which time and day, and if those users became MAUs
142. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
143. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
144. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
145. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
146. RSpec.describe ToStoreController, type: :controller do describe 'each week day' do [:monday, :tuesday, :wednesday, :thursday, :friday, :saturday, :sunday].each do |day| [:morning, :afternoon, :night].each do |period| let(:day_period) {"#{day}_#{period}"} let(:month_day) = case_month_day day let(:time) = case_period period context 'redirects user to VivaReal' do it "page on AppStore based on iPhone UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = IOS_UA get day_period expect(response.location).to match(/iOS-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "page on Google Play based on Android UA for period: #{day} #{period}" do request.env['HTTP_USER_AGENT'] = ANDROID_UA get day_period expect(response.location).to match(/Android-SMStest-2015.03.#{month_day}\-#{time}&my_publisher=SMS/) end it "mobile page: #{day} #{period}" do get day_period expect(response.location).to match(/vivareal.com.br\/mobile/) end end end end end end
147. ~/growth-app master= ∴
148. ~/growth-app master= ∴ ./bin/rspec spec/controllers/to_store_controller_spec.rb
149. ~/growth-app master= ∴ ............................................................... Finished in 0.28754 seconds 63 examples, 0 failures ./bin/rspec spec/controllers/to_store_controller_spec.rb
151. RSpec.describe API::SmartBannerDesktopController, type: :controller do describe 'create', :vcr do context 'valid' do before :each do post :create, format: :json, non_app_user: {mobile_number: '(11) 99999-9900', experiment_id: 50} end it 'should respond 200' do expect(response.status).to eq 200 end it 'should respond success message' do expect(json['message']).to eq 'SMS Enviado com sucesso!' end it 'stores lead number on database' do expect(NonAppUser.last.mobile_number).to eq '+5511999999900' end end context 'invalid phone' do before :each do post :create, format: :json, non_app_user: {mobile_number: '', experiment_id: nil} end it 'should respond unprocessable entity' do expect(response.status).to eq 422 end it 'should respond rejection message' do expect(json['errors']).to eq ['Número do celular não pode ficar em branco'] end end end end
153. AUTOMATED REPORTS
154. AUTOMATED REPORTS • Every single day, a lib run in order to:
155. AUTOMATED REPORTS • Every single day, a lib run in order to: • Get data from our marketing analytics platform
156. AUTOMATED REPORTS • Every single day, a lib run in order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet
157. AUTOMATED REPORTS • Every single day, a lib run in order to: • Get data from our marketing analytics platform • Add and Organize that data on Google Drive Spreadsheet • So, we can keep an eye on all numbers and funnels on a daily basis
158. TRACKING DONE RIGHT Track user events is crucial to grow your business
160. TRACK EVERYTHING
161. TRACK EVERYTHING • Never trust on your AB test platform alone
162. TRACK EVERYTHING • Never trust on your AB test platform alone • Track all your events on your AB test platform, plus track it on your analytics tool
163. TRACK EVERYTHING • Never trust on your AB test platform alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that)
164. TRACK EVERYTHING • Never trust on your AB test platform alone • Track all your events on your AB test platform, plus track it on your analytics tool • Hire a Data Scientist (good luck with that) • Try to build your own user tracking and analytics platform, as we did in VivaReal
166. TRACKING SERIOUS BUSINESS
167. TRACKING SERIOUS BUSINESS • Your AB test platform will probably bring you false positive results
168. TRACKING SERIOUS BUSINESS • Your AB test platform will probably bring you false positive results • Deliver all tracking results to your Data Scientist
169. TRACKING SERIOUS BUSINESS • Your AB test platform will probably bring you false positive results • Deliver all tracking results to your Data Scientist • Re-run you experiments twice at least
171. CEREBRO
172. CEREBRO • If is possible, try to have your own platform
173. CEREBRO • If is possible, try to have your own platform • You have all data raw (non processed data)
174. CEREBRO • If is possible, try to have your own platform • You have all data raw (non processed data) • More accurate results
175. CEREBRO • If is possible, try to have your own platform • You have all data raw (non processed data) • More accurate results • More info: http://engenharia.vivareal.com.br/a-look- inside-cerebro-our-user-tracking-and-analytics-platform/
177. DAILY REPORTS
178. DAILY REPORTS • Look your numbers on a daily basis
179. DAILY REPORTS • Look your numbers on a daily basis • Act based on them
180. DAILY REPORTS • Look your numbers on a daily basis • Act based on them • Be sure, to know why your numbers is going down or up
181. DAILY REPORTS • Look your numbers on a daily basis • Act based on them • Be sure, to know why your numbers is going down or up • Know why is the most important thing, and just tracking your events carefully, you can really know what is going on
182. SUCCESS Growth Hacking is hard
184. Golden metric on mobile apps 2014 2015
185. Golden metric on mobile apps 2014 2015
186. Golden metric on mobile apps 2014 2015
187. Golden metric on mobile apps 2014 2015
188. Golden metric on mobile apps 2014 2015
189. One last thing…
190. IS HIRING, WE NEED YOU
191. http://engenharia.vivareal.com.br/work-with-us/
192. THANK YOU