Metrics we should know - or not! but we are gonna learn it!

This is a story about maths!
In this talk, we are gonna focus completely on metrics, from AB testing to analytics using Firebase and the importance that we need to address metrics when we start working on a project!
We are gonna talk about Bayesian and predictions models to track our AB testings and also we are gonna talk about Conversion vs. Retention vs. Count perusers

Bring your calculators! because this is a story about Firebase and Charts!

Transcript

1. Metrics we should know -
   or not! but we are gonna
   learn it!
   Dinorah Tovar
   
   Google Developer Expert - Android
   @ddinorahtovar
   @ddinorahtovar
   

   View Slide

2. Disclaimer
   

   View Slide

3. A story about
   data
   

   View Slide

4. @ddinorahtovar
   

   View Slide

5. Data gives the answer to the correct
   questions
   @ddinorahtovar
   All hit on the same place
   Planes on the research
   Result
   Research
   

   View Slide

6. Analytics
   

   View Slide

7. Data vs Metrics
   @ddinorahtovar
   •They are not the same
   •Data is
   generated by
   metrics
   
   •You don’t pick
   your data, you
   pick your
   metrics
   Data
   Metrics
   •Metrics are what
   you measure
   
   •Consistent
   
   •Cheap
   
   •Quick to collect
   

   View Slide

8. Analytics
   @ddinorahtovar
   •Analytics is the systematic computational analysis of data or
   statistics
   •Provides insight on app usage and user engagement.
   •Helps you understand how your users behave
   

   View Slide

9. Firebase has what you ned and more
   @ddinorahtovar
   

   View Slide

10. How does an analytic looks in Firebase
    @ddinorahtovar
    firebaseAnalytics.logEvent("SomeNameOfTheEvent") {
    
    
    // Extra parameters if you need more info
    
    
    param(FirebaseAnalytics.Param.ITEM_ID, id)
    
    
    param(FirebaseAnalytics.Param.ITEM_NAME, name)
    
    
    param(FirebaseAnalytics.Param.CONTENT_TYPE, "image")
    
    
    }
    
    
    •On android
    

    View Slide

11. How does an analytic looks in Firebase
    @ddinorahtovar
    Analytics.logEvent(AnalyticsEventSelectContent, parameters: [
    
    
    AnalyticsParameterItemID: "id-\(title!)",
    
    
    AnalyticsParameterItemName: title!,
    
    
    AnalyticsParameterContentType: "cont",
    
    
    ])
    •On iOS
    

    View Slide

12. Conversion
    

    View Slide

13. First, an example
    @ddinorahtovar
    app-ca app-email app-account
    🏆
    

    View Slide

14. Conversion
    @ddinorahtovar
    •Conversion analysis is the process of analyzing data related to
    conversions
    •A conversion is defined as a specific, desirable action that’s
    taken by a user
    •Depends on the analytics selected
    

    View Slide

15. Conversion
    @ddinorahtovar
    •Conversion rate can be calculated over a math func
    coR = totalConversions/totalInteractions
    coR = 50/1000
    coR = 5%
    

    View Slide

16. Conversion
    @ddinorahtovar
    Conversion
    Event
    
    app-account
    Second Event
    
    app-email
    
    First Event
    
    

    app-ca
    

    View Slide

17. Conversion
    @ddinorahtovar
    

    View Slide

18. Conversion
    @ddinorahtovar
    •Conversions are
    related to multiple
    events
    •We decide which is
    the event we are
    interested
    

    View Slide

19. Conversion
    @ddinorahtovar
    •Conversions are directly
    related to the num of users
    and the number of clicks on
    the conversion
    •But it also can be unique
    

    View Slide

20. Conversion
    @ddinorahtovar
    •Conversions are directly
    related to time dimensions
    

    View Slide

21. Retention
    

    View Slide

22. Retention
    @ddinorahtovar
    •Retention analysis (or survival analysis) is the process of analyzing
    user metrics to understand how and why customers churn
    •Retention analysis is key to gain insights on how to maintain a
    profitable customer base by improving retention and new user
    acquisition rates
    

    View Slide

23. Retention
    @ddinorahtovar
    Time - Survival probability
    0
    0.25
    0.5
    0.75
    1
    Time (months)
    0 2 3 5 7 9 10 12
    60% of probability
    of surviving beyond
    5 months
    - 25% of probability
    of surviving beyond
    11 months
    

    View Slide

24. Retention
    @ddinorahtovar
    •Firebase - has this
    chart - you don’t have
    to do anything just
    plug the dependencies
    in your app and you
    are ready to go
    

    View Slide

25. AB Testing
    

    View Slide

26. But first - what is an prediction model
    @ddinorahtovar
    •Assume the
    probabilities
    for both data
    and
    hypotheses(para
    meters
    specifying the
    distribution of
    the data)
    Bayesian
    Frequentist
    •Assume the
    observed data is
    sampled from
    some
    distribution
    •Frequentist vs Bayesian
    

    View Slide

27. Bayesian Models
    @ddinorahtovar
    •How does it looks
    More overlap = Less con
    fi
    dence
    Less overlap = More
    con
    fi
    dence
    

    View Slide

28. Bayesian Models
    @ddinorahtovar
    •How does it looks
    

    View Slide

29. Metrics we should know -
    or not! but we are gonna
    learn it!
    Dinorah Tovar
    
    Google Developer Expert - Android
    @ddinorahtovar
    @ddinorahtovar
    

    View Slide