Business Intelligence Seminar

Slide 1

Slide 1 text

BUSINESS INTELLIGENCE

Slide 2

Slide 2 text

A brief of Business Intelligence that enables you to access and analyze information so you can improve and optimizing your business decision and performance. WHAT

Slide 3

Slide 3 text

Challenges of current situation. Why you need Business Intelligence? Complex Business Problem

Slide 4

Slide 4 text

The number of possible solutions is so large that it precludes a complete search for the best answer. 01 The problem exists in a time-changing environment. 02 The problem is heavily constrained 03 There are many (possibly conﬂicting) objectives 04

Slide 5

Slide 5 text

The necessary data were not recorded. Incomplete Information 1 The data are not reliable. Uncertainty 3 The data contain rounded ﬁgures and estimates. Noisy Data 2

Slide 6

Slide 6 text

Academic Research (Social Science) Theory Hypothesis Data Exploration Metrics Insight Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 7

Slide 7 text

Case of Business Intelligence Overview

Slide 8

Slide 8 text

Motivation “People in both ﬁelds operate with beliefs and biases. To the extent you can eliminate both and replace them with data, you gain a clear advantage” Michael Lewis, Moneyball: The Art of Winning an Unfair Game

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Variables: ● Team ● League ● Year ● Runs Scored (RS) ● Runs Allowed (RA) ● Wins (W) ● On-Base Percentage (OBP) ● Slugging Percentage (SLG) ● Batting Average (BA) ● Playoffs (binary) ● RankSeason ● RankPlayoffs ● Games Played (G) ● Opponent On-Base Percentage (OOBP) ● Opponent Slugging Percentage (OSLG) Moneyball Case: Intelligence Part 1 Collecting Data

Slide 12

Slide 12 text

Moneyball Case: Intelligence Part 2 Information Processing How does a team make it to playoffs? To be exact, how many games did it take to make it to playoffs? Target Wins: 95

Slide 13

Slide 13 text

Moneyball Case: Intelligence Part 3 Gaining More Knowledge How many more runs do we need to score than we allow in order to win 95 games in the regular season? Run Differential (RD) = Run Scored (RS) - Run Allowed (RA)

Slide 14

Slide 14 text

Moneyball Case: Intelligence Part 4 Build a Metrics To achieve the goal, we need to measure how much [metrics] do we need for Run Scored, Runs Allowed, and Run Differential?

Slide 15

Slide 15 text

Moneyball Case: Intelligence Part 4 Build a Metrics W = 80.8814 + 0.1058(RD) RD = (W - 80.8814) / 0.1058 Replace W with 95 RD = (95 - 80.8814) / 0.1058 RD ≈ 133

Slide 16

Slide 16 text

Moneyball Case: Intelligence Part 5 Optimal Decision Support Drop BA (Batting Average)?

Slide 17

Slide 17 text

What OBP and SLG do we need to achieve a run differential of +133? The OOBP and OSLG for the A's in 2001 were: ● OOBP = 0.315 ● OSLG = 0.384 So the estimated RA ≈ 662 The actual value of runs allowed in the 2002 season was 654. Moneyball Case: Intelligence Part 5 Optimal Decision Support

Slide 18

Slide 18 text

Moneyball Case: Intelligence Part 5 Optimal Decision Support What OOBP and OSLG do we need to achieve a run differential of +133? The OBP and SLG for the A's in 2001 were: ● OBP = 0.339 ● SLG = 0.432 So the RA ≈ 808 The actual value of runs scored in the 2002 season was 800.

Slide 19

Slide 19 text

Moneyball Case: Intelligence Part 5 Optimal Decision Support Actual RD = 800 - 654 = 156 W = 80.8814 + 0.1058(RD) W = 80.8814 + 0.1058(156) W ≈ 96 The actual value of games won in the 2002 season was 103.

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Data-Driven for Decision Making The ability to use existing data in a new way or obtain data to make decisions with conﬁdence that creates meaningful change Problem Decision PROBLEM SOLVING

Slide 22

Slide 22 text

Business Intelligence Dimensions

Slide 23

Slide 23 text

What is Business Intelligence? Business intelligence (BI) is a set of theories, methodologies, architectures, and technologies that transform raw data into meaningful and useful information for business purposes. 3 2 1 Lots and lots of data 01 Processing and Aggregation 02 Insight & Visualization 03

Slide 24

Slide 24 text

Descriptive Analytics Diagnostic Analytics Predictive Analytics Prescriptive Analytics Information Optimization What Happened? Why did it happened? What will happen? How can we make it happen? Early Stage of company Amount of data Analytics & Company Maturity Seed Growth Beyond

Slide 25

Slide 25 text

Customer Development Cycle A company is a sequence of hypothesis testing. https://xweb.stanford.edu/group/e145/cgi-bin/wi nter/drupal/upload/handouts/Four_Steps.pdf

Slide 26

Slide 26 text

Data Acquisition

Slide 27

Slide 27 text

Business Intelligence Data Exploration Insight Hypothesis Metrics A/B Test Decision

Slide 28

Slide 28 text

Getting Your First Data Survey FGD

Slide 29

Slide 29 text

Surveys can provide timely data using speciﬁc survey questions. “Designing and Conducting Business Survey” - Ger Snijkers, Gustav Haraldsen, Jacqui Jones. 2013) Getting Your First Data: Survey

Slide 30

Slide 30 text

Getting Your First Data: Survey General Flow Business Intelligence Design Scope, variables Build sample, questions Test Launch, processing, get insight

Slide 31

Slide 31 text

Getting Your First Data: FGD To identify a range of perspectives on some topic/issue, to gain an understanding of the topic/issue from the perspective of participants themselves

Slide 32

Slide 32 text

Getting Your First Data: FGD 1 6 2 3 4 5 Explore 2 Group Process Gain Diversity 4 Explain 5 Evaluate 6 Design

Slide 33

Slide 33 text

The market research approach typically uses focus group discussions to gain consumer views on new products or marketing campaigns (Kroll et al., 2007; Bloor et al., 2001). Getting Your First Data: FGD

Slide 34

Slide 34 text

Problems ● Your opinion/belief is only a hypothesis. ● The obvious things based on your opinion might be a fallacy. ● The ﬁrst person you can be easily fooled is yourself. ● In order to falsify your opinion or belief, you can gather data and compare it as a source of “truth”. Getting Your First Data: Case

Slide 35

Slide 35 text

Case I : Motorola Motorola’s Iridium satellite-based phone system. Engineering triumph and built to support a customer base of millions. No one asked the customer if they wanted it. Cost $5 billion. Yes, billion. Satellites are awfully expensive. Getting Your First Data: Case

Slide 36

Slide 36 text

Case II : Smokeless Cigarettes R.J. Reynolds’ Premier and Eclipse smokeless cigarettes. Understood what the general public (nonsmokers) wanted, but did not understand that their customers didn’t care. Cost: $450 million Getting Your First Data: Case

Slide 37

Slide 37 text

Case III : Toothpaste A toothpaste company want to increase their sales. Q: But How? Let’s do a FGD. Focus Group Discussion Person 1: I like the aluminium tube. The toothpaste smells nice. Person 2: The price is affordable. I can easily squeeze the aluminium tube. Person 3: It taste good. My child might eat it. Getting Your First Data: Case

Slide 38

Slide 38 text

Case III : Toothpaste Design a new tube, made of plastic, so users can’t squeeze all the toothpaste, and buy more. Getting Your First Data: Case

Slide 39

Slide 39 text

Getting Your First Data Conclusion You need to gather your ﬁrst data, either with FGD or Survey or secondary source and use it to falsify your opinion. Data triumph intuition!

Slide 40

Slide 40 text

Data Exploration and Visualization

Slide 41

Slide 41 text

Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 42

Slide 42 text

Problems ● You already have data from survey/FGD. ● You want to understand your user behavior, gain insight from your data. ● You want to ﬁnd patterns in your data. ● These patterns can be used to form hypothesis on how to optimize our objectives. Data Exploration and Visualization

Slide 43

Slide 43 text

Case: NYC Taxi Trip In this competition, Kaggle is challenging you to build a model that predicts the total ride duration of taxi trips in New York City. Your primary dataset is one released by the NYC Taxi and Limousine Commission, which includes pickup time, geo-coordinates, number of passengers, and several other variables.

Slide 44

Slide 44 text

How ● I will follow George Polya’s “How to Solve It” heuristics: Understand your problem. ● Ask questions! ○ What are you asked to ﬁnd or show? ○ Can you think of a diagram that might help you understand the problem? ○ Is there enough information to enable you to ﬁnd a solution? ○ Do you understand all the data used in this problem? ● Do visualizations! Data Exploration and Visualization

Slide 45

Slide 45 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 46

Slide 46 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 47

Slide 47 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 50

Slide 50 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 51

Slide 51 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 52

Slide 52 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 53

Slide 53 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 54

Slide 54 text

Conclusion Data understanding from exploration and visualization make us now the behavior of our users. These understanding can be formed into hypothesis, and we can test those hypothesis to optimize our decision/objectives. Data Exploration and Visualization

Slide 55

Slide 55 text

Hypothesis Making

Slide 56

Slide 56 text

Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 57

Slide 57 text

Hypothesis A hypothesis is a tentative, provisional, or unconfirmed statement derived from theory/ intuitions that can be either verified or falsified.

Slide 58

Slide 58 text

● The questions we ask are more important than the things we measure. ● I will follow Judea Pearl hypothesis making mechanism. ● We will represent hypothesis as “How to Solve It” heuristics mechanism to ﬁnd a good set of hypothesis. Hypothesis

Slide 59

Slide 59 text

Hypothesis Alcohol drinker Lung cancer Smoking Correlation does not imply causation

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Is there any hypothesis you can form based on previous visualizations? Hypothesis

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

Hypothesis Vendor 1 average speed > Vendor 0 average speed Vendor 1 pickup > Vendor 0 Pickup

Slide 64

Slide 64 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Hypothesis

Slide 65

Slide 65 text

Hypothesis Weekdays? Median trip duration weekdays > Median trip duration weekends

Slide 66

Slide 66 text

source: https://www.kaggle.com/headsortails/nyc-taxi-eda-update-the-fast-the-curious Data Exploration and Visualization

Slide 67

Slide 67 text

Hypothesis Direct Distance Trip Distance

Slide 68

Slide 68 text

Is there any hypothesis you can form based on previous visualizations? Hypothesis

Slide 69

Slide 69 text

Business Metrics

Slide 70

Slide 70 text

Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 71

Slide 71 text

Introduction to Business Metrics Measurement is the act of determining a quantitative indication of the extent, amount, dimension, capacity, or size of some attribute of a product or process (Pressman, 2000) Metrics is a quantitative measure of the degree to which a system, component, or process possesses a given attribute. (IEEE, 1990)

Slide 72

Slide 72 text

Introduction to Business Metrics KPI 1 2 Business Metrics

Slide 73

Slide 73 text

Business Metrics: User Metrics

Slide 74

Slide 74 text

Business Metrics: User Metrics Why: - To determine if one methodology produces a faster result than another - Identify hot leads - Improve marketing campaign effectiveness - Determine which marketing campaigns lead to the most proﬁtable customers - Discover which features are getting the most/least use - Reveal technical problems which are hindering your service

Slide 75

Slide 75 text

Business Metrics: User Activity Metrics Metrics Examples: - Churn Rate - DAU/MAU Ratio - Adoption Rate - Lifetime Value - Screen Flow - Completion Rate - Time Based Efﬁciency - Overall Relative Efﬁciency

Slide 76

Slide 76 text

One single most important User Activity Metrics: - Retention Business Metrics: User Activity Metrics % active (monthly) Days from acquisition

Slide 77

Slide 77 text

Business Metrics: User Activity Metrics % active (monthly) Retention Low Retention Days from acquisition

Slide 78

Slide 78 text

Business Metrics: User Activity Metrics When this thing happen: - You probably don’t have a product market ﬁt. - Revise and build a better product. - Your churn rate is a big problem. - You have a loyal customer. - The parallel line of retention with X axis show it converge to a number. - Don’t make yourself biased - 20% of retention, in airlines market, in Indonesia, in daily churn might be big number. People probably only use airplane 2 times a year on average.

Slide 79

Slide 79 text

Business Metrics: User Activity Metrics % active (monthly) Retention High Retention Days from acquisition

Slide 80

Slide 80 text

Business Metrics: User Activity Metrics When this thing happen: - You probably have a product market ﬁt. - Your churn rate is not a big problem. - You have a loyal customer. - The parallel line of retention with X axis show it converge to a number. - Don’t make yourself biased - 95% retention of Nasi Goreng, in NY, 100 people is a small market.

Slide 81

Slide 81 text

Business Metrics: User Activity Metrics % active (monthly) Retention Dead of a product Days from acquisition

Slide 82

Slide 82 text

Business Metrics: User Activity Metrics When this thing happen: - YOU DON’T HAVE A PRODUCT MARKET FIT

Slide 83

Slide 83 text

Business Metrics: User Activity Metrics % active (monthly) Retention Dead of a product Days from acquisition

Slide 84

Slide 84 text

Business Metrics: User Activity Metrics When this thing happen: - Marketing might help you in the short run, but the retention will converge to its natural rate when the marketing campaign gone. - Yes, it can prolong your product life cycle. (and suffering) - Still, the time you buy with marketing campaign need to be translated into a new better product, so the users might stay.

Slide 85

Slide 85 text

Business Metrics Conclusion It is important to track business metrics to know current business states.

Slide 86

Slide 86 text

A/B Testing

Slide 87

Slide 87 text

Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 88

Slide 88 text

Problems ● We want to measure the effect of our product/hypothesis/experiment on some business metrics. ● If the effect is large enough, it can increase our metrics and achieve our business objectives. A/B Testing

Slide 89

Slide 89 text

A/B Testing Population Sample A B Separate these samples into two set

Slide 90

Slide 90 text

A/B Testing Sample A Sample B New Experiment Treatment

Slide 91

Slide 91 text

Optimize Decision Making: A/B Testing Case : Scurvy ● First controlled experiment / randomized trial for medical purposes ● Scurvy is a disease that results from vitamin C deﬁciency ● It killed over 100,000 people in the 16th-18th centuries, mostly sailors ● Lord Anson’s circumnavigation voyage from 1740 to 1744 started with 1,800 sailors and only about 200 returned;most died from scurvy ● Dr. James Lind noticed lack of scurvy in Mediterranean ships. ● Gave some sailors limes (treatment), others ate regular diet (control)

Slide 92

Slide 92 text

A/B Testing Source: https://exp-platfor m.com/Documents /2015-08OnlineCo ntrolledExperiment sKDDKeynoteNR. pdf

Slide 93

Slide 93 text

A/B Testing Source: https://exp-platfor m.com/Documents /2015-08OnlineCo ntrolledExperiment sKDDKeynoteNR. pdf

Slide 94

Slide 94 text

Quasi Experimental Testing Which side of airplanes do we need to strengthen?

Slide 95

Slide 95 text

A/B Testing and Metrics

Slide 96

Slide 96 text

A/B Testing and Metrics Sources: https://web.stanf ord.edu/class/ee 380/Abstracts/1 40129-slides-Ma chine-Learning- and-Econometri cs.pdf

Slide 97

Slide 97 text

A/B Testing Conclusion It is important to measure our hypothesis/product impact on business metrics.

Slide 98

Slide 98 text

Decision Making

Slide 99

Slide 99 text

Business Intelligence Data Exploration Insight Hypothesis Metrics Test Decision

Slide 100

Slide 100 text

Problems Our objective is to choose a range of values to optimize business metrics. How Step 1: Get data. Step 2: Deﬁne a loss function. Step 3: Build a model with a tunable parameter. Step 4: Optimize the loss function given a range of parameter values. Step 5: Choose the best parameter. Decision Making

Slide 101

Slide 101 text

Case: Let’s say I am trying to decide a price at which to list a used phone I want to sell. In this case I may denote my decision space as the entire positive real line such that a∈[0,+∞) . Decision Making

Slide 102

Slide 102 text

Step 1: Get Data Decision Making

Slide 103

Slide 103 text

Step 2: Deﬁne a loss function. how do we ﬁgure out the loss associated with individual decisions when we don’t even know the information we want to use to make a decision? The answer is that we turn to probability theory and instead calculate the “Expected Loss” we would feel if we choose a given action given our beliefs (our probability distribution) about θ Decision Making

Slide 104

Slide 104 text

Step 3: Build model Decision Making Source: http://www.statsathome.com/2017/10/12/bayesian-decision-theory-made-ridiculously-simple/

Slide 105

Slide 105 text

Step 4: Optimize Loss, with Monte Carlo Simulation Source: http://www.statsathome.com/2017/10/12/bayesian-decision-theory-made-ridiculously-simple/ Decision Making

Slide 106

Slide 106 text

Step 4: Get Optimal Parameter Source: http://www.statsathome.com/2017/10/12/bayesian-decision-theory-made-ridiculously-simple/ For a given phone, the optimal price is $71 Decision Making

Slide 107

Slide 107 text

Decision Making Conclusion Bayesian Decision Making is important to optimize our decision to choose the best parameter on a large set of values.

Slide 108

Slide 108 text

Case: Customer Cycle Development

Slide 109

Slide 109 text

Death Spiral: Product Development Cycle https://web.stanford.edu/group/e145/cgi-bin/winter/drupal/upload/handouts/Four_Steps.pdf

Slide 110

Slide 110 text

Customer Development Cycle A company is a sequence of hypothesis testing. https://xweb.stanford.edu/group/e145/cgi -bin/winter/drupal/upload/handouts/Four_ Steps.pdf

Slide 111

Slide 111 text

Customer Development Cycle: Case

Slide 112

Slide 112 text

1. Do FGD/ Customer Survey 2. Do Data exploration 3. Build a prototype 4. Test the prototype to users Customer Development Cycle Product Hypothesis: Do they need a checkins-photo?? Data Exploration Insight Hypothesis Metrics Test Decision

Slide 113

Slide 113 text

1. Do FGD/ Customer Survey 2. Do Data exploration 3. Build a prototype 4. Test the prototype to users Customer Development Cycle Market Hypothesis: Do we need to focus on photo only? Data Exploration Insight Hypothesis Metrics Test Decision

Slide 114

Slide 114 text

1. Do FGD/ Customer Survey 2. Do Data exploration 3. Build a prototype 4. Test the prototype to users Customer Development Cycle Feature Question: What kind of feature do they want to make the users use our product? Data Exploration Insight Hypothesis Metrics Test Decision

Slide 115

Slide 115 text

1. Do FGD/ Customer Survey 2. Do Data exploration 3. Build a prototype 4. Test the prototype to users Customer Development Cycle Marketing Question: Do we need to rebrand our app? Data Exploration Insight Hypothesis Metrics Test Decision

Slide 116

Slide 116 text

EOP Contact: business@pacmannai.com

Slide 117

Slide 117 text

We believe everyone can be Data Scientist Contact: business@pacmannai.com

Slide 118

Slide 118 text

Upcoming ML & BI Class Contact: business@pacmannai.com

Slide 119

Slide 119 text

No content

Slide 120

Slide 120 text

Check our website: www.pacmann.ai Contact: business@pacmannai.com

Slide 121

Slide 121 text

Check our website: www.pacmann.ai Contact: business@pacmannai.com

Slide 122

Slide 122 text

Check our website: www.pacmann.ai Contact: business@pacmannai.com

Slide 123

Slide 123 text

Past Classes Contact: business@pacmannai.com

Slide 124

Slide 124 text

Pacmann AI Classes Quality State of the Art of Machine Learning Research Practical Skills Theoretical Understanding > 50 institutions 400++ alumni 6 Classes in the past business@pacmannai.com https://pacmann.ai

Slide 125

Slide 125 text

60 participants 1 week 28 institutions business@pacmannai.com https://pacmann.ai Previous Classes

Slide 126

Slide 126 text

Previous Classes business@pacmannai.com https://pacmann.ai 96 participants 2 weeks 51 institutions

Slide 127

Slide 127 text

Previous Classes business@pacmannai.com https://pacmann.ai 61 participants 8 weeks 48 institutions

Slide 128

Slide 128 text

Previous Classes business@pacmannai.com https://pacmann.ai 59 participants 8 weeks 48 institutions

Slide 129

Slide 129 text

Previous Classes business@pacmannai.com https://pacmann.ai 44 participants 2 weeks 33 institutions

Slide 130

Slide 130 text

Previous Classes Facts & Figure business@pacmannai.com https://pacmann.ai Level of Education

Slide 131

Slide 131 text

Previous Classes Facts & Figure business@pacmannai.com https://pacmann.ai Field of Work

Slide 132

Slide 132 text

Previous Participants business@pacmannai.com https://pacmann.ai

Slide 133

Slide 133 text

Contact Email: business@pacmannai.com Whatsapp Business: +62 812-8122-1707