Modern Data Literacy

Jonathan Wallace @jonathanwallace
Modern Data Literacy
SoundBoard
Cross-Functional Online Marketing Conference
Jonathan Wallace
good afternoon everyone, thanks for coming out. My name is Jonathan Wallace and i'm glad you're here to listen to my talk. It has been ﬁve years since I
last presented at this conference and I want to thank the organizers for having me out again.
today, I’ve come to talk about what I think is an important topic of which everyone should be aware, modern data literacy.

View Slide

http://www.contentrow.com/tools/link-bait-title-generator
• Statistical Reasoning
• Algorithmic Thinking
• Curiosity
I want you to…
Goals
Real quick, these are my goals for the talk. Really, I want your interest piqued in these topics because with only twenty minutes, we’re only going to
superﬁcially cover these topics with a few examples.

View Slide

http://www.bignerdranch.com/about-us/nerds/jonathan-wallace.html
2014!
But ﬁrst, let me share why I’m worth listening to about this topic. Last time I was here, I shared that I’d been a web developer for the past seven years, ﬁve
of which was at a consulting company.
Believe it or not, it took a lot of takes to get that shot!

View Slide

https://www.linkedin.com/in/jonathan-wallace-8888ba9/
• Director of Medical Billing Company
• Principal Engineer at Stitch Fix
• Representative of Ga. House District 119
• V.P. of Engineering at Softgiving
I’ve had a few more jobs associated with data
Since then..
Anyway, since then, I’ve done all the things listed here. I was the director of a medical billing company where we analyzed claims for medical labs and
helped them get paid by insurance companies. We had over 30 labs so there was a decent, but small, ﬂow of data with which to contend.

View Slide

• Director of Medical Billing Company
• Principal Engineer at Stitch Fix
• Representative of Ga. House District 119
• V.P. of Engineering at Softgiving
I’ve had a few more jobs associated with data
Since then..
After that, I was at Stitch Fix, where our primary database was over 1/2 a terabyte in size. Not huge when compared to FB, Apple, Amazon, Netﬂix, or
Google but still pretty big!
Let’s talk about that for a quick moment.

View Slide

https://web-assets.domo.com/blog/wp-content/uploads/2017/07/17_domo_data-never-sleeps-5-01.png
Have you heard?

View Slide

• Snapchat users share 527,760 photos
• Users watch 4,146,600 YouTube videos
• 456,000 tweets are sent on Twitter
From 2017!
Have you heard?
These stats are for every minute.
What do you think those companies are doing with that data? What kinds of questions are they trying to answer? How much data do you think you
generate?

View Slide

https://www.ﬂickr.com/photos/hagdorned/15021067991/in/photolist-oTmUbT-8y6DpS-4AMJ1k-89KRMv-4Xo8u9-96xajV-aPW1ut-77EZpJ-
dE8g9W-67JHVz-7UgzgP-mLNdH-aiiT82-5HZuzs-8zrt3g-oZn3NB-5rUT4-8CTLZ3-4jF5Zz-yGFki-9JSGSe-5a4Cz9-8PwNgL-8PwR8q-
Technology
While you’re pondering those questions, let’s talk about technology. Technology is what allows us to generate those mind boggling amounts of data. It is a
force multiplier.
It isn’t inherently good or bad. What does this mean when it comes to data? If technology is neutral, does that mean the data is neutral?

View Slide

Technology
No, data is not neutral. It isn’t neutral because the data you acquire that depends on what questions you ask. How do you capture it? How do you store it?
The temptation is to grab all the data you can.
And then later, you’ll try to ﬁgure out what it means.

View Slide

Technology
I’m here to make the case that you be better off by knowing the question that you want to ask before you get started. What do you want to know?

View Slide

• Formulate your question
• Collect data
• Analyze data
• Interpret Results
Statistical Problem Solving
Here’re the basics. First, we need to formulate your question. This might seem elementary and obvious but I assure you it is not. People often skip this.

View Slide

Improve the conversions
on our website
A Simple Example
You have a website that contains multiple steps? Or maybe, even simpler, multiple forms. What might your question be?
“At which step in our signup process do we see the largest drop off?”

View Slide

Improve the conversions
on our website
A Simple Example
Great, now we know what to measure. Let’s measure the amount of views for each page.
Knowing your question is the ﬁrst step. But this example, although it highlights formulating a question, doesn’t address some of the problems that
happens with data at large scales.

View Slide

• Collect data
• Analyze data
Now we’re talking about scale and we’re talking about collecting data. I’m not going in to great detail as with a strong technology team, you shouldn’t have
to know the details, they, or an application that commoditizes that work, should handle that for you. But we’ll talk about a little about scale and
magnitudes.

View Slide

https://www.ﬂickr.com/photos/jimchoate/26697097528/in/photolist-GF8AYY-s7nTBP-cAVnUS-cADPuJ-26yQv1K-5D4Jr9-b9CHqz-A92fW5-cSuRBw-oEfSEB-
bwsn5n-5Qyg2k-5vZa8J-qc39Py-8TASxQ-bWwD61-aEzbHF-gnmsmN-fQ6JtK-VEvfPn-21D4bZz-y4Uqo-dPPK8p-bCc3Rf-bcH8tc-oW3EPe-2EAG9-2entne6-afa2Yg-
Scale
How long is one million seconds? 11.6 days
How long is one billion seconds? 32 years
How can we manage one million of anything let alone one billion? How do we cogently handle that many data points?

View Slide

https://www.ﬂickr.com/photos/jimchoate/26697097528/in/photolist-GF8AYY-s7nTBP-cAVnUS-cADPuJ-26yQv1K-5D4Jr9-b9CHqz-A92fW5-cSuRBw-oEfSEB-
bwsn5n-5Qyg2k-5vZa8J-qc39Py-8TASxQ-bWwD61-aEzbHF-gnmsmN-fQ6JtK-VEvfPn-21D4bZz-y4Uqo-dPPK8p-bCc3Rf-bcH8tc-oW3EPe-2EAG9-2entne6-afa2Yg-
Scale
We have to group the data. We have to aggregate similar data points into groups.
But what problems arise when we start grouping data?

View Slide

• Collect data
• Analyze Results
Problems that arise when grouping data, speaks to analysis. Let’s look at a quick one.

View Slide

You have a classroom of ten students and
the average grade is 93.
Aggregation Bias
https://www.ﬂickr.com/photos/nwabr/5917202414/in/photolist-a1Td8h-nwEGdX-ifmce-63ZU4g-byXhtx-dtD6bE-8AdtKD-amnveH-rQdqSb-dSWa9F-bmzTZC-s5vEDE-MFiCpg-raNGeA-
raNG57-5v3gqp-5v7B7G-9xXzVb-JX3TqX-99utdQ-9NFSQV-8cbDh3-aPmoGr-9GNHEk-2f1GhVi-9GRAny-aZpEUB-7gry7L-aZpER6-9GNHFX-6SddEn-9GRApj-oZfNK7-oZgvwe-peHNN1-NrPyP-
obC4dS-phbAkK-eiafBS-JbrPdz-cEJnWs-SxnDGj-Hwq38u-SxotnQ-GXNzJ-g3FZLC-GXPW4-g3FVHB-2co3WSt-7Uh3fu
Here’s one. Aggregation Bias.
What can we say about how any particular student is doing? We can’t. We would need more data about our grouping.

View Slide

Scores are nine 100s and one 30.
Aggregation Bias
That’s easy enough. Remember we’re working with a small data set so it is easy to reason and think about but consider if we’re talking about a ten
thousand or a billion numbers.

View Slide

Mean: 93 
Median: 100
Mode: 100
“Average”
I initially used the word “average” and that is misleading because we don’t know the distribution of the numbers. Now here are three more common
mathematical concepts that help us understand the data set without knowing the details.

View Slide

Standard deviation: 22.136
Variance: 490
“Average”
We can look at other numbers to help us understand the data we’ve collected. And we should. And it is important to understand how the data is
distributed. Is it randomly distributed? Or does it have a fun distribution shape?
Alright, I have one more thing to cover before I get to a real world example

View Slide

https://bl.ocks.org/bryik/a3d0d7a0d9d69e6afe0fd8b8b3becec1
Algorithmic Thinking
Does anyone know what this is? This is a graph. More speciﬁcally, this called a complete graph because every vertex, that is, the colored circles, is
connected to every other vertex via an edge.
What do the vertices represent? Well, they can represent anything.

View Slide

https://www.ﬂickr.com/photos/alanchan/2269845817/in/photolist-4szy5H-rdfAX8-jLXQwg-8BTgwh-o47WXh-2kNpGM-pP5TAt-aZewz8-a4eSs9-
pBrH2C-dqY18g-pBeme8-6NEaLF-5AUDUZ-dEzMgS-dANMPT-8tgm2H-7MuvmD-pjZcz4-pQWvjM-7nafsM-ni3B3R-6ZCC48-pjYdcb-KvSsM-
Let’s think them as people. There’s something called the handshake problem. How many handshakes have to occur in this room for everyone to shake
everyone else’s hand?

View Slide

The answer can be modeled by this graph. We can manually count up the number of edges but there’s an easier way. (It is 15!)

View Slide

N * (N - 1) / 2
Here’s the formula for calculating the number of edges or connections between those vertices. 6*5 / 2 = 15

View Slide

But what happens when the room is a lot larger. You can see that the number connections or edges, grows quickly. In this case, there are 351 edges.

View Slide

Now let me show you my application from Softgiving. This is one folder in one directory in the application. Here’s where we’ve modeled the domain. We
have 59 models. That would be 1711 connections if each model were fully connected to every other model. My boss the other day asked about adding
multiple accounts per organization. We would now have another dimension, or another node, in the application. This would lead to 1770 connections. You
can see how each new variable, new dimension, new grouping that we add, may add complexity to how we capture, store, and associate data.

View Slide

An for fun, here’s a complete graph with 44 notes, 946 edges. You can visually see how the complexity grows. Getting pretty dark, isn’t it?

View Slide

https://www.ﬂickr.com/photos/corporate-traveller/6826708843/in/photolist-56Gpiy-bpfEGB-9bST8G-7AN25d-8Lda6k-5u3EmK-5u3M6i-
DhTte-284dUNK-22uxc6R-5gBQyo-LDzc-n2rii-3dw21B-sjm42-5SfzgF-EXZkiL-2f2dXtv-qfkVVp-hZtHME-21PWcmu-sbbUQA-6vhUte-rAM4Pr-
So back to the handshake problem, and I promise it has real-world relevance in the next section. Everyone shaking everyone’s hand is a large number as
the number of people goes up.

View Slide

N * (N - 1) / 2
Here’s that formula again. Notice how I’ve made the N’s bigger. The minus one and the divide by two are not what dictates the size of the result. The value
represent by N is what dictates. In programming terms, we think about a function this and we describe the growth rate for a function in terms of Big Oh
notation.

View Slide

• O(1)
• O(n)
• O(n*n)
So you can think about it. A big oh of one, is a constant. So if I want to say high to everyone in this room, I wave my hand and say hello. 
 
If I want to shake each of your hands individual, there are 20 people, then n is twenty and I shake twenty hands.
But, if I want to build a strong community in this room, then I would like for everyone to shake everyone’s hand.

View Slide

• O(1)
• O(n)
• O(n*n)
Spoiler, this is why politicians are gluttons for events and interviews. They can meet a lot of people at once. And with a speech and a hand wave, they’re
very efficient. But they also take the time to shake individual hands as much as they can. Finally, the very good ones, focus on community building which
gets people to shake each other’s hand.
Okay, on to the real world example.

View Slide

https://www.redandblack.com/athensnews/breaking-democratic-candidate-jonathan-wallace-wins-district-state-house-seat/article_dd1849c2-
c432-11e7-9f0e-238235d06a0a.html
Real World
This was me on Nov. 7th, 2017 in a special election for Georgia House District 119.

View Slide

https://www.redandblack.com/athensnews/breaking-jonathan-wallace-loses-seat-in-state-house-district-race/article_d813c034-e22f-11e8-
b0f8-1f98f862af60.html
Real World
This was not me three hundred and ﬁfty ﬁve days later on Nov. 6th, 2018. Let’s talk about how being ignorant of a distribution helped lead to this result.

View Slide

• Collect data
• Analyze data
The question in a campaign is straight forward. How many votes do I need to win? Every campaign does this. You look at history and examine like races,
make some explicit assumptions, and then formulate a strategy.

View Slide

Real World
We used a tool called Votebuilder. Votebuilder provides arbitrary metrics called scores. These scores predict the support for one particular party over
another. When we looked at the score we used for formulating our strategy, we knew that the distribution on the scale of 0-100 was not linear.

View Slide

Real World
https://en.wikipedia.org/wiki/Multimodal_distribution
We knew the distribution was bimodal. On one side, you had a cluster of Democrats / progressives. On the other, you had a cluster of Republicans /
conservative. So we established our win number and the size of our universe, 12000, based on that score.

View Slide

• Collect data
• Analyze data
For collecting data, we used a different tool.

View Slide

Real World
We used a tool called minivan. And also pen and paper. When someone knocked on a constituent’s door and shook their hand, they would ask questions
like, will you vote for Jonathan Wallace. This data was then analyzed on a regular basis to adjust our strategy.

View Slide

• Collect data
• Analyze Results
We analyzed our results and thought we were doing well. I ended with 11929 votes which was more than I thought I needed. So what went wrong?

View Slide

https://www.redandblack.com/athensnews/breaking-jonathan-wallace-loses-seat-in-state-house-district-race/article_d813c034-e22f-11e8-
b0f8-1f98f862af60.html
Real World
When we dug into the question about how we were going to collect data, we made a mistake related to distributions. I.e., how was the data grouped.

View Slide

Real World
Remember, votebuilder provides arbitrary metrics called scores. These scores predict the support for one particular party over another.

View Slide

Real World
We knew it was bimodal. On one side, you had a cluster of Democrats / progressives. On the other, you had a cluster of Republicans / conservative.

View Slide

Real World
We assumed that if we went down to a score of 30 towards the conservative end, that we would have a total universe of 12,000 votes. Plenty to win the
election. We thought the variance was smaller than it really was.

View Slide

Real World
In reality, what we found in our analysis, is that the clusters of this bimodal distribution is extremely concentrated on either end. And we should have used
a score of 15 which would have made our universe even larger and given a better chance of victory in that race. The variance was much larger.

View Slide

http://www.contentrow.com/tools/link-bait-title-generator
• Statistical Reasoning
• Algorithmic Thinking
• Curiosity
I want you to…
Goals
Real quick, these are my goals for the talk.

View Slide

• https://www.govtech.com/data/How-to-Do-Data-Analytics-in-Government.html
• https://oli.cmu.edu/jcourse/lms/students/syllabus.do?
section=8ff067630a0001dc5478c6621fc2dd80
• https://www.coursera.org/learn/algorithmic-thinking-1
• https://towardsdatascience.com/how-to-properly-tell-a-story-with-data-and-common-pitfalls-to-
avoid-317d8817e0c9
• https://apps3.cehd.umn.edu/artist/glossary.html
• https://towardsdatascience.com/how-to-properly-tell-a-story-with-data-and-common-pitfalls-to-
avoid-317d8817e0c9
• https://bl.ocks.org/bryik/a3d0d7a0d9d69e6afe0fd8b8b3becec1
• https://www.datacenterknowledge.com/archives/2013/01/18/facebook-builds-new-data-centers-
for-cold-storage
• https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-
the-mind-blowing-stats-everyone-should-read/#a565d7f60ba9
• https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/
References and Resources

View Slide

Modern Data Literacy
SoundBoard
Cross-Functional Online Marketing Conference
Jonathan Wallace

View Slide

Modern Data Literacy

Modern Data Literacy

Jonathan Wallace

More Decks by Jonathan Wallace

Other Decks in Business

Featured

Transcript