#26 La data visualisation pour la data science (et les data scientists)

Deﬁnitions [De]coding Howto ? Rules Perception Before/After Cases So what
? Références Data Visualization for Data Scientists Christophe Bontemps Toulouse School of Economics, INRA @Xtophe_Bontemps

? Références MERCI À dream-team DE TDS

? Références THE “VISUAL PERCEPTION” OF A GRAPHIC What do you see ? 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 )

? Références THE “VISUAL PERCEPTION” OF A GRAPHIC And here, what do you see ? 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some other points (N = 500 )

? Références “VISUAL PERCEPTION” AS A STATISTICAL TEST 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some other points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 )

? Références “VISUAL PERCEPTION” AS A STATISTICAL TEST 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some other points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) “ The human eye acts is a broad feature detector and general statistical test”. Buja et al. (2009)

? Références “VISUAL PERCEPTION” AS A STATISTICAL TEST 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some other points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) “ The human eye acts is a broad feature detector and general statistical test”. Buja et al. (2009) Test : H0 : {There is "nothing" } = {No relation}

? Références “VISUAL PERCEPTION” AS A STATISTICAL TEST 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some other points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) 0 10 20 30 40 0 10 20 30 40 Some X variable Some Y variable Some points (N = 500 ) “ The human eye acts is a broad feature detector and general statistical test”. Buja et al. (2009) Test : H0 : {There is "nothing" } = {No relation} H1 : { There is "something" } = {There is some relation (Correlation, linearity, heterogeneity, groups..) }

? Références [- WHAT IS DATA VISUALISATION ? -] For Tukey (1977) “The greatest value of a picture is when it forces us to notice what we never expected to see”

? Références [- WHAT IS DATA VISUALISATION ? -] For Tukey (1977) “The greatest value of a picture is when it forces us to notice what we never expected to see” Cleveland (1994) says that “graphical methods and techniques are powerful tools for showing the structure of data”

? Références [- WHAT IS DATA VISUALISATION ? -] For Tukey (1977) “The greatest value of a picture is when it forces us to notice what we never expected to see” Cleveland (1994) says that “graphical methods and techniques are powerful tools for showing the structure of data” Bertin (1970) (translated in Bertin (1983)) deﬁnes it as a "visual language" and, as such, with a semiology, i.e. with a theory of the functions of signs and symbols.

? Références [- WHAT IS DATA VISUALISATION ? -] For Tukey (1977) “The greatest value of a picture is when it forces us to notice what we never expected to see” Cleveland (1994) says that “graphical methods and techniques are powerful tools for showing the structure of data” Bertin (1970) (translated in Bertin (1983)) deﬁnes it as a "visual language" and, as such, with a semiology, i.e. with a theory of the functions of signs and symbols. Tufte (2001) “Graphics are instruments for reasoning about quantitative information”

? Références GOALS OF DATA VISUALISATION Data visualisation serves at least two main purposes Data exploration Graphs as visual tests, comparisons → short time to built and to read

? Références GOALS OF DATA VISUALISATION Data visualisation serves at least two main purposes Data exploration Graphs as visual tests, comparisons → short time to built and to read Data representation Summaries, comparisons, storytelling → long time to build, short time to read

? Références GOALS OF DATA VISUALISATION Data visualisation serves at least two main purposes Data exploration Graphs as visual tests, comparisons → short time to built and to read Data representation Summaries, comparisons, storytelling → long time to build, short time to read The problem is that : “ Communicating implies simpliﬁcation data exploration implies exhaustivity”

? Références THE BIG PICTURE : FROM "DATA TO VIZ"

? Références THE BIG PICTURE : FROM "VIZ TO DATA"

? Références [- DECODING VISUAL INFORMATION -]

? Références GRAPHICS IMMEDIATE TO UNDERSTAND FIGURE – Where do people run in Paris source : (N. Yau) http://flowingdata.com/2014/02/05/where-people-run/

? Références GRAPHICS IMMEDIATE TO UNDERSTAND FIGURE – Climate forecast uncertainty (S. Planton)

? Références GRAPHICS THAT NEED EXPLANATIONS : FIGURE – How people spend their days (NYT).

? Références [- MISTAKES ! -] Source : New Yorker

? Références USUAL MISTAKES FIGURE – (source Wikipedia)

? Références USUAL MISTAKES : TRUNCATED GRAPH (ZERO AXIS) FIGURE – (source Wikipedia)

? Références USUAL MISTAKES FIGURE – Are you looking at the right thing ?

? Références USUAL MISTAKES : LIMITED SCOPE FIGURE – Are you looking at the right thing ? from Flowing data

? Références USUAL MISTAKES FIGURE – Percentage of student starting and completing a “step” in MOOC Source : My 2017’ students (Jan & Mohamed), but also recent researchers’ presentations

? Références USUAL MISTAKES : PIE CHARTS

? Références USUAL MISTAKES : PIE CHARTS Source https://twitter.com/freakonometrics/status/ 612742330160951296

? Références USUAL MISTAKES : STACKED BARS

? Références USUAL MISTAKES : LEGITIMATE COMPARISON IMPOSSIBLE ! FIGURE – from Dix and Ellis (1998) example

? Références VERY USUAL “Mistake”... FIGURE – Major Cause of Worker Disability (1975-2010) (J. Schwabish, 2014).

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ?

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ? In 2010, what is the major cause of disability ?

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ? In 2010, what is the major cause of disability ? Is cancer (red curve) increasing over the period ?

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ? In 2010, what is the major cause of disability ? Is cancer (red curve) increasing over the period ? In the recent years, which causes have increased (decreased) the most ?

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ? In 2010, what is the major cause of disability ? Is cancer (red curve) increasing over the period ? In the recent years, which causes have increased (decreased) the most ? · · ·

? Références LEGITIMATE QUESTIONS : What is the highest value observed ? When was it ? In 2010, what is the major cause of disability ? Is cancer (red curve) increasing over the period ? In the recent years, which causes have increased (decreased) the most ? · · · You do not remember a damn thing of this graph !

? Références VERY USUAL “Mistake” : SMALL MULTIPLES FIGURE – Major Cause of Disability - 1975-2010 (J. Schwabish, 2014).

? Références VERY USUAL “Mistake” II...

? Références VERY USUAL “Mistake” II... What do you see here ?

? Références VERY USUAL “Mistake” II... What do you see here ? Easier to see the trend of each curve...

? Références VERY USUAL “Mistake” II... What do you see here ? Easier to see the trend of each curve... Idea shared by Gelman (2004) and Munzner (2014)

? Références USUAL “Mistakes” : RADAR PLOTS How to represent many characteristics on a single graph ? http://data.visualisation.free.fr/Blog/ Why-you-should-never-use-radar-plots.nb.html

? Références USUAL “Mistakes” : BEWARE OF RADAR PLOTS Representation (areas) change with axis order ! http://data.visualisation.free.fr/Blog/ Why-you-should-never-use-radar-plots.nb.html

? Références [- WHY CODING ? -] If decoding is hard, why coding ?

? Références GRAPHICS reveal DATA : ANSCOMBE (1973) QUARTET We use here 4 couples of random variables : (X1, Y1), (X2, Y2) (X3, Y3) and (X4, Y4). All four data sets have the same descriptive statistics. Xs Mean Std. Dev. Ys Mean Std. Dev. corr(Xi, Yi) N X1 9 3.32 Y1 7.5 2.03 0.8164 11 X2 9 3.32 Y2 7.5 2.03 0.8162 11 X3 9 3.32 Y3 7.5 2.03 0.8163 11 X4 9 3.32 Y4 7.5 2.03 0.8165 11

? Références ANSCOMBE (1973) QUARTET All four data sets are described by the same linear model (Yi = α + βXi + i), revealing apparently the same relationships : Dependent variable : Y1 Y2 Y3 Y4 Regressed on : Xi , i=1,...,4 0.500 ∗∗∗ 0.500∗∗∗ 0.500∗∗∗ 0.500∗∗∗ Constant 3.000∗∗ 3.001∗∗ 3.002∗∗ 3.002∗∗ R2 0.667 0.666 0.666 0.667 Resid Std. Error 1.237 1.237 1.236 1.236 F Statistic 17.990∗∗∗ 17.966∗∗∗ 17.972∗∗∗ 18.003∗∗∗ Note : Data from Anscombe (1973). ∗ p <0.1 ; ∗∗ p < 0.05 ; ∗∗∗ p < 0.01

? Références ANSCOMBE (1973) QUARTET A simple scatter plot (regression overlaid) shows something very different. 4 8 12 5 10 15 x1 y1 Regression of Y1 on X1 (with constant) 4 8 12 5 10 15 x2 y2 Regression of Y2 on X2 (with constant) 4 8 12 5 10 15 x3 y3 Regression of Y3 on X3 (with constant) 4 8 12 5 10 15 x4 y4 Regression of Y4 on X4 (with constant)

? Références ANSCOMBE (1973) QUARTET NP : Plots of the residuals shows also same differences −2 −1 0 1 2 5 6 7 8 9 10 Fitted values Residuals Residual vs Fitted Plot −2 −1 0 1 5 6 7 8 9 10 Fitted values Residuals Residual vs Fitted Plot −1 0 1 2 3 5 6 7 8 9 10 Fitted values Residuals Residual vs Fitted Plot −1 0 1 2 7 8 9 10 11 12 Fitted values Residuals Residual vs Fitted Plot

? Références [- RULES OF THUMB -] “There are no “good” nor “bad” graphics (...), there are graphics answering legitimate questions and graphics that do not answer question at all ” Bertin (1970) It is easy to criticize ... but are there some rules ?

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics Use comparable elements → data visualisation

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics Use comparable elements → data visualisation And many people have already worked on that !

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics Use comparable elements → data visualisation And many people have already worked on that ! Jacques Bertin

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics Use comparable elements → data visualisation And many people have already worked on that ! Jacques Bertin Edward Tufte

? Références COMPARISON IS DIFFICULT Yes ! there are some basic rules : Compare what is comparable → statistics Use comparable elements → data visualisation And many people have already worked on that ! Jacques Bertin Edward Tufte Tuckey, Cleveland, .....

? Références FUNDAMENTAL QUESTIONS (CHEN ET AL. (2007)) What to Whom, How and Why ? A graphic may be linked to three pieces of text : its caption, a headline and an article it accompanies. → should be consistent and complement each other.

? Références FUNDAMENTAL QUESTIONS (CHEN ET AL. (2007)) What to Whom, How and Why ? A graphic may be linked to three pieces of text : its caption, a headline and an article it accompanies. → should be consistent and complement each other. Show or explore data ? Different purpose, different comparisons, different requirements !

? Références FUNDAMENTAL QUESTIONS (CHEN ET AL. (2007)) What to Whom, How and Why ? A graphic may be linked to three pieces of text : its caption, a headline and an article it accompanies. → should be consistent and complement each other. Show or explore data ? Different purpose, different comparisons, different requirements ! Choice of Graphical form ? Choice depends on the type of data to be displayed (e.g. univariate continuous data, bivariate categorical data, etc..) and on what is to be shown, what is tested (compared).

? Références FUNDAMENTAL QUESTIONS (CHEN ET AL. (2007)) What to Whom, How and Why ? A graphic may be linked to three pieces of text : its caption, a headline and an article it accompanies. → should be consistent and complement each other. Show or explore data ? Different purpose, different comparisons, different requirements ! Choice of Graphical form ? Choice depends on the type of data to be displayed (e.g. univariate continuous data, bivariate categorical data, etc..) and on what is to be shown, what is tested (compared). Unique solution ? There is not always a unique optimal choice → alternatives can be equally good or good in different ways, emphasizing different aspects of the same data.

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data Question : Induce the viewer to think about the substance rather than about methodology, graphic design. Encourage the eye to compare different piece of data.

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data Question : Induce the viewer to think about the substance rather than about methodology, graphic design. Encourage the eye to compare different piece of data. Data-ink ratio : Maximize the ink-data ratio. Erase all non data ink, Erase redundant information

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data Question : Induce the viewer to think about the substance rather than about methodology, graphic design. Encourage the eye to compare different piece of data. Data-ink ratio : Maximize the ink-data ratio. Erase all non data ink, Erase redundant information Integrity : Avoid distorting what the data have to say

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data Question : Induce the viewer to think about the substance rather than about methodology, graphic design. Encourage the eye to compare different piece of data. Data-ink ratio : Maximize the ink-data ratio. Erase all non data ink, Erase redundant information Integrity : Avoid distorting what the data have to say General to speciﬁc : Reveal the data at different levels of detail (from broad picture to ﬁne structure)

? Références [RULES :] EDWARD R. TUFTE’S In his seminal book, Tufte (2001) propose some principles for displaying quantitative information. Data : Above all, show the data Question : Induce the viewer to think about the substance rather than about methodology, graphic design. Encourage the eye to compare different piece of data. Data-ink ratio : Maximize the ink-data ratio. Erase all non data ink, Erase redundant information Integrity : Avoid distorting what the data have to say General to speciﬁc : Reveal the data at different levels of detail (from broad picture to ﬁne structure) Context : Graphical display should be closely integrated with the statistical and verbal descriptions of the data set.

? Références PRACTICAL EXAMPLE : DATA-INK RATIO Let’s start with a classical graph (R default - Boxplot ) g1 g2 g3 g4 g5 98 100 102 104 106 108 110 112 Groupe Response FIGURE – Distribution of a continuous variable on 4 groups

? Références ERASE ALL NON DATA INK Groupe Response 1 2 3 4 5 98 100 102 104 106 108 110 112 FIGURE – Distribution of a continuous variable on 4 groups

? Références ERASE ALL REDUNDANT ! Groupe Response 1 2 3 4 5 98 100 102 104 106 108 110 112 FIGURE – Distribution of a continuous variable on 4 groups

? Références GOING FURTHER... Groupe Response 1 2 3 4 5 98 100 102 104 106 108 110 112 FIGURE – Distribution of a continuous variable on 4 groups

? Références AND SHOW THE DATA... Groupe Response 101.0 100.0 101.0 103.8 109.1 1 2 3 4 5 FIGURE – Distribution of a continuous variable on 4 groups

? Références HAVE WE LOST SOMETHING ? g1 g2 g3 g4 g5 98 100 102 104 106 108 110 112 Groupe Response Groupe Response 101.0 100.0 101.0 103.8 109.1 1 2 3 4 5 FIGURE – Distribution of a continuous variable on 4 groups Did you noticed that group 1 and group 3 had the same median (101.0) ? see the ggplot theme + theme_tufte()

? Références PRACTICAL EXAMPLE : INTEGRITY (THE LIE FACTOR) LieFactor = Size of effect shown in graphic Size of effect in data (1) A Lie Factor = 1 indicates a substantial distortion FIGURE – Fuel economy standards. (E. Tufte - from NY Times 1978)

? Références FIGURE – Fuel economy standards (revisited) The "18 mpg" line measures 1.5 cm (in 1978) ; the "27,5 mpg" measures 13 cm (in 1985) −→ Lie factor = 14.5% ! ! !

? Références PRACTICAL EXAMPLE : CONTEXT In the next 2 minutes, you will design “the best way” to compare two numbers 75and 37. From S. Ortiz

? Références CONTEXT MATTERS ! Solutions proposed by S. Ortiz

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ?

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ? Context ?

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ? Context ? Audience ?

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ? Context ? Audience ? What to compare ?

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ? Context ? Audience ? What to compare ? Units ?

? Références CONTEXT MATTERS (FOLLOW-UP) What are the problems you’ve encountered ? Context ? Audience ? What to compare ? Units ? Multiple solutions ?

? Références [RULES :] BERTIN’S APPROACH (A VISUAL LANGUAGE) If graphs are used to communicate, it is a form of language. Any language has a grammar, “words” and logic. Let us study the science that deals with signs or sign language : “The Semiology”. TABLE – Bertin’s deﬁnition of 8 visual variables Position (x, y) Size Value Texture Colour Orientation Shape

? Références SHAPE IS NOT SUITABLE FOR PROPORTIONALITY Price of land in the East of France Bertin (1970)

? Références SIZE IS SUITABLE FOR PROPORTIONALITY Price of land in the East of France Bertin (1970)

? Références WHAT ELSE ? Is color suitable for proportionality ?

? Références A NOTE ON COLORS “Colors” are not suited for ordering nor for proportionality ! Try putting the following hues in order from low to high.

? Références A NOTE ON COLORS These colors are easy to order from low to high. Few (2008) provides meaningful solutions for choosing palettes of colors, for example for heatmaps. See also the ggplot theme theme_few()

? Références [RULES :] CLEVELAND-MCGILL

? Références THE CLEVELAND-MCGILL EFFECT

? Références THE CLEVELAND-MCGILL EFFECT From Cleveland and McGill (1984)

? Références WEBER’S LAW AND FRAMED BOXES

? Références WEBER’S LAW AND FRAMED BOXES From Cleveland and McGill (1984)

? Références [RULES :] ACCESSIBILITY Normal → Color-blind →

? Références [- PERCEPTION -] // Source : http://flowingdata.com/2014/06/25/duck-vs-rabbit-plot/

? Références “PREATTENTIVE” VARIABLES How many "3" in that sequence ? (from Ware (2012))

? Références AND NOW... Find the red dot !

? Références TEST : FIND THE RED DOT !

? Références HARDER : IS THERE A "STRANGER" ?

? Références THAT WASN’T EASY Preattentive concept, Treisman (1985) Some visual elements or patterns are detected immediately But there may be interferences (colour and form) Very useful (detection, explanatory and presentation) Helpful to highlight a message !

? Références TOO MUCH VARIATION DOESN’T HELP From Ware (2012)

? Références MOST PREATTENTIVE VISUAL VARIABLES From Ware (2012)

? Références [- BEFORE/AFTER -] Source : Emma Rathbone - The New Yorker

? Références BEFORE (SCHWABISH - JEP, 2014) FIGURE – An Unbalanced Chart - Original

? Références AFTER (SCHWABISH - JEP, 2014) FIGURE – An Unbalanced Chart - Revised

? Références BEFORE (SCHWABISH - JEP, 2014) FIGURE – A Clutterplot Example - Original

? Références AFTER (SCHWABISH - JEP, 2014) FIGURE – A Clutterplot Example - Revised

? Références BEFORE - NETWORK EXAMPLE Victor Hugo’s characters network (Les Miserables).

? Références ALTERNATIVE : ADJACENT MATRIX PLOT An adjacency matrix, where each cell ij represents an edge from vertex i to vertex j.

? Références ALTERNATIVE : SORTED ADJACENT MATRIX PLOT

? Références ALTERNATIVE : ARC DIAGRAM PLOT FromGaston Sanchez

? Références ALTERNATIVE : VERTICAL ARC DIAGRAM PLOT

? Références BEFORE - STUDENTS GRADES IN 4 COURSES Source Munzner (2014)

? Références AFTER - STUDENTS GRADES IN 4 COURSES Source Munzner (2014)

? Références BEFORE : US AIRLINES MAP (C. HURTER) Source Christophe Hurter

? Références AFTER : BUNDLING THE NETWORK (C. HURTER) To simplify the understanding, researcher propose bundling techniques. 1 Source Christophe Hurter 1. To bundle : to tie or gather things together

? Références AFTER : BUNDLING IN NETWORKS (C. HURTER) Edges get closer and density gets sharper : Source Christophe Hurter

? Références BUNDLING IN NETWORKS (C. HURTER) Edges get even closer and density gets even sharper : Source Christophe Hurter

? Références BUNDLING IN NETWORKS (C. HURTER) At the end of the day...one can see through darkness ! Source Christophe Hurter

? Références CASE STUDIES

? Références CASE 1 : FUNNEL DATA ANALYSIS Original data are percent of users in funnel data stages, or gates : Each stage represents some portion of the previous stage. Source : www.storytellingwithdata.com

? Références CASE 1 : FUNNEL DATA ANALYSIS Original data observed for 4 regions of the world

? Références CASE 1 : FUNNEL DATA ANALYSIS ... and for 3 cohorts (Q1, Q2 & Q3)

? Références CASE 1 : FUNNEL DATA ANALYSIS Identify the interesting feature : What comparisons ?

? Références CASE 1 : FUNNEL DATA ANALYSIS By cohort

? Références CASE 1 : FUNNEL DATA ANALYSIS Remove superﬂuous information (Tufte’s rule)

? Références CASE 1 : FUNNEL DATA ANALYSIS Compare cohorts trends

? Références CASE 1 : FUNNEL DATA ANALYSIS Relabel the graphic

? Références CASE 1 : FUNNEL DATA ANALYSIS Make comparison& visual choices (reference line)

? Références CASE 1 : FUNNEL DATA ANALYSIS Provide insights

? Références CASE 1 : FUNNEL DATA ANALYSIS Add color, context & analysis

? Références CASE 1 : BEFORE/AFTER Source : www.storytellingwithdata.com

? Références CASE 2 : WHAT’S THE GOOD POINT SIZE ? FIGURE – Sample size = 500, point size = 3

? Références CASE 2 : WHAT’S THE GOOD POINT SIZE ? FIGURE – With Transparency ! (Sample size = 500, point size = 13)

? Références CASE 2 : WHAT’S THE GOOD POINT SIZE ? FIGURE – Context matters ! (Sample size = 500, point size = 13)

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC How learners behave during the course ?

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC How learners behave during the course ? Learner’s "activity" over time /space

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC How learners behave during the course ? Learner’s "activity" over time /space The MOOC’s stucture : FIGURE – The MOOC is made of 60 different steps (ressources)

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC How learners behave during the course ? Learner’s "activity" over time /space The MOOC’s stucture : FIGURE – The MOOC is made of 60 different steps (ressources) Learners may pick whatever step, whenever they want.

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Basic graphic # 1 : Barplot (stacked) 4.11 4.10 4.9 4.8 4.7 4.6 4.5 4.4 4.3 4.2 4.1 3.21 3.20 3.19 3.18 3.17 3.16 3.15 3.14 3.13 3.12 3.11 3.10 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 2.12 2.11 2.10 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 1.16 1.15 1.14 1.13 1.12 1.11 1.10 1.9 1.8 1.7 1.6 1.5 1.4 1.3 1.2 1.1 0 1000 2000 3000 Learners Step Completed Visited but not completed Comments Activity per step

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Basic graphic # 2 : Heatmap FIGURE – Heatmap showing activity over time

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Idea # 1 : Visualize steps visited over time (1 line = 1 learner) FIGURE – How to visualize “Trajectories” over time ?

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Too many starting points and trajectories ! FIGURE – How to visualize “Trajectories” over time ?

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Idea # 2 : Align starting points ! Change time for clicks sequence FIGURE – “Trajectories” as a sequence of “clicks”

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Steps vs clicks sequence should reveal = patterns FIGURE – “Trajectories” as a sequence of “clicks”

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Steps vs clicks sequence should reveal = patterns FIGURE – “trajectories” as a sequence of “clicks”

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Should work with many lines (many learners) FIGURE – “trajectories” as a sequence of “clicks”

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC Back to R..... FIGURE – First try : Each line is a learner’s path

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC FIGURE – Trick one : Apply transparency and grey lines

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC FIGURE – Trick two : Tiny lines and "jitter"

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC FIGURE – Trick three : Colored lines, "jitter" and simple labels

? Références CASE 3 : SEARCHING FOR LEARNERS IN A MOOC FIGURE – Trick four : Add context (analysis)

? Références WHAT TO REMEMBER From the viewer“data visualisation” are implicitly or explicitly comparisons or even tests (in the statistical sense) Graphics should help questioning

? Références WHAT TO REMEMBER From the viewer“data visualisation” are implicitly or explicitly comparisons or even tests (in the statistical sense) Graphics should help questioning They should provide elements, to answer (data at least)

? Références WHAT TO REMEMBER From the viewer“data visualisation” are implicitly or explicitly comparisons or even tests (in the statistical sense) Graphics should help questioning They should provide elements, to answer (data at least) If the question implies comparison, they should truthfully show the comparison

? Références WHAT TO REMEMBER : THERE ARE RULES Data visualisation is a visual language, so there are : Elements of language

? Références WHAT TO REMEMBER : THERE ARE RULES Data visualisation is a visual language, so there are : Elements of language Rules of use (spelling)

? Références WHAT TO REMEMBER : THERE ARE RULES Data visualisation is a visual language, so there are : Elements of language Rules of use (spelling) Grammar

? Références THE TRUTH DOES NOT LIVE IN 2D The “truth”....

? Références GRAPHS I LIKE Visualizing thousands of runners (A. Ottenheimer)

? Références GRAPHS I LIKE Atlas of Places (Muriz Djurdjevic & Thomas Paturet)

? Références GRAPHS I LIKE Experiments in Time-Distance map (Kohei Suguira)

? Références REFERENCES I Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1) :17–21. Bertin, J. (1970). La graphique. Communications, 15(1) :169–185.

? Références REFERENCES II Bertin, J. (1983). Semiology of graphics, translation from sémilogie graphique (1967). Buja, A., Cook, D., Hofmann, H., Lawrence, M., Lee, E.-K., Swayne, D. F., and Wickham, H. (2009). Statistical inference for exploratory data analysis and model diagnostics. Philosophical Transactions of the Royal Society of London A : Mathematical, Physical and Engineering Sciences, 367(1906) :4361–4383. Chen, C.-h., Härdle, W. K., and Unwin, A. (2007). Handbook of data visualization. Springer Science & Business Media. Cleveland, W. S. (1994). The Elements of Graphing Data. Hobart Press, Summit : NJ, 2 edition. Cleveland, W. S. and McGill, R. (1984). Graphical perception : Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association, 79(387) :531–554. Dix, A. and Ellis, G. (1998). Starting simple - adding value to static visualisation through simple interaction. In Eds. T. Catarci, M. F. Costabile, G. S. and Tarantino, L., editors, Proceedings of Advanced Visual Interfaces, pages 124–134. L’Aquila, Italy, ACM Press.

? Références REFERENCES III Few, S. (2008). Practical rules for using color in charts. Visual Business Intelligence Newsletter, (11). Gelman, A. (2004). Exploratory data analysis for complex models. Journal of Computational and Graphical Statistics, 13(4). Munzner, T. (2014). Visualization Analysis and Design. AK Peters Visualization Series. A K Peters/CRC Press, 1 edition. Treisman, A. (1985). Preattentive processing in vision. Computer Vision, Graphics, and Image Processing, 31(2) :156–177. Tufte, E. R. (2001). The Visual Display of Quantitative Information. Graphics Press, 2 edition. Tukey, J. W. (1977). Exploratory data analysis. Reading, Mass. Ware, C. (2012). Information visualization : perception for design. Elsevier.

? Références STAY IN TOUCH ! My website Data.visualisation.free.fr @Xtophe_Bontemps

#26 La data visualisation pour la data science ...

#26 La data visualisation pour la data science (et les data scientists)

More Decks by Toulouse Data Science

Other Decks in Technology

Featured

Transcript