FISH 6002: Week 7 - Displaying Data Visually 1

Oct 2, 2017 lecture of FISH 6002 - revised again oct 21 2019

Transcript

1. Week 7: Displaying Data Visually FISH 6000: Science Communication for Fisheries Brett Favaro 2017 This work is licensed under a Creative Commons Attribution 4.0 International License
2. http://derekogle.com/fishR/data/data-html/WhitefishLS.html
3. Ellen Edmonson and Hugh Chrisp [Public domain]
4. #year: Year of catch #state: State of catch (MI, MN, or WI) #catch: Catch in lbs #value: Value of catch in dollars Date
5. Use the lubridate package https://lubridate.tidyverse.org/ Before we can graph effectively, we must prepare our data
6. Use the lubridate package https://lubridate.tidyverse.org/
7. Month Day Year Hour Minute Second df$date df$date mdy_hms("August 03 2017 07:14:36 PM") [1] "2017-08-03 19:14:36 UTC" mdy_hms("August 03 2017 07:14:36 PM", tz="America/St_Johns") [1] "2017-08-03 19:14:36 NDT"
8. [1] "2017/08/03 13:53" Month Day Year Hour Minute ymd_hm("2017/08/03 13:53", tz="America/St_Johns") [1] "2017-08-03 13:53:00 NDT"
9. > dmy("3/8/2017") [1] "2017-08-03" > dmy("3/August/2017") [1] "2017-08-03" > dmy("3/Aug/2017") [1] "2017-08-03" > dmy("3/AUG/2017") [1] "2017-08-03" > dmy("3/AuG/2017") [1] "2017-08-03"
10. Datetime ymd_hm("2017/08/03 13:53", tz="America/St_Johns") [1] "2017-08-03 13:53:00 NDT" hm("13:53") [1] "13H 53M 0S" Date Time Number of seconds since 0:00 ymd("2017/08/03") [1] "2017-08-03"
11. Example date_df <- read.csv("./data/DateExample.csv") head(date_df) fishing boat by Symbolon from the Noun Project Question: How long was each deployment?
12. Break the problem into steps Each row is an action (setting or hauling gear) within a deployment. Deployment is given by deployment_id. For each action we have a date and a time. We need to create a date-time and calculate the amount of time that passed Problem: Remember, if we use hms to look at time data, we get TIME PASSED SINCE 0:00:00 and AM/PM is ignored. No good!
13. Combine the date and time into one column Then, make it a Date-Time date_df <- date_df %>% unite(Date, Time, col="DateTime", sep=" ")
14. Now, calculate durations Let’s make it wide, so within a deployment we have a START and END time date_wide <- date_wide %>% mutate(SoakTime = as.duration(set %--% haul))
15. Durations are stored in seconds, and expressed by default in the most appropriate unit (hrs, days, etc) mean(date_wide$SoakTime) [1] 866880 median(date_wide$SoakTime) [1] 189000 If you perform mathematical operations, they will be performed on seconds
16. date_df <- read.csv("./data/DateExample.csv") date_df %>% unite(Date, Time, col="DateTimeRaw", sep=" ") %>% #Combine Dates and Times mutate(DateTime = mdy_hms(DateTimeRaw, tz="America/St_Johns")) %>% # Make Date-Time select(-DateTimeRaw) %>% # Remove the non Date-Time column spread(key=action, value=DateTime) %>% # Make it wide format mutate(SoakTime = as.duration(set %--% haul)) #Calculate duration btwn set and haul A note about piping: This does everything we just did, in a single chunk of code
17. #year: Year of catch #state: State of catch (MI, MN, or WI) #catch: Catch in lbs #value: Value of catch in dollars Date
18. whitefish$year <- date_decimal(whitefish$year) Where we have year only: Now, let’s make some graphs!
20. Graphs are visual representations of data Humans need this
21. Data visualization, or dataviz, can completely alter our perception of a subject Rose and Rowe (2015) DFO (2016) Both depictions are true… do they change your interpretation?
22. Graphs should be: • An honest representation of the data • An effective display • Efficient – i.e. they should use space well • Visually appealing! (Scientists like aesthetics too) Graphs should tell a story
23. 3 plotting “ecosystems” in R Base ggplot2 lattice Each has different strengths. Very different syntax We will focus on Base, and ggplot2, not lattice. https://bookdown.org/rdpeng/exdata/plotting-systems.html#the-base-plotting-system
24. What graph do I make? Depends on: • Type of data • Continuous • Discrete https://github.com/allisonhorst/stats-illustrations
25. One-variable plots What stories can we tell with these? Continuous Discrete 0 10000 Apple Banana Count Count Density Proportion
26. Start with the easiest table(whitefish$state) MI MN WI 31 11 31 plot(table(whitefish$state)) One-variable – Discrete X Response is “count”
27. barplot(table(whitefish$state)) plot(table(whitefish$state)) The plot command executes a default plot, based on data type. These plots are minimalist in nature. Contrast barplot with plot – what’s different?
28. https://www.edwardtufte.com/tufte/ Data ink -------------------------- Total ink used in graphic Data:ink ratio = Fill up the plot with meaningful ink – not decorative ink # of pieces of data -------------------------- Area of graphic Data density = R’s plot command, by default, uses plots that maximize data:ink ratio Edward Tufte
29. Everything on a plot should be meaningful Avoid Chartjunk http://www.infovis-wiki.net/index.php/Chart_Junk
30. One-variable plots – Continuous Continuous data (Measured) Counts rows of data This is a scatterplot plot(whitefish$catch) In this context, a scatterplot is only useful as it helps us look for outliers
31. A histogram displays frequency of observations that fit within a specified range Height = count of observations that fall into that interval Intervals are (a, b] by default: They include right hand endpoint There were 17 observations where catch was between zero and ~200,000 hist(whitefish$catch)
32. In R, all graphical elements can be customized (We will look at aesthetics in future weeks)
33. hist(whitefish$catch, breaks=10) hist(whitefish$catch, breaks=20) hist(whitefish$catch, breaks=30)
34. hist(whitefish$catch, breaks=c(0,500000,1000000,1500000, 2000000)) hist(whitefish$catch, breaks=c(0,100000,200000,300000,400000,500000, 1000000,1500000,2000000))
35. When breaks aren’t uniform: Add up all bar areas to get 1 When breaks uniform: Add up all bar heights to get count of observations
36. # Many calculations are conducted by the hist function prior to plotting # and you can extract those values histinfo <- hist(whitefish$catch)
37. Everything in this plot has meaning Title – what is this plot? Axis labels – what is being plotted? Values Bar area and height convey meaning – width tells interval. Height tells count
38. Plot of mean catch by state (two-variable) How many pieces of information are present in the plot space?
39. These are, essentially, the same plot. The bar area does not convey information
40. 3 points of data visible data:ink ratio low All 73 observations visible, summarized as 9 bars data:ink ratio higher
41. ggplot2 # ggplot(data = df, # aes(DATA YOU WANT TO PLOT)) + # GRAPHICAL ELEMENTS, ADDED ONE BY ONE ggplot(data = whitefish, aes(state)) + geom_bar()
42. ggplot2 a <- ggplot(data = whitefish, aes(state)) + geom_bar() print(a) a <- ggplot(data = whitefish, aes(state)) print(a) Key point: ggplot2 works with layered elements
43. How did I know to type geom_bar()? I knew I had… ONE VARIABLE and it is… discrete https://github.com/rstudio/cheatsheets/raw/master/data-visualization-2.1.pdf
44. a <- ggplot(data = whitefish, aes(state)) + geom_bar() print(a) barplot(table(whitefish$state)) In base plot, I had to manually calculate a count (using table()), and use the plot() or barplot() commands. Different commands for different plots, and elements must be manually calculated. ggplot uses the same basic syntax for all plots. Elements are layered
45. One-variable plots – Continuous with ggplot() a <- ggplot(data=whitefish, aes(catch))
46. ggplot() plots are customizable a + geom_histogram(bins=10) a + geom_histogram(bins=20) a + geom_histogram(bins=30) a + geom_histogram(bins=40)
47. Defaults differ across ecosystems: a + geom_area(stat="bin") `stat_bin()` using `bins = 30`. Pick better value with `binwidth`. R teachable moment: Always understand the defaults
48. a + geom_histogram(bins = 10) + coord_flip() ggplot has built-in functions to make common adjustments to plots with little code e.g. sideways histograms are common in population data Lindeman (1942)
49. What graph do I make? Depends on: • Type of data • Continuous • Discrete • Nominal • Ordinal • Binary https://github.com/allisonhorst/stats-illustrations
50. Nominal Watermelons Oranges Number eaten Cherries Watermelons Oranges Number eaten Cherries Ordinal (based on size) Not Citrus Citrus Number eaten Binary
51. Building a graph is building a narrative What Contrast do I want to emphasize? What story am I trying to tell? Must be honest, efficient, and effective
52. Two-variable plots Discrete Continuous Discrete Discrete Continuous Continuous
53. Continuous Continuous plot(value ~ catch, data=whitefish) Scatterplot Each circle = one point of data. Conveys X and Y value Base plot uses empty circles by default, so you can see partially- overlapping points
54. When designing graphs, ask what question does this graph answer? Y should be a response variable X, an explanatory variable Contrast: Catch Catch Value Value What question is each graph answering?
55. Another example: Is one of these plots more meaningful? Why? Catch Year Catch Year
56. In base plot, you add elements manually “Make a straight line, of the results of a linear model” “Add it to whatever has previously been plotted” Scatterplots are often enhanced with visual aids plot(catch ~ year, data=whitefish) abline(lm(catch~year, data=whitefish)) The order of execution matters…
57. abline(lm(catch~year, data=whitefish)) Error in int_abline(a = a, b = b, h = h, v = v, untf = untf, ...) : plot.new has not been called yet plot(catch ~ year, data=whitefish) plot(catch ~ year, data=whitefish) abline(lm(catch~year, data=whitefish))
58. In ggplot a <- ggplot(data=whitefish, aes(x = year, y = catch)) a + geom_point() The aesthetics are: Year on X Catch on Y Then, add points
59. Note ggplot syntax: a <- ggplot(data=whitefish, aes(x = year, y = catch)) a + geom_point() a <- ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() print(a) #or just a Same result
60. a <- ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() + geom_smooth(method = “lm”) # add a straight line of best fit a ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() + geom_smooth() # note no method=“lm” Can you explain all elements?
61. Know what your settings are doing! geom_smooth {ggplot2}
62. ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() + geom_smooth(method = “lm”, level=0.99) level=0.95 level=0.5 ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() + geom_smooth(span=1) span=100
63. Disclose, in methods or figure caption how you made the plot Figure 1: Scatterplot of catch (Y) by year (X) of Lake Whitefish in Lake Superior. The blue line depicts a loess smoother with a span value of 100 (implemented in ggplot2), and the grey shading is a 95% confidence interval.
64. plot(catch ~ year, data=whitefish) abline(lm(catch~year, data=whitefish)) ggplot(data=whitefish, aes(x = year, y = catch)) + geom_point() + geom_smooth(method = “lm”) When X and Y are both continuous you’re visualizing a relationship between two variables
65. Also: be careful with drawing lines through things.. http://guessthecorrelation.com/
66. Two-variable plots Discrete Continuous Discrete Discrete Continuous Continuous
67. Continuous Y, Discrete X Discrete Continuous When X is discrete, you are contrasting across groups “What’s the median catch across states”?
68. “What’s the median catch across states”? Raw data: - How many points of data are shown? - Is data:ink ratio high or low? - Is data density high or low? - How many points of data are shown? - Is data:ink ratio higher, lower, or same? - Is data density higher, lower, or same?
69. https://doi.org/10.1371/journal.pbio.1002128 Weissgerber et al. 2015 Too minimal = miss insights
70. plot(catch ~ state, data=whitefish) Boxplots allow for visual comparison of median while also showing spread of data
71. Median Boxplot 1st quartile 3rd quartile min max Outlier Outlier > Q3 + 1.5*IQR < Q1 - 1.5*IQR Default: whiskers extend UP TO 1.5 times the IQR from Q3 – no further IQR: Inter-quartile range
72. But: boxplots can mislead too https://timo-roettger.weebly.com/thinking-outside-the-boxplot All visualizations can have problems…
73. https://www.data-to-viz.com/caveat/boxplot.html Options: Plot raw data AND summary visual ggplot(data=whitefish, aes(x = state, y = catch)) + geom_jitter() + geom_boxplot(alpha=0.7)
74. Options: use another visualization a + geom_col() Visuals for cumulative amount
75. plot(catch ~ state, data=whitefish) ggplot(data=whitefish, aes(x = state, y = catch)) + geom_boxplot()
77. Two-variable plots Discrete Continuous Discrete Discrete Continuous Continuous
78. Discrete Discrete Need to add another discrete variable… whitefish <- whitefish %>% mutate(BigState = ifelse(state=="MI", "Y", "N")) Is this observation from a “big state”? ggplot(data=whitefish, aes(x=state, y = BigState)) + geom_count()
79. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0113118
80. One-variable plots What questions can we answer? Continuous Discrete 0 10000 Apple Banana Count Count Density Proportion Q: How many apples vs. bananas did we eat? We ate more apples than bananas Q: What was the distribution of catch data? Mostly small catches, a few large catches Catch
81. Two-variable plots Discrete Continuous Continuous Continuous Q: Is there a relationship between my explanatory (Y) and response variable (X)? year catch 1970 2000 A: Maybe, catch increases over time sort of state catch Q: Is there a difference between catches across states? A: Yes, Catch higher in MI, etc. MI MN WI
82. Defining your question is the most important part of graph-making The second most important part is accurately describing your data: - Continuous? - Discrete - Ordinal - Nominal - Binary 3rd: Decide what is explanatory and what is response variables (for two-variable plots) Finally… make the graph! (In base or ggplot)
83. https://github.com/rstudio/cheatsheets/raw/m aster/data-visualization-2.1.pdf Use the ggplot cheat sheet to help you figure out what type of plot to make
84. Activity There are four graphs printed. In pairs or 3’s, take one. • What type of graph is it? • What type(s) of data are in the graph? (discrete, continuous) • What question is the graph asking? • What inference could you draw from this graph? • Describe the main finding shown in the graph
85. Example Apple Banana Count This is a barplot. It is a one-variable plot, and the variable is “fruit type” which is discrete – nominal The question it is asking is: “How many fruits of each type were eaten” and the inference you would draw is “Which fruit type is eaten more?” Described: “Many more apples were eaten than bananas”
86. Activity There are three graphs printed. In pairs or 3’s, take one. • What type of graph is it? • What type(s) of data are in the graph? (discrete, continuous) • What question is the graph asking? • What inference could you draw from this graph? • Describe the main finding shown in the graph
87. This figure shows the CATCH PER UNIT EFFORT (i.e. snow crabs caught per trap deployment) across several types of bait on the X axis. https://peerj.com/articles/6874/ Araya-Schmidt et al. 2019
88. Number of hurricanes per two-week period Number of “wrecked birds” per two-week period – i.e. dead birds found on beaches, presumably killed due to hurricanes https://peerj.com/articles/3287/ Huang et al. 2017
89. https://peerj.com/articles/5306/ Tixier et al. 2018 Depredation is when something takes bait off a hook. Here, they went fishing many days and recorded whether or not depredation occurred on their fishing gear. Black bars are Amsterdam/St. Paul, grey bars are SE Australia
90. Consider a number series, with values as follows EXTRA SLIDES: Boxplot explainer
91. Consider a number series, with values as follows Median 1st quartile 3rd quartile
92. IQR = Q3 – Q1 13 – 3 = 10
93. Q3 + 1.5 * IQR 13 + 1.5 * 10 = 28 So… everything > 28 = boxplot outlier IQR = Q3 – Q1 13 – 3 = 10
94. Q3 + 1.5*IQR 13 + 1.5 * 10 = 28 Everything > 28 = outlier IQR = Q3 – Q1 14 – 3 = 11 If we also had 27 in the number series… No more outliers! IQR = Q3 – Q1 13 – 3 = 10 IQR = Q3 – Q1 14 + 1.5 * 11 = 26.5 Everything > 26.5