R-DMDX and reproducibility What is DMDX? freely available written by psychologists used by psychologists (e.g. MACCS, CBU, UM-Wisconsin) active help-list supported Figure : Forster & Forster (2003), DMDX, Behavior Research Methods
R-DMDX and reproducibility What is DMDX? DMDX runs on Win32 machines Runs things - stimulus display - through DirectX User writes an .rtf item file DMDX parses the script and instructs DirectX Figure : Example script drawn from Matt Davis (MRC CBU) DMDX tutorial
R-DMDX and reproducibility Picture-word interference experiments with precise stimulus onset asynchronies (SOAs) In 2000, I was using Superlab and trying to conduct picture-word interference experiments ‘tree’ ‘shirt’
R-DMDX and reproducibility Picture-word interference experiments with precise stimulus onset asynchronies (SOAs) Picture-word interference effects were/are thought to obtain in narrow SOA windows semantic effect at -150ms phonological effect at +150ms Figure : Schriefers, Meyer, & Levelt (1990), JML
R-DMDX and reproducibility Picture-word interference experiments with precise stimulus onset asynchronies (SOAs) DMDX allowed me to get precise SOAs DMDX: event timing is coordinated with respect to a PC’s screen refresh cycle ticks via DirectX Figure : drawn from Matt Davis (MRC CBU) DMDX tutorial
R-DMDX and reproducibility Things DMDX continues to do that Superlab or other applications cannot . . . It is now no longer acceptable to extract vocal response reaction time using voicekeys Figure : Roelofs (2006)
R-DMDX and reproducibility Kessler & Treiman (JML): Phonetic biases in voicekey response time measurements Words with different initial phonemes have significantly different RTs Voiceless, posterior, and obstruent consonants detected later than others Phoneme-bases biases are large, persistant and pervasive, confounding RT interpretation Figure : Screen shot of spectrogram analysis window for stimulus “ask” naming response, CheckVocal (Protopapas, 2007, BRM)
R-DMDX and reproducibility Kessler & Treiman (JML): Phonetic biases in voicekey response time measurements Words with different initial phonemes have significantly different RTs Voiceless, posterior, and obstruent consonants detected later than others Phoneme-bases biases are large, persistant and pervasive, confounding RT interpretation
R-DMDX and reproducibility The DMDX-checkvocal-phoneme-coding combination The ‘gold standard’ Present stimuli and record spoken responses to hard disk (DMDX) Analyze spectrograms (Checkvocal) Code phonetic features and incorporate as covariates in regression analyses
R-DMDX and reproducibility DMDX vs. proprietary software DMDX advantages Active user community help: http://www.u.arizona.edu/ kforster/dmdx/listserv.htm Reactive creators: if enough people require a feature, it will be built in (within reason) Did I mention it is ‘free as in beer’? Reproducible research: where do your data come from? Figure : Example script drawn from Matt Davis (MRC CBU) DMDX tutorial
R-DMDX and reproducibility What is R? a system for statistical computation and graphics a language plus a run-time environment with graphics, a debugger and ability to run programmes stored in script files
R-DMDX and reproducibility R is used by a fast growing dynamic community “lingua franca” of professional statistics (Revolution-R) “second language” for post grads (Pfizer) “hard to overvalue” (google) Figure : Muenchen (website rforsasandspssusers.com): impact of software on publications/hits on google scholar
R-DMDX and reproducibility R is used by a fast growing dynamic community SAS has “customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet” (NYT, 2009) 2009: SAS announced plans to offer R integration Figure : Muenchen (website rforsasandspssusers.com): impact of software on publications/hits on google scholar
R-DMDX and reproducibility R is used by a fast growing dynamic community . . . because of rapid growth in capacity capabilities extended by user generated packages developed primarily in R (also sometimes Java, C and Fortran) alow specialized techniques: 4300 in March 2011
R-DMDX and reproducibility R better that SPSS Linear mixed-effects analyses I started using R in 2007 because SPSS took 12+ hours to complete an analysis completed by R in < 12 minutes
R-DMDX and reproducibility R better that SPSS Superficial differences R console: stripped down and intimidating but do you want pretty windows or effective functions?
R-DMDX and reproducibility R better that SPSS Profound differences R analysis is reproducible analysis: reproducibility is important and using SPSS makes it hard
R-DMDX and reproducibility R advantages R graphics capacities are supreme: How Michael Jackson Billboard Rankings Compare With Other Notable Artists (NYT) Figure : NYT, 2009
R-DMDX and reproducibility R advantages Active user community help: R help list, stack overflow, nabble, specialist user list servs . . . Reactive creators: if you want to do it, someone else has done it before and put code online Did I mention it is ‘free as in beer’? Reproducible research: where do your data come from? Figure : CC flickr 1875Brian
R-DMDX and reproducibility SPSS plus Word Where does this stuff come from? 1. how do researchers report (advertise) observations? 2. Do study – stimulus presentation parameters, programme version? 3. Collate data – who did what and under what circumstances? 4. Collate data – munge and save ‘final version’ for analysis, which version? 5. Perform analysis and choose outputs from one – which analysis, what analysis choices? 6. Write observations and copy plots into Word – which plots, which version analysis?
R-DMDX and reproducibility DMDX plus R plus latex 1. how do researchers report (advertise) observations? 2. Do study – stimulus presentation parameters self-report in DMDX item file 3. Collate data – collate, treat, combine self-report in .R analysis script 4. Collate data – data analyzed for report listed, all treatment noted, on-the-fly 5. Perform analysis and choose outputs from one – analysis choices commented 6. Write observations and copy plot names into latex – transparent link to analysis, data, source
R-DMDX and reproducibility Reproducibility: a scientific principle Perform analysis and choose outputs from one – analysis choices commented Figure : R plot code Figure : latex report code