Alex K Gold, RStudio Workweek 2019 Why Data Scientists Stink at Software Engineering 

Hello RStudio! ● Background in Math and Econ ○ Proud econ PhD dropout ● Think tank work ○ Economic policy ○ Microsimulation modeling ● “Data Science” ○ Ran voter outreach experiments for progressive causes and candidates ○ Ran small data science consulting team 

Am I talking to you? 

Data Scientists Stink at: ● Version control. ● Testing (esp test-driven development). They should do these things... But they often don’t... 

Version Control 

The difference... Client Report Data Scientist Software Engineer Delivery: Merge to master 

The Result... Please use version control. Nah. Alex Team 

Testing 

All of Data Science in 1 Slide* Data call firm_size it_sophistication Prediction Pr(call) ~ f(firm_size, it_sophistication) Inference call = ꞵ0 + ꞵ1 * firm_size + ꞵ2 * it_sophistication Clustering Group A Group B .... *Well, mostly. call firm_size it_sophistication call firm_size it_sophistication 

Let’s Do Test-Driven Development... Don’t know answer... ● If I knew, I wouldn’t need data. Unit tests? ● Just algorithmic validity... Model metrics? R2 AUC F1 Gini RMSE Pr(call) or ꞵ0, ꞵ1, ꞵ2 or Group A, B QA and testing should be done… ● Can’t really specify ahead ● 

Bottom Line 

The difference... Software Engineers Data Scientist Deliverable Working Code Code? Paper Slide deck Dashboard Model Best Practices Version Control Automated Testing/TDD ????? ?????* Thank you! ➔ Empathy? ➔ It’s not hopeless ➔ Talk to me about “agile data science”