Improving Research With Advanced REDCap Interfaces

REDCap is a general-purpose data storage system that both researchers and clinicians of all technical backgrounds can easily use to make robust databases. Because of how it is designed, REDCap provides advanced features for interacting and manipulating stored data. These advanced features are very useful, though used rarely. For example, the Application Programming Interface (API) and Data Entry Triggers are two advanced features that can fundamentally change how a lab or clinical group interacts with subject or patient data. These features are foundational to building advanced data management techniques that improve the efficacy, increase reliability and ultimately improve science and clinical work. This talk will introduce these features, discuss their need and provide implementation examples.

Transcript

1. Improving Research Through Advanced REDCap Interfaces scott.s.burns@vanderbilt.edu Education and Brain Science Research Lab
2. EBRL • Very wide databases • Very expensive datasets • Novel tasks (in & out of magnet) • Many projects We study reading disabilities in children using behavioral and MR imaging measures
3. Before REDCap • Members touched every piece of data • Issues joining across paradigms • Saved and shared data in spreadsheets • Always behind in analyses • No traceable analyses Input ≫ Output
4. After REDCap • Analyze some data within milliseconds • Automate everything possible • Automate the automation • Start analyses from a single source
5. Goals • Advocate for advanced data management workflows • Discuss solving problems using REDCap’s Application Programming Interface • Explain how Data Entry Triggers can connect infrastructure
6. Scaling Science • More subjects & more captured data • Humans don’t scale efficiently • How to do better work in less time with less money?
7. Ideally... • Perform reproducible work • Operate deterministically • Orders of magnitude faster and cheaper Machines perform all definable analyses:
8. REDCap • Is: • A service for collecting and storing data • Secure for the storage of PHI • An online spreadsheet • Is not: • A relational database
9. Better than... • A real database: • No administration • Easy schema definition • No security worries • A spreadsheet: • GUI is browser-based • Client-Server architecture • Advanced web features
10. General Architecture REDCap Server Web-App API Lab Server Humans
11. Advanced Features • Application Programming Interface (API) • Programmatic access to REDCap • Data Entry Triggers • Automated notifications All the building blocks we need
12. API A method for software programs to ask for and push data to REDCap projects
13. REDCap Server Web-App API Lab Server Humans API
14. Using the API HTTP POST to API URL Any programming environment with an HTTP library can use the API (http://sburns.github.io/PyCap)
15. Major API Methods • Metadata Export • Data Export • Data Import • File Import, Export & Deletion (https://redcap.vanderbilt.edu/api/help)
16. API: Possible Uses • Advanced & automated field calculation • Otherwise-impossible data upload • REDCap as the input for external systems • Shared Filesystem • Across-project data movement
17. API: Field Calculation Download, Implement, Upload REDCap Calculated Fields API Problem: How to update (many) fields across (many) records?
18. Impossible Data Uploads • Analyses can produce >1000 fields per record • Collect 1000s of records per day
19. API: External Systems • Hooks to external databases • Reproducible cohort/group determination • Automated database cleanup & backup
20. API: Shared Filesystem How to insert or generate intermediate data to/from our analysis infrastructure? • Secure • Easy • Automated
21. API: Shared Filesystem File ⟶ fields: • Software will: • Download file locally • Analyze file • Upload results to REDCap
22. API: Shared Filesystem Fields ⟶ file: • Software will: • Download data for that record • Substitute into a predefined template • Upload new report to REDCap • Alert lab members through email
23. API: Shuttle Data • Capture data in one project • Export and analyze through API • Import results into same or other • No need to duplicate data entry fields
24. API: Shuttle Data • Capture data in public survey • Manually verify • Easily copy to new record in private project
25. API
26. API: Pitfall API only serves external requests • One-off scripts • Scheduled programs
27. API: Pitfall Better idea about when to run analyses?
28. Data Entry Triggers • Independent of but complimentary to API • Register a URL to your project • Internet notification when data is saved • Notification contains context of the save
29. Data Entry Triggers
30. Data Entry Triggers REDCap Server Web-App API Lab Server Humans
31. (Super) Data Entry Triggers REDCap Server Web-App API Lab Server Humans
32. Data Entry Triggers: Pitfalls • Not every research group: • Can setup, maintain & secure a web server • Has the resources to write the web-app But every lab should have access to this infrastructure!
33. Switchboard • I wrote a web-app to: • Parse incoming REDCap requests • Execute functions that “match” the request • In production for our lab (http://github.com/sburns/switchboard)
34. Data Entry Triggers In a perfect world, we all share a KC-wide web-server • Just one server to maintain & protect • Sharing is good • Remove excuses for buy-in • Everyone benefits from optimization
35. Conclusions
36. Engineering Better Science • All the pieces exist to offload a massive amount of data-management work from humans to machines • Cost-effective and improves work through improved accuracy and reproducibility • Let machines do that which can be defined • Let humans do the hard work
37. Automation improves research (Easier to automate machines than humans) REDCap Server Web-App API Lab Server Humans
38. Automate the automation (Machines don’t make excuses) REDCap Server Web-App API Lab Server Humans
39. Thank you Laurie Cutting, Ph.D. Nikki Davis, Ph.D. Sheryl Rimrodt, M.D. REDCap Team (Paul Harris, Rob Taylor, etc)
40. Email: scott.s.burns@vanderbilt.edu Github: http://github.com/sburns API& PyCap Tutorial: http://bit.ly/pycap-tutorial
41. Questions & Comments?