Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Improving Research With Advanced REDCap Interfaces

Improving Research With Advanced REDCap Interfaces

REDCap is a general-purpose data storage system that both researchers and clinicians of all technical backgrounds can easily use to make robust databases. Because of how it is designed, REDCap provides advanced features for interacting and manipulating stored data. These advanced features are very useful, though used rarely. For example, the Application Programming Interface (API) and Data Entry Triggers are two advanced features that can fundamentally change how a lab or clinical group interacts with subject or patient data. These features are foundational to building advanced data management techniques that improve the efficacy, increase reliability and ultimately improve science and clinical work. This talk will introduce these features, discuss their need and provide implementation examples.


Scott Burns

July 26, 2013


  1. Improving Research Through Advanced REDCap Interfaces scott.s.burns@vanderbilt.edu Education and Brain

    Science Research Lab
  2. EBRL • Very wide databases • Very expensive datasets •

    Novel tasks (in & out of magnet) • Many projects We study reading disabilities in children using behavioral and MR imaging measures
  3. Before REDCap • Members touched every piece of data •

    Issues joining across paradigms • Saved and shared data in spreadsheets • Always behind in analyses • No traceable analyses Input ≫ Output
  4. After REDCap • Analyze some data within milliseconds • Automate

    everything possible • Automate the automation • Start analyses from a single source
  5. Goals • Advocate for advanced data management workflows • Discuss

    solving problems using REDCap’s Application Programming Interface • Explain how Data Entry Triggers can connect infrastructure
  6. Scaling Science • More subjects & more captured data •

    Humans don’t scale efficiently • How to do better work in less time with less money?
  7. Ideally... • Perform reproducible work • Operate deterministically • Orders

    of magnitude faster and cheaper Machines perform all definable analyses:
  8. REDCap • Is: • A service for collecting and storing

    data • Secure for the storage of PHI • An online spreadsheet • Is not: • A relational database
  9. Better than... • A real database: • No administration •

    Easy schema definition • No security worries • A spreadsheet: • GUI is browser-based • Client-Server architecture • Advanced web features
  10. General Architecture REDCap Server Web-App API Lab Server Humans

  11. Advanced Features • Application Programming Interface (API) • Programmatic access

    to REDCap • Data Entry Triggers • Automated notifications All the building blocks we need
  12. API A method for software programs to ask for and

    push data to REDCap projects
  13. REDCap Server Web-App API Lab Server Humans API

  14. Using the API HTTP POST to API URL Any programming

    environment with an HTTP library can use the API (http://sburns.github.io/PyCap)
  15. Major API Methods • Metadata Export • Data Export •

    Data Import • File Import, Export & Deletion (https://redcap.vanderbilt.edu/api/help)
  16. API: Possible Uses • Advanced & automated field calculation •

    Otherwise-impossible data upload • REDCap as the input for external systems • Shared Filesystem • Across-project data movement
  17. API: Field Calculation Download, Implement, Upload REDCap Calculated Fields API

    Problem: How to update (many) fields across (many) records?
  18. Impossible Data Uploads • Analyses can produce >1000 fields per

    record • Collect 1000s of records per day
  19. API: External Systems • Hooks to external databases • Reproducible

    cohort/group determination • Automated database cleanup & backup
  20. API: Shared Filesystem How to insert or generate intermediate data

    to/from our analysis infrastructure? • Secure • Easy • Automated
  21. API: Shared Filesystem File ⟶ fields: • Software will: •

    Download file locally • Analyze file • Upload results to REDCap
  22. API: Shared Filesystem Fields ⟶ file: • Software will: •

    Download data for that record • Substitute into a predefined template • Upload new report to REDCap • Alert lab members through email
  23. API: Shuttle Data • Capture data in one project •

    Export and analyze through API • Import results into same or other • No need to duplicate data entry fields
  24. API: Shuttle Data • Capture data in public survey •

    Manually verify • Easily copy to new record in private project
  25. API

  26. API: Pitfall API only serves external requests • One-off scripts

    • Scheduled programs
  27. API: Pitfall Better idea about when to run analyses?

  28. Data Entry Triggers • Independent of but complimentary to API

    • Register a URL to your project • Internet notification when data is saved • Notification contains context of the save
  29. Data Entry Triggers

  30. Data Entry Triggers REDCap Server Web-App API Lab Server Humans

  31. (Super) Data Entry Triggers REDCap Server Web-App API Lab Server

  32. Data Entry Triggers: Pitfalls • Not every research group: •

    Can setup, maintain & secure a web server • Has the resources to write the web-app But every lab should have access to this infrastructure!
  33. Switchboard • I wrote a web-app to: • Parse incoming

    REDCap requests • Execute functions that “match” the request • In production for our lab (http://github.com/sburns/switchboard)
  34. Data Entry Triggers In a perfect world, we all share

    a KC-wide web-server • Just one server to maintain & protect • Sharing is good • Remove excuses for buy-in • Everyone benefits from optimization
  35. Conclusions

  36. Engineering Better Science • All the pieces exist to offload

    a massive amount of data-management work from humans to machines • Cost-effective and improves work through improved accuracy and reproducibility • Let machines do that which can be defined • Let humans do the hard work
  37. Automation improves research (Easier to automate machines than humans) REDCap

    Server Web-App API Lab Server Humans
  38. Automate the automation (Machines don’t make excuses) REDCap Server Web-App

    API Lab Server Humans
  39. Thank you Laurie Cutting, Ph.D. Nikki Davis, Ph.D. Sheryl Rimrodt,

    M.D. REDCap Team (Paul Harris, Rob Taylor, etc)
  40. Email: scott.s.burns@vanderbilt.edu Github: http://github.com/sburns API& PyCap Tutorial: http://bit.ly/pycap-tutorial

  41. Questions & Comments?