Improving Research
Through Advanced REDCap
Interfaces
[email protected]
Education and Brain Science Research Lab
Slide 2
Slide 2 text
EBRL
• Very wide databases
• Very expensive datasets
• Novel tasks (in & out of magnet)
• Many projects
We study reading disabilities in children using
behavioral and MR imaging measures
Slide 3
Slide 3 text
Before REDCap
• Members touched every piece of data
• Issues joining across paradigms
• Saved and shared data in spreadsheets
• Always behind in analyses
• No traceable analyses
Input ≫ Output
Slide 4
Slide 4 text
After REDCap
• Analyze some data within milliseconds
• Automate everything possible
• Automate the automation
• Start analyses from a single source
Slide 5
Slide 5 text
Goals
• Advocate for advanced data management
workflows
• Discuss solving problems using REDCap’s
Application Programming Interface
• Explain how Data Entry Triggers can
connect infrastructure
Slide 6
Slide 6 text
Scaling Science
• More subjects & more captured data
• Humans don’t scale efficiently
• How to do better work in less time with
less money?
Slide 7
Slide 7 text
Ideally...
• Perform reproducible work
• Operate deterministically
• Orders of magnitude faster and cheaper
Machines perform all definable analyses:
Slide 8
Slide 8 text
REDCap
• Is:
• A service for collecting and storing data
• Secure for the storage of PHI
• An online spreadsheet
• Is not:
• A relational database
Slide 9
Slide 9 text
Better than...
• A real database:
• No administration
• Easy schema definition
• No security worries
• A spreadsheet:
• GUI is browser-based
• Client-Server architecture
• Advanced web features
Slide 10
Slide 10 text
General Architecture
REDCap
Server
Web-App
API
Lab Server
Humans
Slide 11
Slide 11 text
Advanced Features
• Application Programming Interface (API)
• Programmatic access to REDCap
• Data Entry Triggers
• Automated notifications
All the building blocks we need
Slide 12
Slide 12 text
API
A method for software programs to ask for and
push data to REDCap projects
Slide 13
Slide 13 text
REDCap
Server
Web-App
API
Lab Server
Humans
API
Slide 14
Slide 14 text
Using the API
HTTP POST to API URL
Any programming environment with an HTTP
library can use the API
(http://sburns.github.io/PyCap)
Slide 15
Slide 15 text
Major API Methods
• Metadata Export
• Data Export
• Data Import
• File Import, Export & Deletion
(https://redcap.vanderbilt.edu/api/help)
Slide 16
Slide 16 text
API: Possible Uses
• Advanced & automated field calculation
• Otherwise-impossible data upload
• REDCap as the input for external systems
• Shared Filesystem
• Across-project data movement
Slide 17
Slide 17 text
API: Field Calculation
Download,
Implement,
Upload
REDCap
Calculated
Fields
API
Problem: How to update
(many) fields across
(many) records?
Slide 18
Slide 18 text
Impossible Data Uploads
• Analyses can produce >1000 fields per
record
• Collect 1000s of records per day
Slide 19
Slide 19 text
API: External Systems
• Hooks to external databases
• Reproducible cohort/group
determination
• Automated database cleanup & backup
Slide 20
Slide 20 text
API: Shared Filesystem
How to insert or generate intermediate
data to/from our analysis infrastructure?
• Secure
• Easy
• Automated
API: Shared Filesystem
Fields ⟶ file:
• Software will:
• Download data for that record
• Substitute into a predefined template
• Upload new report to REDCap
• Alert lab members through email
Slide 23
Slide 23 text
API: Shuttle Data
• Capture data in one project
• Export and analyze through API
• Import results into same or other
• No need to duplicate data entry fields
Slide 24
Slide 24 text
API: Shuttle Data
• Capture data in public survey
• Manually verify
• Easily copy to new record in private
project
Slide 25
Slide 25 text
API
Slide 26
Slide 26 text
API: Pitfall
API only serves external requests
• One-off scripts
• Scheduled programs
Slide 27
Slide 27 text
API: Pitfall
Better idea about when to run analyses?
Slide 28
Slide 28 text
Data Entry Triggers
• Independent of but complimentary to API
• Register a URL to your project
• Internet notification when data is saved
• Notification contains context of the save
Slide 29
Slide 29 text
Data Entry Triggers
Slide 30
Slide 30 text
Data Entry Triggers
REDCap
Server
Web-App
API
Lab Server
Humans
Slide 31
Slide 31 text
(Super) Data Entry
Triggers
REDCap
Server
Web-App
API
Lab Server
Humans
Slide 32
Slide 32 text
Data Entry Triggers:
Pitfalls
• Not every research group:
• Can setup, maintain & secure a web
server
• Has the resources to write the web-app
But every lab should have access to this
infrastructure!
Slide 33
Slide 33 text
Switchboard
• I wrote a web-app to:
• Parse incoming REDCap requests
• Execute functions that “match” the
request
• In production for our lab
(http://github.com/sburns/switchboard)
Slide 34
Slide 34 text
Data Entry Triggers
In a perfect world, we all share a KC-wide
web-server
• Just one server to maintain & protect
• Sharing is good
• Remove excuses for buy-in
• Everyone benefits from optimization
Slide 35
Slide 35 text
Conclusions
Slide 36
Slide 36 text
Engineering Better
Science
• All the pieces exist to offload a massive
amount of data-management work from
humans to machines
• Cost-effective and improves work through
improved accuracy and reproducibility
• Let machines do that which can be defined
• Let humans do the hard work
Slide 37
Slide 37 text
Automation improves
research
(Easier to automate machines than humans)
REDCap
Server
Web-App
API
Lab Server
Humans
Slide 38
Slide 38 text
Automate the automation
(Machines don’t make excuses)
REDCap
Server
Web-App
API
Lab Server
Humans
Slide 39
Slide 39 text
Thank you
Laurie Cutting, Ph.D. Nikki Davis, Ph.D. Sheryl Rimrodt, M.D.
REDCap Team (Paul Harris, Rob Taylor, etc)