Surrogate outcomes

Luca Pozzi presents Surrogate Outcomes And Their Use In Statistical Experimentation

Synopsis: A surrogate endpoint is an outcome variable that can be used instead of the main outcome of interest in the evaluation of an experimental treatment. While the treatment effect associated with a surrogate endpoint might not be of any direct value, it can be used to predict the corresponding effect that would have been achieved by key outcomes. The talk will focus on alternative definitions and applications of surrogates and discuss few real world examples.

Bio: Luca Pozzi got his PhD from UC Berkeley and worked as a consultant for Novartis, FAO, Facebook and many more, in fields that range from Multiple Sclerosis Clinical Research to Breast Cancer Epidemiology, from Genomics to Social Networks. He's currently a Data Scientist and Machine Learning expert at Radius Intelligence. When he's not building a Mathematical model or designing an experiment Luca can be found swimming with the Sea Lions at Aquatic Park.

Transcript

1. YOU PAY FOR THIS,
   BUT THEY GIVE
   YOU THAT. Luca Pozzi Surrogacy & Instruments for Statistical Inference.
2. MS Applied Mathematics PhD Biostatistics U.C. Berkeley Independent Consultant (European Union, Novartis Pharma, FAO, etc…) Facebook Inc. Radius Intelligence Escaped from Alcatraz multiple times ABOUT ME
   
3. #computationalstatistics #machinelearning #bayesianstatistics #causalinference #genomics #machinelearning #timeseries #montecarlo #dataviz #designofexperiment #muchmore RESEARCH INTERESTS
   
4. CONTENTS
   • Design of Experiment • Surrogate Endpoints • Instrumental Variables • Example 1: Multiple Sclerosis • Example 2: Online Activity
5. DESIGN OF EXPERIMENTS
   1. Fixed Designs the characteristics of the study remain constant. e.g. AB Testing, Factorial Designs, etc. 2. Adaptive Designs adjust characteristics of the ongoing trial as more information is accrued. e.g. Bayesian Rules, Multi Armed Bandits, etc.
6. “A response variable for which a test of the null hypothesis of no relationship to the treatment groups under comparison is also a valid test of the corresponding null hypothesis based on the outcome of interest” Prentice (1989) SURROGATE ENDPOINT
   
7. f(S|Z) = f(S) iff f(C|Z) = f(C) Conditional independence assumption suggests a causal interpretation. PRENTICE CRITERION
8. INSTRUMENTAL VARIABLE X Endogenous Variable Z Instrument Z X Y
9. SOME EXAMPLES • Vietnam Draft draft number as an instrument to predict long term earnings • Protocol Compliance treatment assignment as an instrument to predict treatment effect.
10. VALIDATION VS. EVALUATION
    • Validation Establish the existence of a Surrogacy relationship. • Evaluation Exploit the way the surrogate provides information on the primary outcome.
11. 2 STAGE LEAST SQUARES •Main Idea: Z as an Instrument for propagating an impulse from X to Y. • 2SLS: Stage 1: regress X on Z; Stage 2: regress Y on the fitted values of the first step. • Weighted Average (LATE)
12. RELAPSING REMITTING MULTIPLE SCLEROSIS
    • MRI based lesion counts. • ARR (Annualized Relapse Rate). • EDSS (Expanded Disability Scale Score).
13. • Meta-Analysis: data from systematic review (19 trials, 44 arms, 25 contrasts, 11,375 patients). • Missing Data: not all trials measure the 3 outcomes. • Measurement Error. PROBLEM SETTING
    
14. • Sormani et al. weighted regression: Simple weighted regression. • Daniels and Hughes Bayesian model Gaussian random effect model. WHAT’S OUT THERE
15. DANIELS & HUGHES (1997)
16. 3 LEVELS BAYESIAN MODEL
17. 2 LEVELS VS. 3 LEVELS • • • • • • • • • • • • • • • • • • • • • • • • • −1.5 −1.0 −0.5 0.0 −1.5 −1.0 −0.5 0.0 log(RRARR ) log(RREDSS ) 2−Levels Surrogate Model 3−Levels Surrogate Model Predictive Distribution Posterior Distribution • • • • • • • • • • • • • • • • • • • • • • • • •
18. •Narrower confidence bands / Lower Variability of estimates. •Support to Decisions even with Limited Information. •Joint Modeling of the three outcomes. •Accounts for Measurement Errors. CONCLUSIONS
    
19. •Gaining Insight in Long Term Effects via Short Term Surrogates •“Friends like Wood’’ (hut/bonfire analogy) •Long Term Engagement FRIENDING MECHANISMS
    
20. DATA • 2.4M Users • 19 Test Groups: nested contrasts, promoting/demoting website features • Long Term Experiment • Common Control
21. OLS VS. 2SLS −1 0 1 2 3 30 35 40 45 50 55 OLS and IV fit make+accept change in friend count OLS IV promoted demoted
22. •Validated Surrogacy Relationship. •Improved understanding of friending experiment. •Activity Measures are more Complex. CONCLUSIONS
23. •General Overview in Ghosh et al. “Links between analysis of surrogate endpoints and endogeneity” Statistics in Medicine (2010). •With Great Power comes Small Effect Size. FINAL REMARKS
24. • RRMS: joint work with Dr. David Ohlssen and Dr. Heinz Schmidli, Novartis Pharma. • Friending joint work with Dr. Dean Eckles, Facebook Inc. • Special thanks to Dr. Suzanne Baker for her brain. AKNOWLEDGEMENTS
25. THANK YOU! Luca Pozzi poz.luc@gmail.com