Michael Elliott - Using Synergies Between Survey Statistics and Causal Inference to Improve Transportability of Clinical Trials - JPSM MPSDS Seminar

Michael Elliott is a Professor of Biostatistics and a Research Professor at the Institute for Social Research. He received his Ph.D. in biostatistics in 1999 from the University of Michigan. Prior to joining the University of Michigan in 2005, he held an appointment as an Assistant Professor at the Department of Biostatistics and Epidemiology at the University of Pennsylvania School of Medicine. Dr. Elliott's research interests include the design and analysis of sample surveys, causal inference, and missing and latent variable data structures.

Using Synergies Between Survey Statistics and Causal Inference to Improve Transportability of Clinical Trials

Presented During:  CS008 - Contributed: Experimental and non-experimental tools to inform healthcare policymaking

Medical researchers have understood for many years that treatment effect estimates obtained from a randomized clinical trial (RCT) -- termed efficacy'' -- can differ from those obtained in a general population -- termed effectiveness''. Only in the past decade has extensive work begun in the statistical literature to bridge this gap using formal quantitative methods. As noted by Rod Little in a letter to the editor in the New Yorker ...randomization in randomized clinical trials concerns the allocation of the treatment, not the selection of individuals for the study. The latter can have an important impact on the average size of a treatment effect,'' with RCT samples often designed, sometimes explicitly, to be more likely to include individuals for whom the treatment may be more effective.

This issue has been various termed generalizability'' or transportability." Why do we care about transportability? In RCTs we are in the happy situation were treatment assignment is randomized, so confounding due to either observed or unobserved (pre-treatment) covariates is not an issue. But while randomization of treatment eliminates the effect of unobserved confounders, at least net of non-compliance, it does not eliminate the effect of unobserved effect modifiers, which can impact the causal effect of treatment in a population that differs from the RCT sample population. The impact of these interactions on the marginal effect of treatment thus can differ between the RCT population and the final population of interest.

Concurrent with research into transportability has been research into making population inference from non-probability samples. There is a close overlap between these two approaches, particularly with respect to the non-probability inference methods that rely on information from a relevant probability sample of the target population to reduce selection bias effects. When there are relevant censuses or probability samples of the target patient population of interest, these methods can be adapted to transport information from the RCT to the patient population. Because the RCT setting focuses on causal inference, this adaptation involves extensions to estimate counterfactuals. Thus approaches that treat population inference as a missing data problem are a natural fit to connect these two strands of methodological innovation.

In particular, we propose to extend a pseudo-weighting'' methodology from other non-probability settings to a doubly robust'' estimator that treats sampling probabilities or weights as regression covariates to achieve consistent estimation of population quantities. We explore our proposed approach and compare with some standard existing methods in a simulation study to assess the effectiveness of the approach under differing degrees of selection bias and model misspecification, and compare it with results obtained using the RT data only and with existing methods that use inverse probability weights. We apply it to a study of pulmonary artery catheterization in critically ill patients where we believe differences between the trial sample and the larger population might impact overall estimates of treatment effects.