David Dutwin & Ipek Bilgen - Everything You Need to Know When Utilizing Probability Panels: Best Practices in Planning, Fielding, and Analysis - September 27, 2023

 

Speakers: David Dutwin & Ipek Bilgen

 

Probability-based panel survey research is more widespread than ever, as the continuing decline in survey response rates makes cross-sectional sample surveys less and less accessible both in terms of fit for purpose data quality and cost. The attraction of probability panels for surveys is their ability to attain, dependent upon their recruiting methods, comparable response rates to cross-section polls, but at a lower cost and more expeditious execution. Panels are a unique type of survey research platform: Unlike cross-sections, panels recruit respondents specifically for future participation in surveys.  In return, panelists are financially compensated, typically to join the panel in the first place, and then secondarily for each survey in which they participate.

 

These differences to cross-sectional surveys have a range of potential implications. How does the method and effort of recruiting impact who joins, and as a consequence what is best practice? What do panels do to retain panelists over time and which strategies are more successful than others? How much of a concern is panel conditioning, that is, the impact of persons repetitively taking surveys over time, and what are the implications for how frequently panelists should take surveys? How do panels, which exclusively request that panelists take surveys on the Internet, deal with people who do not have or are not comfortable using the Internet? What is the impact of panelist attrition and what are best efforts to replenish retired panelists? How successful are panels are executing true longitudinal surveys? And, given the additional layers of complexity, how are panel surveys properly weighted and estimated?

 

This seminar is meant to serve two purposes. First, it will serve as a guide for consumers of probability-based panels to understand what, in short, they are working with: What questions to ask and what features to understand about probability panels in evaluating their use for data collections, and how to best use probability-based panel data. Second, it will serve as an exploration of best practices for the practitioners of surveys: Raising issues of data quality, cost, and execution.

 

 

1. For consumers of panel data: Understanding the features of panels with which to be knowledgeable; to know the important questions to ask panel vendors when assessing their fit for purpose of your research.

2. For researchers and practitioners: To understand the many dimensions and decision points in the building, maintenance, deployment, and delivery of multi-client panels and panel data.

 

 

David Dutwin, PhD, is Senior Vice President for Strategic Initiatives, Business Ventures and Initiatives and Chief Scientist of AmeriSpeak at NORC at the University of Chicago. David provides scientific and programmatic thought leadership in support of NORC’s ongoing innovations. In addition to identifying new business opportunities, he lends expertise on research design conceptualization, methodological innovation, and product development. He leads the panel operations and the statistics and methods divisions of AmeriSpeak. David assists in NORC strategic vision and strategy, project acquisition and management of advance research methods. Prior research has focused on election methodology, surveying of low-incidence populations, the use of big data in survey research, and data quality in survey panels. He is a senior fellow of the Program for Opinion Research and Election Studies at the University of Pennsylvania. An avid member of the AAPOR community, David served as president from 2018-2019. He previously served on AAPOR’s Executive Council as conference chair and has served full terms on several committees. For over twenty years, he has taught courses in survey research and design, political polling, research methods, rhetorical theory, media effects, and other courses as an adjunct professor at the University of Pennsylvania, the University of Arizona, and West Chester University.

 

Ipek Bilgen, PhD, is a Principal Research Methodologist in the Methodology and Quantitative Social Sciences Department at NORC at the University of Chicago. Ipek is the Deputy Director of NORC’s Center for Panel Survey Sciences. Additionally, she oversees AmeriSpeak’s methodological research and innovations. As part of her role within AmeriSpeak, she also provides survey design expertise, questionnaire development and review support, and leads cognitive interview and usability testing efforts for client studies. Ipek received both her Ph.D. and M.S. from the Survey Research and Methodology (SRAM) Program at the University of Nebraska-Lincoln. She has published and co-authored articles in Journal of Official Statistics, Public Opinion Quarterly, Journal of Survey Statistics and Methodology, Survey Practice, Social Currents, Social Science Computer Review, Field Methods, Journal of Quantitative Methods, SAGE Research Methods, and Quality and Quantity on issues related to interviewing methodology, web surveys, online panels, internet sampling and recruitment approaches, nonresponse and measurement issues in surveys. In the past, she has served on AAPOR’s and MAPOR’s Executive Councils. Ipek is currently teaching at the Irving B. Harris Graduate School of Public Policy Studies at the University of Chicago and serving as Associate Editor of Public Opinion Quarterly (POQ).