PHS Monday Seminar: Melody Goodman, PhD- Development & Validation of the Research Engagement Survey Tool (REST)

Associate Dean for Research Associate Professor of Biostatistics School of Global Public Health New York University Learning objectives:

1. Discuss measurement of stakeholder engagement in research
2. Assess the utility of a quantitative measure of stakeholder engagement
3. Evaluate the psychometric properties of a quantitative measure to assess stakeholder engagement in research, the Research Engagement Survey Tool (REST)

Abstract: Background: Stakeholder engagement is a crucial part of participatory public health research, yet the measurement of stakeholder engagement in research is varied, inconsistent, and not methodologically sound. As the level of stakeholder engagement across studies can vary greatly from minimal engagement to fully collaborative partnerships, there is a need for a comprehensively validated quantitative measure of stakeholder engagement in research. Methods: We use stakeholder-engaged research approaches and a mixed-methods (qualitative/quantitative) study design to validate a measure to assess the level of stakeholder engagement in research, the research engagement survey tool (REST). Through a modified Delphi process & cognitive interviewing, REST, previously developed building on existing literature, was modified several times with a focus on shortening the measure and validating the content of the measure. Concurrently, participants who were involved in stakeholder-engaged research were recruited and asked to complete a series of 4 web-based surveys that presented the modified versions of REST and other measures of stakeholder engagement to examine correlative validity. REST items are asked on both a quality (poor to excellent) and quantity (never to always) scale. To reduce partner burden, we developed a condensed version of REST. Subsequently, we identified items in the condensed REST showing differential functioning by participant demographic characteristics (race, gender, education) and created a computer-adapted version of REST to address this. The computer-adapted REST was pilot tested among partner groups of the research team. Finally, the comprehensive version of REST was implemented in practice among 20 partner/stakeholder engaged projects to test how REST functions in practice. Results: The resulting comprehensive version of REST was narrowed to 32 items corresponding to eight engagement principles. Psychometric results show acceptable internal consistency for the revised comprehensive measure’s eight engagement principles, both on the quality and quantity scale (Cronbach’s Alpha from 0.79 to 0.92). Results also show low to moderate correlative validity with similar measures (Spearman correlation coefficient range: 0.19 to 0.69). Results show promise for a 9-item condensed version of REST that is highly correlated with the comprehensive version of REST (quality & quantity: r=0.97, p<0.01) and has similar overall and engagement principle specific means. Preliminary results suggest REST can be implemented in practice among research teams. Discussion: Emerging data suggest REST is a valid and reliable measure that could potentially assess associations between research outcomes and stakeholder engagement. A 9-item condensed version of REST shows potential to decrease partner burden when measuring stakeholder engagement. Melody Goodman CV