Clinical researchers contemporaneously evaluate three general questions through the design, analysis, and interpretation of a study. First, is an effect real or should it be attributed to chance? Second, if it is real, how big is it? Third, is that magnitude enough to matter? The statistical significance of an effect, usually operationalized with a p-value, informs the first question. The effect size, which is usually some standardized metric of magnitude like Cohen’s d or Spearman’s r, is used to evaluate the second. For the third, it is essential to consider indices of practical significance, because even the tiniest effect can be made statistically significant with a large enough sample size. But metrics of practical significance do not always convey to clinicians, consumers, or even the researcher themselves information about real-world implications that is easily understood or interpreted. Among other scientific, cultural, and regulatory forces, recognition of this fact led to a push for more consideration of clinical meaningfulness in the late 1990s, but this has been especially focused on the reporting of clinical trials. Responses to this included the use of risk potency metrics (e.g., relative risk, number needed to treat) for communicating meaningfulness to clinicians, and the more purposeful inclusion of the patient voice in the drug development process, such as attempts to define the minimal clinically important difference. However, less attention has been paid to non-interventional clinical research, and under which circumstances it is necessary to understand and convey information about the clinical meaningfulness about individual effects, or how that information might be generated. This symposium will primarily be discussion-based, allowing the experiences and opinions of researchers from a range of settings to be presented and evaluated in a first step toward the creation of guidance for quantifying, evaluating, and conveying information about clinical meaningfulness of statistical effects in IDD research. The four panelists will introduce their backgrounds and viewpoints on how the concept of clinical meaningfulness applies to statistical results in their work, followed by a facilitated panel discussion. The topics will range from a specific statistical method to a big-picture assessment of how the clinical meaningfulness of individual analyses translates to whether and how results are implemented in practice. Dr. Aaron Kaat will describe impact analysis, a method for generating clinical meaningfulness from measurement invariance analyses. Dr. Caitlin Hudac will discuss approaches to addressing clinical meaningfulness in neuroscience research, and Dr. Natasha Ludwig will focus on neuropsychological research, with special attention to the utilization of qualitative methods and the patient voice. Finally, Dr. Leonard Abbeduto will discuss implementation science in IDD, focusing on how the evaluation and promotion of clinical meaningfulness in our analyses will help to translate theoretically oriented projects to ideas and results with real-world implications. We hope that attendees will leave with an invigorated understanding of the role of clinical meaningfulness in all types of research projects, and with some ideas for how to enhance and evaluate it in their own work.
Chair: Cristan Farmer, National Institute of Mental Health,
Second Chair: Audrey Thurm, National Institute of Mental Health,
Discussant: Cristan Farmer, National Institute of Mental Health
First Presentation: Statistical and Practical Considerations for Multi-Group Research
Aaron J Kaat, Northwestern University Feinberg School of Medicine,
Second Presentation: Beyond the Static: Addressing Clinical Meaning in the Context of the Developing, Maturing, and Dynamic Brain
Caitlin Hudac, University of South Carolina,
Third Presentation: Utilization of Qualitative Methods and Patient Partnerships to Inform Meaningful Measure Development in IDD
Natasha Ludwig, Kennedy Krieger Institute, Johns Hopkins School of Medicine
Fourth Presentation: How Do We Move from Theoretical or Research-Oriented Projects to Interventions or Ideas That Can Change the Lives of People with IDD in Meaningful Ways in the “Real World”?
Leonard Abbeduto, University of California, Davis,