Limited consensus in expert opinions on studies evaluating the design, conduct, analysis, or reporting of health research: a survey study
Silva, Jessyca Matos Lima, João Pedro O Lawson, Daeria Nyambi, Agatha Sadeghirad, Behnam Logie, Carmen Moher, David Pieper, Dawid Collins, Gary Guyatt, Gordon
Silva, Jessyca Matos
Lima, João Pedro
O Lawson, Daeria
Nyambi, Agatha
Sadeghirad, Behnam
Logie, Carmen
Moher, David
Pieper, Dawid
Collins, Gary
Guyatt, Gordon
Abstract
Background and objectives: Methodological studies critically evaluate how health research is designed, conducted, analyzed, and reported. Despite their growing importance, currently, there is no reporting tailored to this type of research, which hampers the visibility, reproducibility, and overall utility of the findings of methodological studies.
Methods: We administered a survey to researchers with expertise in designing and performing methodological studies to gather their opinions on appropriate terminology, how they should be categorized, and key reporting elements. Quantitative data were analyzed descriptively, with a content validity ratio applied to determine appropriateness. Qualitative survey responses were analyzed using inductive content analysis.
Result: Of 499 invited, a total of 119 participants completed the survey (response rate 23%). None of the 13 proposed nomenclatures met the threshold for appropriateness. Of the four proposed study categories, two (study aim and study design) were retained based on expert ratings. Among 23 proposed reporting items, 15 were endorsed for further evaluation. Qualitative responses were condensed and categorized, identifying the importance of flexibility in terminology and concerns about categorization.
Conclusion: There is substantial disagreement among experts regarding key aspects of methodological studies, particularly related to terminology. While some agreement was observed around study categories and reporting elements, diverse and sometimes conflicting perspectives underscore the complexity of standardizing methodological studies. These findings reinforce the need for a collaborative consensus process to develop reporting guidance that is both practical and adaptable to the nuances of this field.
MIDER Authors
Date
2026-03-20
Type
Article
Subject
Terminology as a topic, Research, Research design, Surveys and questionnaires, Terminology as a topic
Collections
Citation
Silva JM, Lima JP, O Lawson D, Nyambi A, Sadeghirad B, Logie C, Moher D, Pieper D, Collins G, Guyatt G, Van Spall H, Ariel Franco JV, Thabane L, Puljak L, Murad MH, Vanstone M, Tugwell P, Schandelmaier S, Welch V, Samaan Z, Brignardello-Petersen R, Darzi AJ, Mbuagbaw L. Limited consensus in expert opinions on studies evaluating the design, conduct, analysis, or reporting of health research: a survey study. J Clin Epidemiol. 2026 Mar 20;194:112245. doi: 10.1016/j.jclinepi.2026.112245. Epub ahead of print.
Journal / Source Title
Journal of Clinical Epidemiology
DOI
10.1016/j.jclinepi.2026.112245
PMID
41865805
Publisher
Elsevier
Publisher’s URL
https://www.sciencedirect.com/journal/journal-of-clinical-epidemiology