COS are valuable, but methodological evidence can improve robustness
Beune IM*, Ganzevoort W†, Gordijn SJ*
*Department of Obstetrics and Gynecology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
†Department of Obstetrics and Gynecology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands.
Correspondence: Irene M. Beune, irenebeune@hotmail.com
A core outcome set (COS) is the agreed minimum set of outcomes to be measured in studies regarding a specific topic. A COS is considered to encompass the most relevant outcomes and does not restrict researchers. One should realize that outcomes not included in the COS may actually be important for specific research questions and different study-designs.
The COMET handbook (Core Outcome Measures in Effectiveness Trials), used in the current study (Duffy et al. BJOG 2020 xxxx), describes consensus methodology for COS development. (Williamson et al. Trials 2017, 18(Suppl 3):280). In a nutshell, it is advised to start with a systematic review to identify all possible outcomes; then use the Delphi strategy to converge opinions to consensus; and finally, the prioritized list of outcomes is discussed in a face-to-face consensus meeting in which the final COS is conducted. Published underlying evidence for this approach is limited and alternatives may be considered.
1. It remains unknown whether a systematic review, is preferable over a more liberal ‘scoping review’. A scoping review may cover all important outcomes and chances are low that an outcome that requires asystematic review to identify it, is considered to be fundamental for all research in the field.
2. COMET advises a response rate of 80% for each stakeholder group, but there is no frame of reference to establish what attrition rate is acceptable to avoid losing the strength of the panel. Did the drop-out of 37% of the total group in the Delphi rounds in this study have a significant effect on the final COS?
3. The crucial contribution of lay-experts is recognized by COMET, but there is no advise for the number or percentage of lay-experts in a panel. In previous COS procedures, lay-experts contribution varied from 4-50% (Williamson et al. Trials 2017, 18(Suppl 3):280).
4. A consensus-meeting facilitates acceleration of the consensus building procedure because the panel members are in direct contact and clarifications are readily available. The consensus meeting has inherent pitfalls.
Delphi and COMET methodology are valid and valuable tool for consensus building, particularly because a COS is never (only) a gold standard. Since there is also no gold standard of the methodology, it remains pivotal to appreciate the strengths and vulnerabilities of the methodology by doing studies that strengthen the COMET and Delphi methodology.
No disclosures: Completed disclosure of interest forms are available to view online as supporting information.