Methodological review reveals essential gaps and inconsistencies in clinical claims, effects, and outcomes in HTA reviews of diagnostic tests
Abstract
Background and objective: Essential first steps in performing a health technology assessment (HTA) for a diagnostic test include consideration of the clinical pathway in which the test will be used, specifying the clinical claim for the test (how the test may add benefit, introduce harm, or have other disadvantages beyond impact on the individual patient) and specifying the outcomes that would need to be measured to assess whether the test achieves its aims. We aimed to examine how a test evaluation framework (TEF) outlining the intended and unintended effects of tests could support the HTA process and to identify additional ways in which tests add benefit or introduce harm.
Method: We included 45 HTAs reporting 50 review questions. The study focused on HTA reports with a full English-language evidence review, clear methods and results sections, and evaluations of a single testing strategy or technology type. We looked for mechanisms of effect included in, and additional to, a TEF previously published by our group.
Results: The clinical pathway and positioning of the new test were described in 98% of review questions (49/50) and illustrated in 62% (31/50). The test's clinical claims were easily identifiable in 56% (28/50). Claims, mechanisms of effect, and prespecified outcomes were frequently not coherent. For instance, at least one constituent test effect mechanism (mainly timing- and confidence-related mechanisms) could not be linked to prespecified outcomes in 54% of reviews. Most HTAs (41, 82%) listed outcomes to be evaluated in the evidence reviews that we were unable to link to the claims for the tests (acceptability of the test, test failures, accuracy, therapeutic yield, and effectiveness). Four mechanisms of effect additional to those in the existing TEF were identified: two concerned impact on individuals beyond the person being tested and two concerned organizational impact.
Conclusion: Important gaps and inconsistencies in the reporting of test claims and associated outcomes in HTA reviews risk incomplete appraisal of a test's impact on patients and the health-care system. We recommend that tools are developed to support and standardize this complex process. This could be facilitated by tools in development and an expanded TEF.
Plain language summary: Health technology assessments (HTAs) are reports that assess whether medical tests, technologies, or treatments are worth using. HTA reports consider both effectiveness (how well something works) and costs. Tests can affect patients in many ways. It is important to understand the accuracy of a test as well as other impacts it may have on an individual's health and care. An important first step for an HTA of a test is to describe how the new test fits into regular medical care. A second step is to describe how the test could benefit, or harm, patients compared to regular care. HTA authors can then make sure that the result of using the test is measured using the right outcomes. We looked at 50 HTAs of medical tests that were written in English. We found that almost all reviews (98%) explained where and when the test would be used in patient care. About two-thirds of them (62%) included diagrams to show this. Only half clearly stated how the test could benefit, or harm, patients. There was often a disconnection between the expected benefits and harms from use of the test (clinical claim for the test) and the effects from the test that the authors of the review set out to assess. This could lead to the wrong conclusions about how well the test works and whether it is worth the cost. We also described broader effects of tests on health-care systems and society. A more structured approach to help describe how a new test fits into regular medical care and to identify what a test claims to do is needed. This would help to ensure that all important outcomes are measured.
Date
2025-11-02
Type
Article
Subject
Critical pathways, Delivery of health care, Diagnostic techniques and procedures
Collections
Citation
Dinnes J, Davenport C, Harris IM, Ferrante di Ruffano L, Mallett S, Takwoingi Y, Deeks JJ, Hyde C. Methodological review reveals essential gaps and inconsistencies in clinical claims, effects, and outcomes in HTA reviews of diagnostic tests. J Clin Epidemiol. 2026 Jan;189:112040. doi: 10.1016/j.jclinepi.2025.112040. Epub 2025 Nov 2.
Journal / Source Title
Journal of Clinical Epidemiology
DOI
10.1016/j.jclinepi.2025.112040
PMID
41187895
Publisher
Elsevier
Publisher’s URL
https://www.sciencedirect.com/journal/journal-of-clinical-epidemiology