Tackling the challenges of test uncertainty
26 June 2018
As the field of precision medicine gains momentum, there is a requirement for increasingly sophisticated diagnostic tests to inform treatment decisions. However, difficulties in producing evidence around the long-term impact of tests on patient outcomes and healthcare resources means there is a danger that standards are being lowered for these kinds of technologies.
One area of concern relates to the measurement performance of tests i.e. how well a test identifies the target substance of interest in a test sample. A range of factors can affect the measurement performance of tests – from how a blood or tissue sample is taken to how that sample is stored and processed within the laboratory. This introduces uncertainty around the observed test result (i.e. measurement uncertainty), which can have a significant impact on patient outcomes if test results are incorrectly observed as lying above or below cut-off values used to determine diagnoses.
These issues were highlighted by a team of NIHR Leeds MIC researchers at the Health Technology Assessment International (HTAi) Annual Meeting, held in Vancouver (Canada) in June.
“When developing diagnostic tests, it’s much more difficult to get the rigorous evidence that we’ve come to expect in drug development,” says Ms Alison Smith, an NIHR Doctoral Fellow from the University of Leeds Academic Unit of Health Economics (AUHE). “We want to highlight the particular impact that measurement uncertainty can have and, by improving understanding in this area, help researchers find ways of overcoming uncertainty.”
The researchers discussed three different areas: uncertainty in the laboratory; uncertainty in diagnostic accuracy; and uncertainty in patient preferences.
Alison presented work on laboratory uncertainty, highlighting the range of circumstances that could influence a test outcome, from the physical state of the patient at the time of the test, to the laboratory environment in which the test is processed.
“Traditionally within heath technology assessment this type of uncertainty hasn’t been considered,” says Alison. “People think that uncertainty in the lab is somehow contained within the lab and that laboratory scientists deal with it – but we know that’s not true. There’s lots of literature showing that uncertainty in the lab can result in misdiagnoses.”
Dr Beth Shinkins, Health Economics Theme Lead at NIHR Leeds MIC, presented work on uncertainty in diagnostic accuracy and health outcomes, demonstrating statistical methods for evaluating diagnostic uncertainty. Dr Dean Regier, an Assistant Professor in the School of Population and Public Health at the University of British Columbia, also presented work on patient preferences, outlining the different impacts that diagnostic uncertainty and information can have on patients and families.
“If we can better understand these uncertainties and their likely impact, we can better understand the true value of tests,” says Alison. “By working together with laboratory scientists the Health Technology Assessment community could further help here, by providing the right sort of information needed to establish laboratory test performance specifications based on patient outcome and cost-effectiveness data.”