The new National Radiology Data Registry, announced this week by the American College of Radiology, is capable of monitoring AI results and collecting a range of contextual information, such as patient data, clinical metadata and radiology report results.
It is intended to compare individual radiology practice results to aggregated national performance benchmarks from other sites using identical or similar products, ACR says.
WHY IT’S IMPORTANT
Clinical sites and AI developers can use ACRs Rate-AIthat will monitor the performance of a range of real-world imaging AI algorithms in clinical settings to obtain performance reports from the deployed AI, the organization said on Monday.
The ACR Data Science Institute will facilitate oversight of the data, which could also help developers improve future versions of algorithms, ACR said.
Radiology’s legacy systems were not built to support activities to ensure that algorithms used in clinical environments work as expected, said Dr. Christoph Wald, Vice Chairman of the ACR Board of Chancellors and Chairman of the ACR Commission on Informatics.
“Users of AI technology in radiological care must ensure that algorithms perform adequately in their local environment,” Wald said in a statement.
“As rising demand for imaging exceeds radiologists’ supply, AI is seen as an essential tool to bridge the gap and enable radiologists to maintain high standards of care while meeting increasing demand,” Dr. Woojin Kim, Chief Medical Officer of ACR DSI, added.
The ACR DSI-facilitated surveillance provides “a tangible, hands-on approach to addressing a challenge that radiologists increasingly face today,” Wald said.
Participating radiology facilities have access to:
- Monitoring the stability of algorithms over time, including imaging equipment, protocols and software version.
- Continuous monitoring of concordance/concordance of AI results with radiology reports.
- Aggregated observations in reports and dashboards.
THE BIG TREND
Innovation in radiology is happening rapidly and a number of clinical and financial AI use cases have proven their worth, but the long-term safety of algorithms is still an important consideration for ACR.
Scottsdale, Arizona-based SimonMed is one of many practices where AI tools have improved workflows. It says algorithms can render radiology reports about 82% faster than measurements without automation.
“In terms of improved diagnoses, the tools can be truly remarkable, so it is important to be open-minded and curious as this is a rapidly evolving field,” Dr. John Simon, CEO of SimonMed. Healthcare IT news in February.
Earlier in July, ACR launched the Recognized Center for Healthcare as a unique quality assurance program to benchmark radiology facilities using AI in their imaging workflows.
“Even an AI product approved by the U.S. Food and Drug Administration must be tested locally to ensure it works safely and as intended,” Wald warned in the quality assurance center’s announcement.
“Practice leaders must take precautions to maximize the benefits of AI products while minimizing the risks.”
ON THE RECORD
“Assess-AI will play a critical role in safely and effectively accelerating the clinical adoption of AI in radiology by ensuring AI products perform optimally in clinical environments so radiologists can focus on what matters most: providing high-quality care to their patients.” Kim said in a statement.
“Building on its commitment to the profession of radiology, the launch of Assess-AI is ACR’s latest step to empower radiologists to implement AI safely, effectively and transparently.”
Andrea Fox is editor-in-chief of Healthcare IT News.
Email: afox@himss.org
Healthcare IT News is a HIMSS Media publication.