Rady Children’s Introduces Free 3D Imaging Software, Available to Other Providers
Three-dimensional reconstruction and modeling technology is not new to hospitals. In fact, many organizations have dedicated radiology technologists who perform 3D reconstructions of cross-sectional images.
THE PROBLEM
The results of their work are typically stored in PACS, essentially as a video of predetermined angles – often referred to as a rotational cine or turntable cine. While these “videos” are accessible to surgeons, the prescribed display and textures are a barrier to a full understanding of the patient’s morphology.
“If surgeons could use the full PACS suite, they could create their own views. However, there is a significant user difference between the radiology-focused software and the surgical or interventional user,” said Justin Ryan, director and investigator in the Webster Foundation 3D Innovations Lab at Rady Children’s Hospital in San Diego.
Ryan, who is also an associate professor in the department of neurosurgery at the University of California, San Diego, has a doctorate in biomedical engineering.
“Simply put, surgeons are typically not trained to make the most of the 3D data available by using all the radiology software,” he added.
This challenge can best be described by a recent case, he continued.
“A cross-sectional CT scan was performed on a patient with a double outlet right ventricular anatomy and an abnormal airway,” he recalls. “As per our institution’s standard of care, the CT was given a 3D reconstruction in radiology post-processing software. Two cines were produced with the 3D reconstructions: a spin and a tumble.
“The surgeon gave feedback that they wanted the same spin/tumble with the airway included; then another request came in to make the aorta partially transparent, followed by additional transparent requests and unique angles,” he continued. “Our team had produced over 12 cines in an attempt to accommodate the surgeon’s requests – where a holistic solution would have allowed the surgeon to do all of these processes in real time.”
PROPOSAL
Ryan looked at other interactive domains, particularly video game development, and saw that there is a focused effort to create an easy-to-use (user interface) but effective (user experience) application. This theory is known as UIUX.
“Our team wanted to leverage real-time 3D rendering that exists in the video game development domain, as well as UIUX principles to create a solution for surgeons and interventionalists as the primary users, not radiologists,” he noted. “We were fortunate to be able to recruit a team member who previously worked for a video game company. We started an ideation process between our surgeons, interventionalists, and this innovation engineer to decide to use the Unity platform to visualize anatomy in real time.
“Our team wanted to leverage real-time 3D rendering, which is common in video game development, and UIUX principles to create a solution primarily intended for surgeons and interventionalists, rather than radiologists.”
Justin Ryan, Rady Children’s Hospital
“In collaborating with our stakeholders, we realized that the primary goal was to visualize 3D structures with as few obstructions as possible,” he continued. “To achieve this goal, our team implemented periodic focus group and pilot testing. We gave a simple prompt such as ‘open the heart model on the desktop, then hide the aorta and make the airway partially transparent’ without any additional instructions.”
The team would watch the user and take notes on every stumbling block. Maybe the iconography wasn’t ideal or the implemented method – while logical to the engineers on the team – ultimately didn’t make sense to the intended user.
“These stumbling blocks would have pushed our development team back to the drawing board,” Ryan said. “This constant feedback from the intended user base during development was a significant benefit to our process. While medical software companies may have a medical advisory board, we had the benefit of constant feedback from our team members.”
The result is Arc 3D Model Viewera 3D imaging platform specifically designed for assessing 3D medical reconstructions.
TOOK UP THE CHALLENGE
Ryan and the team eventually started using the Unity platform for real-time visualization. Many people may be familiar with popular video games such as Pokémon Go, Marvel Snap, and Among Us. These games are built on Unity.
“Readers may not be aware that Unity also has non-video game applications in the entertainment, education, and defense industries, as these industries also have a need for real-time visualization,” Ryan noted. “The Unity platform allowed us to create a solution where surgeons could fully interact with 3D datasets.
“We designed Arc 3D Model Viewer to fill the niche of real-time 3D viewing, not segmentation and reconstruction,” he continued. “Our process is to use FDA-cleared segmentation software – we use Mimics from Materialise, but other platforms exist – to do the segmentation and 3D reconstruction. We output the final 3D model that can then be imported into Arc 3D Model.”
Throughout the development process, staff worked continuously with their information security team, as the ultimate goal was to deploy the software to the hospital’s computers.
“When we completed a release candidate application, we gave our information security team the code to review for vulnerabilities; ultimately, an outside information security firm was hired to provide additional review,” Ryan said. “With the acceptance of our software, it was then able to be deployed to hospital PCs.
“Now our users can hide and show structures or virtually cut through structures in real time; this would not be possible with the status quo cines in PACS,” he added. “To further simplify the process, our engineers are building in DICOM query and retrieve capabilities.”
RESULTS
According to Ryan, the team’s greatest achievement is taking the step from idea to implementation within the organization.
“Walking into a clinic and seeing Arc 3D Model Viewer being used on the computer to explain a patient’s condition to a family has been personally rewarding,” he said. “Now we’re spinning off branches of our software for education and patient education. After implementing our solution, we’ve seen an increase in 3D requests, which I see as a testament to the UIUX promise of the software.”
“We’ve been doing 3D reconstructions at the point of care for years, but the volume increased dramatically after we implemented our solution,” he noted. “When we made the non-diagnostic-use software public, it was encouraging to see the number of downloads increase, but in ways we didn’t expect.”
Nearly half of the downloads come from academic or medical organizations outside the United States. It’s rewarding, Ryan said, to see 3D applications reaching far beyond the patients of Rady Children’s Hospital.
FINDING THE FREE TOOL
“We have an academic and technical mindset,” Ryan concluded. “We encourage people to reach out to us if they have a barrier to adoption or an idea about how to visualize 3D in new ways. We’re eager to build partnerships to give more people more control over their 3D data.”
To get the Arc 3D Model Viewer, click hereTo contact Ryan and his team, write to 3DPrintLab@rchsd.org.
Follow Bill’s HIT reporting on LinkedIn: Bill Siwicki
Send him an email: bsiwicki@himss.org
Healthcare IT News is a publication of HIMSS Media.