Robotic AI performs successful operations after watching videos for training
Watch old episodes of ER You won’t become a doctor yet, but watching videos may be the only training a robot surgeon’s AI brain needs to sew you up after a procedure. Researchers from Johns Hopkins University and Stanford University have published a new paper showing that a surgical robot is as capable as a human at performing certain procedures after simply watching humans do them.
The research team tested their idea with the popular da Vinci Surgical System, which is commonly used for non-invasive surgery. Programming robots usually requires you to manually input every move you want them to make. The researchers got around this using imitation learning, a technique that implanted human-level surgical skills into the robots by letting them observe how humans do it.
The researchers put together hundreds of videos recorded with wrist-mounted cameras showing human doctors performing three specific tasks: needle manipulation, tissue lifting and suturing. The researchers essentially used the kind of training that ChatGPT and other AI models use, but instead of text, the model absorbed information about the way human hands and the tools they hold move. This kinematic data essentially converts motion into math that the model can apply to perform the procedures on demand. After watching the videos, the AI model was able to use the da Vinci platform to mimic the same techniques. It’s not much different from the way Google is experimenting with teaching AI-powered robots to navigate spaces and complete tasks by showing them videos.
Surgical AI
“It’s truly magical to have this model and all we do is give it camera input and it can predict the robot movements needed for operations. We believe this represents a significant step forward towards a new frontier in medical robotics,” senior author and JHU assistant Professor Axel Krieger said in a edition. “The model is so good at learning things we haven’t taught him. If he drops the needle, he automatically picks it up and moves on. I didn’t teach him this.”
The idea of an AI-powered robot holding blades and needles around your body may sound scary, but the precision of machines could make them better than human doctors in some cases. Robotic surgery is becoming increasingly common in some cases. A robot that independently performs complex procedures could even be safer, with fewer medical errors. Human doctors could have more time and energy to focus on unexpected complications and the more difficult parts of surgery that machines can’t yet handle.
The researchers plan to use the same techniques to teach an AI how to perform a full operation. They’re not the only ones pursuing the idea of AI-enabled robotic healthcare. Earlier this year, AI dental technology developer Perceptive demonstrated the success of an AI-guided robot that performs a dental procedure on a human without supervision.