Robot learned surgical tasks from videos and AI
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Aïda Amer/Axios
A research team used videos of surgeries and the machine learning architecture behind ChatGPT to successfully train a robot to do basic tasks like manipulate a needle, lift body tissue and suture.
Why it matters: Showing a robot can perform with the skill of a doctor opens up new possibilities for devices like the widely used da Vinci surgical system and reduce the risk of medical errors, the Johns Hopkins-led scientists said.
Existing robotic systems can be imprecise and need to be taught step-by-step with code, meaning it could take a decade to develop a model to teach suturing for just one type of surgical procedure.
- The video-AI input trains the robot to do relative movements instead of specific actions. That not only speeds up training but makes the robot highly adaptable, so that if it drops a needle, it can automatically pick it up and continue, researchers said.
- "All we need is image input and then this AI system finds the right action," said lead researcher Ji Woong "Brian" Kim.
How it works: The model was fed hundreds of videos recorded from wrist cameras placed on the arms of da Vinci robots during surgical procedures that are normally used for post-operative analysis.
- The robot was then trained to perform basic procedures with kinematics, a language that breaks down the angles of robotic motion into math.
- The team, which also includes Stanford scientists, is now trying to train a robot to perform a full surgery.
- The system uses the same Transformer machine learning architecture behind ChatGPT. But ChatGPT works with words and text, not kinematics.
Context: Nearly 7,000 da Vinci robots are used worldwide, and more than 50,000 surgeons are trained on the system, researchers noted.
- Robotic systems consist of automated arms to hold instruments, high-definition cameras and a console operated by a surgeon. Specialists use them for procedures such as appendectomies, hernia repairs and hysterectomies, per the Mayo Clinic.
- While they can deliver benefits like less post-surgical pain and lowered risk of infection, they're only available in centers that have specially trained surgeons.
