Atlas Meditech uses A.I. and V.R. to create simulated brains with haptic feedback.
Dell has submitted this post.
Just as an athlete trains for a game or an actor rehearses for a performance, surgeons prepare themselves ahead of an operation.
Atlas Meditech is letting brain surgeons experience a new level of realism in their pre-surgery preparation by adopting tools — including the MONAI medical imaging framework and NVIDIA Omniverse 3D development platform — to build artificial intelligence (AI)-powered decision support and high-fidelity surgery rehearsal platforms.
Its mission: improving surgical outcomes and patient safety.
“The Atlas provides a collection of multimedia tools for brain surgeons, allowing them to mentally rehearse an operation the night before a real surgery,” explains Dr. Aaron Cohen-Gadol, founder of Atlas Meditech and its nonprofit counterpart, Neurosurgical Atlas. “With accelerated computing and digital twins, we want to transform this mental rehearsal into a highly realistic rehearsal in simulation.”
Neurosurgical Atlas currently offers case studies, surgical videos and 3D models of the brain to more than a million online users. Dr. Cohen-Gadol, also a professor of neurological surgery at Indiana University School of Medicine, estimates that more than 90 percent of brain surgery training programs in the U.S. — as well as tens of thousands of neurosurgeons in other countries — use the Atlas as a key resource during residency.
Atlas Meditech’s Pathfinder software integrates AI algorithms that can suggest safe surgical pathways for experts to navigate through the brain to reach a lesion.
With NVIDIA Omniverse, a platform for connecting and building custom 3D pipelines and metaverse applications, the team aims to create custom virtual representations of individual patients’ brains for surgery rehearsals.
Custom 3D Models of Human Brains
A key benefit of Atlas Meditech’s advanced simulations — either onscreen or in immersive virtual reality (VR)— is the ability to customize the simulations, so that surgeons can practice on a virtual brain that matches the patient’s in size, shape and lesion position.
“Every patient’s anatomy is a little different,” says Dr. Cohen-Gadol. “What we can do now with physics and advanced graphics is create a patient-specific model of the brain and work with it to see and virtually operate on a tumor. The accuracy of the physical properties helps to recreate the experience we have in the real world during an operation.”
To create digital twins of patients’ brains, the Atlas Pathfinder tool has adopted MONAI Label, a tool for radiologists that automatically annotates MRI and CT scans to segment normal structures and tumors.
“MONAI Label is the gateway to any healthcare project because it provides us with the opportunity to segment critical structures and protect them,” explains Dr. Cohen-Gadol. “For the Atlas, we’re training MONAI Label to act as the eyes of the surgeon, highlighting what is a normal vessel and what is a tumor in an individual patient’s scan.”
With a segmented view of a patient’s brain, Atlas Pathfinder can adjust its 3D brain model to morph to the patient’s specific anatomy, capturing how the tumor deforms the normal structure of their brain tissue.
Based on the visualization — which radiologists and surgeons can modify to improve the precision — Atlas Pathfinder will suggest the safest surgical approaches to access and remove a tumor without harming other parts of the brain. Each approach links out to the Atlas website, which includes a written tutorial of the operative plan.
“AI-powered decision support can make a big difference in navigating a highly complex 3D structure where every millimeter is critical,” Dr. Cohen-Gadol says.
Realistic Rehearsal Environments for Practicing Surgeons
Atlas Meditech is using NVIDIA Omniverse to develop a virtual operating room that gives surgeons a realistic environment in which to rehearse their upcoming procedures. They can even modify how the patient and equipment are positioned in the simulation.
Using a VR headset, surgeons will be able to work within this virtual environment, going step by step through the procedure and receiving feedback on how closely they are adhering to the target pathway to reach the tumor. AI algorithms can be used to predict how brain tissue would shift as a surgeon uses medical instruments during the operation, and apply that estimated shift to the simulated brain.
“The power to enable surgeons to enter a virtual, 3D space, cut a piece of the skull and rehearse the operation with a simulated brain that has very similar physical properties to the patient would be tremendous,” says Dr. Cohen-Gadol.
To improve simulations of the brain’s physical properties, the team adopted NVIDIA PhysX, an advanced real-time physics simulation engine that is part of NVIDIA Omniverse. This enabled them to experiment with adding haptic feedback to the virtual environment, mimicking the feeling of working with brain tissue.
Envisioning AI, Robotics in the Future of Surgery Training
Dr. Cohen-Gadol believes that, in the coming years, AI models will be able to enhance surgery further by providing additional insights during a procedure. Examples include warning surgeons about critical brain structures that are adjacent to the area they’re working in, tracking medical instruments during surgery and providing guides to next steps in the surgery.
Atlas Meditech plans to explore the NVIDIA Holoscan platform for streaming AI applications to power these real-time, intraoperative insights. Applying AI analysis to a surgeon’s actions during a procedure can provide the surgeon with useful feedback to improve their technique.
In addition to being used for surgeons to rehearse operations, Dr. Cohen-Gadol says that digital twins of the brain and of the operating room could help train intelligent medical instruments, such as microscope robots using Isaac Sim, a robotics simulation application developed on Omniverse.
