Dr. Hans von Lücken, Senior Physician for Head and Neck surgery (ENT) at the Catholic Marienkrankenhaus in Hamburg, can now perform complex surgery using the Virtual Surgery Instructor (VSI). This intelligent programme converts CT and MRI scans into a 3D model that can be projected directly onto a patient using mixed reality glasses called the Microsoft HoloLens.
Doctor and software developer revolutionizing surgery - Apoqlar's digital models pass tests
The Pokémon Go craze in 2016 augured well for augmented reality, which adds virtual, digital content to everyday reality. Meanwhile, awareness of the technology has risen from a mere buzzword. Commerce and industry are seizing the opportunities offered by augmented reality. “Around 82 per cent of companies launching AR or VR in their businesses are having good experiences. And 46 per cent and 38 per cent respectively expect such technologies to become standard in three to five years, according to the survey “Augmented and Virtual Reality in Operations” published last September by Capgemini consultancy. A total of 700 managers in the automotive, manufacturing and utilities industries in Germany, France, the United States, China, Britain and Scandinavia, who are working on AR and VR projects, were interviewed for the survey. More and more medical professionals are also discovering combined real and virtual information.
3D models of CT and MRI scans thanks to VSI
Voice-controlled visualization on demand
“The 3D images are translucent, so I can still see the reality underneath. The software recognizes the patient and the 3D image is adapted exactly to the shape of the head. Of course, I can also simply move the images out of my field of vision – and this is all done per voice or gesture control,” von Lücken explained. The 3D images can be rotated or enlarged during the operation and supplemented with more information such as laboratory results, X-ray or ultrasound images. In this way, von Lücken can look into a patient’s head and get a close-up, precise view of all the anatomical structures.
Hamburg-based Apoqlar
Von Lücken and Sirko Pelzl, who founded the Apoqlar start-up in 2017, developed VSI jointly. MRI and CT scans are holographically displayed on the Microsoft Hololenses. Last year, Apoqlar became a partner to Microsoft. The start-up will soon be able to use the upcoming generation of smaller, lighter and more powerful HoloLenses, which will yield far better 3D images.
Greater safety thanks to digital models
“Mixed reality will revolutionize the possibilities of operations. It’s not a question of whether this will happen, but whether we will lead this revolution,” said Pelzl, 41. Meanwhile, VSI has proven itself during a one-year practical trial in Marienkrankenhaus. “On August 28, the world’s first operation with VSI took place,” said von Lücken, 54. Since then, he has performed 12 virtually-assisted operations on the maxillary sinus. Although there are other applications for virtual navigation in medicine, “VSI is far more elegant. The representation of the finest anatomical structures ensures the greatest possible safety during the operation,” he noted. Virtually assisted surgery is here to stay, he believes. “And not just in ENT. VSI can also be used e.g. in neurosurgery, accident surgery or orthopaedics,” von Lücken added.
VSI – from Hamburg to hospitals all over the globe
Hospitals in Germany and abroad have already expressed interest. “In addition to Marienkrankenhaus, Essen University Hospital and a clinic in Calgary, Canada, and one in Valencia, Spain, have already purchased the system,” said Pelzl. VSI is currently undergoing practical tests in the Asklepios clinics in Barmbek and Altona. Tests are planned at another ten hospitals. Last year, Pelzl presented VSI in Singapore, Auckland and London. Trips to India and Thailand are planned for 2019. Pelzl is currently trying to balance sales and further develop the system.
Natural rendering brings colour to 3D images
“We have just hired a new employee for sales so that I can devote more time to further development,” said Pelzl. But that also means travelling to learn more about doctors’ needs. Von Lücken wanted nerves and vascular structures to be shown in different colours. “We can now offer this in a completely new way using Natural Rendering,” said Pelzl. Natural Rendering is a new visualization method in which black-and-white MRI and CT images are presented in colour, so that even the finest tissue structures are easier to recognize and appear vivid. Doctors are not the only experts benefitting from improved visualization. Last year, VSI was used to explain medical conditions to patients in Marienkrankenhaus. “We can use 3D models to better explain their illness to patients and to tell them exactly what will happen during the operation. This is extremely helpful,” von Lücken stressed.
ys/pb