During the project, the human intelligence of healthcare professionals will be combined with artificial intelligence (AI) and mixed reality (MR). AI-supported virtual 3D avatars of medical professionals will be generated, which can be displayed on mobile devices, 3D headsets or other MR applications. The avatars will use the sensors to record their environment as well as patient data such as blood pressure, blood glucose levels or weight in real time. Unlike previous voice assistants, the 3D avatars will be able to speak and show facial expressions and gestures to evoke familiarity among participants.
The AI-controlled avatars will be able to perform simple, routine tasks themselves. Medical experts will monitor the work of the virtual assistant later. If a patient wishes to speak to a doctor or nurse, they can arrange a real or virtual appointment through the assistant. "Patients can interact seamlessly with intelligent virtual assistants and avatars of doctors, therapists and nursing staff," said Prof. Dr. Frank Steinicke, a computer scientist at the University of Hamburg and spokesman for the joint project.
