HOME ABOUT CV ADVENTURE CONTACT
HUMAN TELEOPERATION

Human Teleoperation: A Haptically-Enabled Mixed Reality System for Teleultrasound

Date: July 2020 - Present

Many remote communities have poor access to healthcare, even in wealthy countries like Canada. As a result, they spend up to almost half their annual healthcare budget on transporting patients to cities for treatment and diagnosis. While robotic telemedicine systems are often expensive and complex for such small communities, and video conferencing systems are inefficient and imprecise, we are developing a novel method of "teleoperating" a novice person via mixed reality to carry out medical ultrasound with tightly-coupled expert guidance. We have built a fully functioning prototype system and have characterized human performance in this mode of control, finding that it is comparable to robotic systems. Hence my research focuses on applying concepts from bilateral robotic teleoperation with time delays to this human-in-the-loop system. Additionally, we will explore AI for autonomous guidance of exams, and we are working on computer vision, HCI, high-speed communication over 5G, and instrumentation.

Awards and Presentations

For this project, I received the Mitacs Innovation Awards Outstanding Innovation prize in November, 2024. This is a prestigious award given by Mitacs, Canada's premier innovation organization, which connects industry, academia, and entrepreneurship and funds the resultant research.

My research on this project has been featured by numerous news outlets, including: Global TV, CBC News, CBC Radio 1 (starts around 30:45), Mitacs, Healthing.ca, Techouver, Canadian Healthcare Technology, CanadianSME, T-Net BC, as well as UBC Research and UBC Applied Science, Vancouver Coastal Health, the UBC ECE Department, and the UBC Institute for Computing, Information, and Cognitive Science.

I have also presented this work to Prof. Nassir Navab's groups at TU Munich (TUM) and Johns Hopkins (JHU), as well as Prof. Emad Boctor's lab at JHU, ImFusion GmbH in Munich, the German Aerospace Centre (DLR), and many demos listed below. I presented to a panel of judges who awarded me the BC Medical Device Design Centre Innovation Prize, and to the UBC Biomedical Imaging and AI Cluster Fall Research Showcase, where I also won the 3 minute thesis award. At the Hamlyn Symposium for Medical Robotics at Imperial College London, I received the best paper award for my presentation.

Finally, I shared the project at two Rogers-UBC collaboration workshops with UBC professors and Rogers experts, and applied for a grant to collaborate with and be funded by Rogers. We were awarded the grant as well as Mitacs funding, and are working in close consultation with the Coastal First Nations, Heiltsuk Nation, and Haida Gwaii, as well as several sonographers, radiologists, and emergency physicians. In 2023 we were also awarded a Vancouver Coastal Health Research and Innovation grant with Dr. Silvia Chang. The project is being undertaken under the supervision of Prof. Tim Salcudean of the Robotics and Controls Lab, UBC.

Publications