Many remote communities have poor access to healthcare, even in wealthy countries like Canada. As a result, they spend up to almost half their annual healthcare budget on transporting patients to cities for treatment and diagnosis. While robotic telemedicine systems are often expensive and complex for such small communities, and video conferencing systems are inefficient and imprecise, we are developing a novel method called human teleoperation. Human teleoperation is a framework for tightly coupled guidance of a novice follower person by an expert sonographer through a remote mixed reality (MR) system. By combining high-speed communication, MR, and haptics we can achieve intuitive, fast, precise teleoperation of an ultrasound procedure with almost robot-like performance but human flexibility and intelligence. This can enable greatly improved access to healthcare in small or remote communities.
This project has been featured by numerous news outlets, including: CBC News, Global TV, CTV News, the Toronto Sun and Ottawa Sun, Electronic Products and Technology, CBC Radio 1 (starts around 30:45), The Tyee, Mitacs, Mitacs again, Healthing.ca, Techouver, Canadian Healthcare Technology, CanadianSME, T-Net BC, as well as UBC Research and UBC Applied Science, Vancouver Coastal Health, the UBC ECE Department, and the UBC Institute for Computing, Information, and Cognitive Science.
This page contains some of our work so far, listed below. For an overview of the system and a test deployment to Skidegate in Haida Gwaii, see the videos below.
A PowerPoint slide deck about the system is also available here.
D. Black, S. Salcudean. Differential Magnetic Multi-axis Force Sensing, US Patent Office Provisional Patent. May 2023.
D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. System and Method for Low-Cost Remote Guidance of Procedures, US Patent Office Patent Application. March 2023.
D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. System and Apparatus for Remote Interaction with an Object, US Patent Office Patent Application. July 2021.
D. Black, S. Salcudean. Stability and Transparency in Mixed Reality Bilateral Teleoperation submitted to IEEE Transactions on Robotics, October, 2024.
Paper which models and analyzes the bilateral teleoperation system mathematically, following robotics and control literature and testing a variety of controllers for performance and robustness to time delays.
R. Yeung, D. Black, P. Chen, V. Lessoway, J. Reid, S. Rangel-Suarez, S. Chang, S. Salcudean, Mixed Reality Tele-ultrasound over 750 km: a Clinical Study, submitted to IEEE Transactions on Biomedical Engineering
The paper we have submitted about our tests in Skidegate in the Haida Gwaii islands, in which the expert was located over 750km away at our lab at UBC.
D. Black, S. Salcudean. Robust Object Pose Tracking for Augmented Reality Guidance and Teleoperation, in IEEE Transactions on Instrumentation and Measurement, May 2024.
D. Black, A.H. Hadi Hosseinabadi, M. Nogami, N. Rangga, S. Salcudean. Low-Profile 6-Axis Differential Magnetic Force/Torque Sensing, in IEEE Transactions on Medical Robotics and Bionics, May 2024.
D. Black, D. Andjelic, S.E Salcudean. Evaluation of Communication and Human Response Latency for (Human) Teleoperation, in IEEE Transactions on Medical Robotics. January, 2024.
D. Black, M. Nogami, S. Salcudean. Mixed Reality Human Teleoperation with Device-Agnostic Remote Ultrasound: Communication and HCI, in Computers and Graphics, January, 2024
D. Black, A.H. Hadi Hosseinabadi, M. Nogami, M. Pol, N. Rangga, S. Salcudean. Towards Differential Magnetic Force Sensing for Ultrasound Teleoperation, in IEEE World Haptics Conference, July 10-13, 2023, Delft, NL.
D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. Human Teleoperation - a Haptically-Enabled Mixed Reality System for Teleultrasound, Human Computer Interaction. June, 2023 (submitted July 2021).
D. Black, H. Moradi, S.E Salcudean. Human-as-a-Robot Performance in Augmented Reality Teleultrasound, in International Journal of Computer Assisted Radiology and Surgery. April, 2023.
D. Black, S. Salcudean. Mixed Reality Human Teleoperation, in IEEE VR, March 25-29, 2023, Shanghai, China. Workshop Presentation & Paper.
D. Black, S. Salcudean. A Mixed Reality System for Human Teleoperation in Tele-Ultrasound, in the Hamlyn Symposium for Medical Robotics, June 26-29, 2022, London, UK. Podium Presentation.