Link to Content Area

National Yang Ming Chiao Tung University


  • Industry Cooperation

  • Publish Date:2023-10-31
Boosting Automation Efficiency: NYCU Unveils MR-Based Robotics Advancement
President Tsai presented the 2023 Presidential Science Prize to Chenming Hu
Enhancing Automation Efficiency: Our University Unveils Mixed Reality Robotics
Translated by Pei-Hsuan Lee
Edited by Elaine Chuang


In a groundbreaking collaboration, Professor Kai-Tai Song and his team from the Department of Electronics and Electrical Engineering at National Yang Ming Chiao Tung University (NYCU) partnered with Techman Robot Inc., the world's second-largest collaborative robot manufacturer, to develop a cutting-edge technology for planning robotic arm paths using Mixed Reality (MR). This technology made its debut at ROScon 2023, the most highly anticipated automation and robotics conference, from October 18th to 20th.

MR Technology Enables Intuitive Robot Path and Task Setting with Hand Gestures

Professor Kai-Tai Song emphasized that this MR-based robot can assist frontline automation operators without prior robot programming experience. It allows them to intuitively set robot paths and tasks using hand gestures, thus enabling frontline workers to achieve their objectives more swiftly. Additionally, it facilitates human-robot interaction tasks in automated environments, showcasing the potential of MR technology in robotics.

Unlike previous automation processes where machines were controlled and operated via flat screens, this MR technology incorporates gesture recognition. By simply wearing glasses equipped with cameras, the operator can view the correspondence between the operator, the robotic arm, and the environment. Also, the technology can track and identify hand movements to plan virtual paths and then execute in the real-world environment. By completing the virtual path with intuitive gestures, the operator can send messages for precise adjustments and control the robotic arm to perform actions such as gripping or placing objects.

MR Tech Advancement Enhances Human-Robot Interaction and Automation Productivity

The groundbreaking technology was unveiled at ROSCon 2023 in New Orleans, USA, demonstrating this combined effort's power in enhancing collaborative robots' precision and efficiency. This advancement in MR technology increases human-robot interaction friendliness and boosts the automation industry's productivity.
Chenming Hu
Operators can set robot paths and tasks through intuitive hand gestures.