UNMANNED GROUND VEHICLE WITH VIRTUAL REALITY VISION

(Received: 2017-11-21, Revised: 2018-02-08 , Accepted: 2018-03-03)

Authors Mahmood Al-Khalil, Rami Abu-Rhayyem, Ahmad Hammoudeh, Talal A. Edwan*,

Keywords #UGV #Virtual reality #Search and rescue #Robotics #Human-robot interaction.

Abstract This paper aims to describe the design and implementation of an Unmanned Ground Vehicle (UGV) and a smart phone virtual reality (VR) head mounted display (HMD) which enables visual situation awareness by giving the operator the feel of "head on rover" while sending the video feeds to separate operator computer for object detection and 3-D model creation of the UGV surrounding objects. The main contribution of this paper is of three folds: (i) the novel design of the HMD; the paper proposes an alternative design to the 3-D interface designs recently used in tele-operated search and rescue (SAR) UGVs. Unlike other designs that suggest to automatically move the whole UGV about two axes (pitch and yaw) with the movement of the head, this design suggests to let a separate unit of the UGV automatically move with the movement of the head and provide the user with VR. (ii) the distributed feature; the design allows multiple users to connect to the UGV using a wireless link in a secure way to receive video feeds from three on-board cameras. This feature facilitates cooperative team work in urban search and rescue (USAR) applications (a contemporary research issue in SAR UGV). (iii) a novel feature of the design is the simultaneous video feeds which are sent to the operator station computer for object detection using the scale-invariant feature transform (SIFT) algorithm and 3-D model construction of the UGV’s surrounding objects from 2-D images of these objects. The design was realized using a smart phone-based HMD, which captures head movements in real time using its inertial measurement unit (IMU) and transmits it to three motors mounted on a rover to provide the movement about three axes (pitch, yaw and roll). The operator controls the motors via the HMD or a gamepad. Three on-board cameras provide video feeds which are transmitted to the HMD and operator computer. A software performs object detection and builds a 3-D model from the captured 2-D images. The realistic design constraints were identified, then the hardware/software functions that meet the constraints were listed. The UGV was implemented in a laboratory environment. It was tested over soft and rough terrain. Results showed that the UGV has higher visual- inspection capabilities compared to other existing SAR UGVs. Furthermore, it was found that the maximum speed of 3.3 m/s, six-wheel differential-drive chassis and spiked air-filled rubber tires of the rover gave it high manoeuvrability in open rough terrain compared to other SAR UGVs found in literature. The high visual inspection capabilities and relatively high speed of the UGV make it a good choice for planetary exploration and military reconnaissance. The three-motors and stereoscopic camera can be easily mounted as a separate unit on a chassis that uses different locomotion mechanism (e.g. leg type or tracked type) to extend the functionality of a SAR UGV. The design can be used in building disparity maps and in constructing 3-D models, or in real time face recognition, real time object detection and autonomous driving based on disparity maps.

References

[1] K. Cavallin and P. Svensson, "Semi-autonomous, Teleoperated Search and Rescue Robot," 2009.

[2] J. L. Drury, J. Scholtz and H. A. Yanco, "Awareness in Human-robot Interactions," IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 912-918, Oct. 2003.

[3] N. Michael, S. Shen, K. Mohta, Y. Mulgaonkar, V. Kumar, K. Nagatani, Y. Okada, S. Kiribayashi, K. Otake, Yoshida et al., "Collaborative Mapping of an Earthquake-damaged Building via Ground and Aerial Robots," Journal of Field Robotics, vol. 29, no. 5, pp. 832-841, 2012.

[4] R. R. Murphy, J. Kravitz, S. L. Stover and R. Shoureshi, "Mobile Robots in Mine Rescue and Recovery," IEEE Robotics & Automation Magazine, vol. 16, no. 2, 2009.

[5] Y. Baudoin, D. Doroftei, G. De Cubber, S. A. Berrabah, C. Pinzon, F. Warlet, J. Gancet, E. Motard, M. Ilzkovitz, Nalpantidis et al., "View- finder: Robotics Assistance to Re-fighting Services and Crisis Management," IEEE International Workshop on Safety, Security & Rescue Robotics (SSRR), pp. 1-6, 2009.

[6] D. Doroftei, G. De Cubber, E. Colon and Y. Baudoin, "Behavior Based Control for an Outdoor Crisis Man-agement Robot," Proceedings of the IARP International Workshop on robotics for Risky Interventions and Environmental Surveillance, pp. 12-14, 2009.

[7] G. De Cubber, D. Doroftei, H. Sahli and Y. Baudoin, "Outdoor Terrain Traversability Analysis for Robot Navigation Using a Time-of- light Camera," Proc. RGB-D Workshop on 3D Perception in robotics, Vasteras, Sweden, 2011.

[8] C. Marques, J. Cristvo, P. Alvito, P. Lima, J. Frazo, I. Ribeiro and R. Ventura, "A Search and Rescue Robot with Teleoperated Tether Docking System," Industrial Robot: An International Journal, vol. 34, no. 4, pp. 332-338, 2007,[Online], Available: https://doi.org/10.1108/01439910710749663.

[9] N. Enayati and F. Najafi, "Design and Manufacturing of a Teleoperative Rescue UGV with a Novel Track Arrangement," Industrial Robot: An International Journal, vol. 38, no. 5, pp. 476-485, 2011,[Online], Available: https://doi.org/10.1108/01439911111154045.

[10] K. Berns, A. Nezhadfard, M. Tosa, H. Balta and G. De Cubber, "Unmanned Ground Robots for Rescue Tasks," Search and Rescue Robotics-From Theory to Practice, InTech, 2017.

[11] H. Martins and R. Ventura, "Immersive 3-D Teleoperation of a Search and Rescue Robot Using a Head-mounted Display," IEEE Conference on Emerging Technologies Factory Automation, pp. 1-8, Sept. 2009.

[12] "Pololu: Robotics & Electronics,"[Online], Available: http://www.pololu.com/, 2017, accessed: 3- October-2017.

[13] M. Pedley, "Tilt Sensing Using a Three-axis Accelerometer," Freescale Semiconductor Application Note, vol. 1, pp. 2012-2013, 2013.

[14] D. G. Lowe, "Distinctive Image Features from Scale-invariant Key Points," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.

[15] C. M. Johnson and S.-J. Hsieh, "Board# 143: Maker: Urban Search and Rescue UGV: Visual localization and Navigation," ASEE Annual Conference & Exposition, 2017.

[16] Z. Wang and H. Gu, "A Review of Locomotion Mechanisms of Urban Search and Rescue Robot," Industrial Robot: An International Journal, vol. 34, no. 5, pp. 400-411, 2007,[Online], Available: https://doi.org/10.1108/01439910710774403.

[17] M. Neumann, T. Predki, L. Heckes and P. Labenda, "Snakelike, Tracked, Mobile Robot with Active Flippers for Urban Search and Rescue Tasks," Industrial Robot: An International Journal, vol. 40, no. 3, pp. 246-250, 2013,[Online], Available: https://doi.org/10.1108/01439911311309942.

[18] P. U. Lima, "Search and Rescue UGVs: The Civil Protection Teams of the Future," 3rd International Conference on Emerging Security Technologies, pp. 12-19, Sept. 2012.

[19] M. W. Kadous, R. K.-M. Sheh and C. Sammut, "Caster: A Robot for Urban Search and Rescue," Proceedings of the 2005 Australasian Conference on Robotics and Automation, pp. 1-10, 2005.

[20] A. Kleiner, Mapping and Exploration for Search and Rescue with Humans and Mobile Robots, University of Freiburg, 2009.

[21] N. Cardwell, Y. Cheng, C. S. Gunn, S. H. Yeganeh and V. Jacobson, "BBR: Congestion-Based Congestion Control," ACM Queue, vol. 14, pp. 20-53, 2016.

[22] Y. Wang, Z. Song, Y. Zhou and C. Zhou, "Design of A Tele-Operated Field Robot," Defense Science Research Conference and Expo (DSR), pp. 1-4, Aug. 2011.

[23] D. Sanders "Comparing Speed to Complete Progressively More Difficult Mobile Robot Paths Between Human Tele-operators and Humans," Assembly Automation, vol. 29, pp. 230-248, 2017.

[24] D. Sanders, "Analysis of the Effects of Time Delays on the Teleoperation of a Mobile Robot in Various Modes of Operation," Industrial Robot: An International Journal, vol. 36, pp. 570-584, 2009.

Abstract: This paper aims to describe the design and implementation of an Unmanned Ground Vehicle (UGV) and a smart phone virtual reality (VR) head mounted display (HMD) which enables visual situation awareness by giving the operator the feel of "head on rover" while sending the video feeds to separate operator computer for object detection and 3-D model creation of the UGV surrounding objects. The main contribution of this paper is of three folds: (i) the novel design of the HMD; the paper proposes an alternative design to the 3-D interface designs recently used in tele-operated search and rescue (SAR) UGVs. Unlike other designs that suggest to automatically move the whole UGV about two axes (pitch and yaw) with the movement of the head, this design suggests to let a separate unit of the UGV automatically move with the movement of the head and provide the user with VR. (ii) the distributed feature; the design allows multiple users to connect to the UGV using a wireless link in a secure way to receive video feeds from three on-board cameras. This feature facilitates cooperative team work in urban search and rescue (USAR) applications (a contemporary research issue in SAR UGV). (iii) a novel feature of the design is the simultaneous video feeds which are sent to the operator station computer for object detection using the scale-invariant feature transform (SIFT) algorithm and 3-D model construction of the UGV’s surrounding objects from 2-D images of these objects. The design was realized using a smart phone-based HMD, which captures head movements in real time using its inertial measurement unit (IMU) and transmits it to three motors mounted on a rover to provide the movement about three axes (pitch, yaw and roll). The operator controls the motors via the HMD or a gamepad. Three on-board cameras provide video feeds which are transmitted to the HMD and operator computer. A software performs object detection and builds a 3-D model from the captured 2-D images. The realistic design constraints were identified, then the hardware/software functions that meet the constraints were listed. The UGV was implemented in a laboratory environment. It was tested over soft and rough terrain. Results showed that the UGV has higher visual- inspection capabilities compared to other existing SAR UGVs. Furthermore, it was found that the maximum speed of 3.3 m/s, six-wheel differential-drive chassis and spiked air-filled rubber tires of the rover gave it high manoeuvrability in open rough terrain compared to other SAR UGVs found in literature. The high visual inspection capabilities and relatively high speed of the UGV make it a good choice for planetary exploration and military reconnaissance. The three-motors and stereoscopic camera can be easily mounted as a separate unit on a chassis that uses different locomotion mechanism (e.g. leg type or tracked type) to extend the functionality of a SAR UGV. The design can be used in building disparity maps and in constructing 3-D models, or in real time face recognition, real time object detection and autonomous driving based on disparity maps.
URL: https://jjcit.org/paper/39

,abstract={This paper aims to describe the design and implementation of an Unmanned Ground Vehicle (UGV) and a smart phone virtual reality (VR) head mounted display (HMD) which enables visual situation awareness by giving the operator the feel of "head on rover" while sending the video feeds to separate operator computer for object detection and 3-D model creation of the UGV surrounding objects. The main contribution of this paper is of three folds: (i) the novel design of the HMD; the paper proposes an alternative design to the 3-D interface designs recently used in tele-operated search and rescue (SAR) UGVs. Unlike other designs that suggest to automatically move the whole UGV about two axes (pitch and yaw) with the movement of the head, this design suggests to let a separate unit of the UGV automatically move with the movement of the head and provide the user with VR. (ii) the distributed feature; the design allows multiple users to connect to the UGV using a wireless link in a secure way to receive video feeds from three on-board cameras. This feature facilitates cooperative team work in urban search and rescue (USAR) applications (a contemporary research issue in SAR UGV). (iii) a novel feature of the design is the simultaneous video feeds which are sent to the operator station computer for object detection using the scale-invariant feature transform (SIFT) algorithm and 3-D model construction of the UGV’s surrounding objects from 2-D images of these objects. The design was realized using a smart phone-based HMD, which captures head movements in real time using its inertial measurement unit (IMU) and transmits it to three motors mounted on a rover to provide the movement about three axes (pitch, yaw and roll). The operator controls the motors via the HMD or a gamepad. Three on-board cameras provide video feeds which are transmitted to the HMD and operator computer. A software performs object detection and builds a 3-D model from the captured 2-D images. The realistic design constraints were identified, then the hardware/software functions that meet the constraints were listed. The UGV was implemented in a laboratory environment. It was tested over soft and rough terrain. Results showed that the UGV has higher visual- inspection capabilities compared to other existing SAR UGVs. Furthermore, it was found that the maximum speed of 3.3 m/s, six-wheel differential-drive chassis and spiked air-filled rubber tires of the rover gave it high manoeuvrability in open rough terrain compared to other SAR UGVs found in literature. The high visual inspection capabilities and relatively high speed of the UGV make it a good choice for planetary exploration and military reconnaissance. The three-motors and stereoscopic camera can be easily mounted as a separate unit on a chassis that uses different locomotion mechanism (e.g. leg type or tracked type) to extend the functionality of a SAR UGV. The design can be used in building disparity maps and in constructing 3-D models, or in real time face recognition, real time object detection and autonomous driving based on disparity maps.},
keywords={UGV,Virtual reality,Search and rescue,Robotics,Human-robot interaction.},
ISSN={2413-9351},
month={April}}

AB - This paper aims to describe the design and implementation of an Unmanned Ground Vehicle (UGV) and a smart phone virtual reality (VR) head mounted display (HMD) which enables visual situation awareness by giving the operator the feel of "head on rover" while sending the video feeds to separate operator computer for object detection and 3-D model creation of the UGV surrounding objects. The main contribution of this paper is of three folds: (i) the novel design of the HMD; the paper proposes an alternative design to the 3-D interface designs recently used in tele-operated search and rescue (SAR) UGVs. Unlike other designs that suggest to automatically move the whole UGV about two axes (pitch and yaw) with the movement of the head, this design suggests to let a separate unit of the UGV automatically move with the movement of the head and provide the user with VR. (ii) the distributed feature; the design allows multiple users to connect to the UGV using a wireless link in a secure way to receive video feeds from three on-board cameras. This feature facilitates cooperative team work in urban search and rescue (USAR) applications (a contemporary research issue in SAR UGV). (iii) a novel feature of the design is the simultaneous video feeds which are sent to the operator station computer for object detection using the scale-invariant feature transform (SIFT) algorithm and 3-D model construction of the UGV’s surrounding objects from 2-D images of these objects. The design was realized using a smart phone-based HMD, which captures head movements in real time using its inertial measurement unit (IMU) and transmits it to three motors mounted on a rover to provide the movement about three axes (pitch, yaw and roll). The operator controls the motors via the HMD or a gamepad. Three on-board cameras provide video feeds which are transmitted to the HMD and operator computer. A software performs object detection and builds a 3-D model from the captured 2-D images. The realistic design constraints were identified, then the hardware/software functions that meet the constraints were listed. The UGV was implemented in a laboratory environment. It was tested over soft and rough terrain. Results showed that the UGV has higher visual- inspection capabilities compared to other existing SAR UGVs. Furthermore, it was found that the maximum speed of 3.3 m/s, six-wheel differential-drive chassis and spiked air-filled rubber tires of the rover gave it high manoeuvrability in open rough terrain compared to other SAR UGVs found in literature. The high visual inspection capabilities and relatively high speed of the UGV make it a good choice for planetary exploration and military reconnaissance. The three-motors and stereoscopic camera can be easily mounted as a separate unit on a chassis that uses different locomotion mechanism (e.g. leg type or tracked type) to extend the functionality of a SAR UGV. The design can be used in building disparity maps and in constructing 3-D models, or in real time face recognition, real time object detection and autonomous driving based on disparity maps.

Download Full Text