The Development of a Novel Eye Tracking Based Remote Camera Control for Mining Teleoperation

The use of teleoperation in mining is increasing due to requirements to improve safety and reduce the number of people required to work in remote and difficult environments. We recently completed a project to develop a teleoperated control system for a rock breaker at an iron ore mine in the Pilbara region of Western Australia, with the objective of demonstrating the feasibility of remote rock breaking over long distances. This required controlling both the rock breaker and an array of pan-tilt-zoom (PTZ) cameras using joysticks. Controlling the cameras distracted operators from the primary task of controlling the rock breaker to position and fire the hammer. This lengthened task times as both the operator's attention and their hands have to swap between different control interfaces. In this paper, we present a novel design using human eye gaze as an interactive input for the remote camera control. It follows a simple and natural design principle: Whatever you look at the screen, it goes to the centre. A prototype system has been implemented by integrating computer vision based eye tracking technology, with the advantage of being real-time, robust and marker-less. We conducted a user evaluation, where users undertook a control task while obscuring the direct view of the working area and using a remote camera to transfer the live video stream as the visual feedback. Subjects were required to participate in the experiment using eye tracking camera control and traditional joystick control respectively. The statistical analysis of objective measures indicated that our novel eye tracking control significantly outperformed the joystick control. The results of the post-experimental subjective measures also revealed much higher user preference for using eye tracking control, which provides clear evidence that this design is a better interface for remote camera control in mining teleoperation settings.