Ben M. Chen

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong 999077, China

In recent decades, academia and industry have put increasing attention and investment into the research and development of autonomous unmanned systems.An autonomous unmanned vehicle is a machine equipped with necessary data-processing units,sophisticated sensors, environment perception, automatic control,motion planning, task planning, mission management, and communication systems. It is an integration of advanced technologies in many fields, including artificial intelligence (AI), and can perform and complete certain specific tasks autonomously without a human operator.Autonomous systems,such as unmanned ground vehicles (UGVs) [1], unmanned aerial vehicles (UAVs) [2],unmanned surface vehicles(USVs)[3],and unmanned underwater vehicles(UUVs)[4](Fig.1),are projected to play significant roles in industrial applications, such as reconnaissance for search and rescue, security surveillance, environmental and traffic monitoring,powerline and pipeline inspections, building inspections, geographic mapping, tunnel inspections, film productions, logistic delivery, and warehouse management. Autonomous unmanned vehicles have thus become a new economic focus with rapidly growing development.

The market for autonomous unmanned systems, such as UAVs and UGVs for industrial applications, has expanded tremendously in recent years. Technologies that have been developed mainly for military purposes are now being applied to commercial uses.It is expected that the market for civilian autonomous unmanned systems will be expanded and developed to cover most aspects in people’s daily lives. Advanced autonomous systems are gaining promising success due to their low cost,flexibility,and minimized risk of operation.

In the foreseeable future, unmanned systems—including land,surface,and air vehicles—will increase in popularity.Such vehicles can be employed to handle a wide range of tasks that have hitherto been under the purview of human operators. Unmanned systems must be able to operate under complex and dense clustered conditions for various missions.Moreover,most existing unmanned systems have been designed for specified tasks and environments,which cannot easily be extended to different tasks or dynamic environments. Hence, the research and development of intelligent autonomous unmanned systems is still in its infancy.

Fig.2 depicts the key technical components involved in forming an intelligent autonomous unmanned system. The discussion below briefly describes the functionality of each module and outlines some ongoing and potential research elements.

(1) Hardware platform and sensors.These consist of a mechanical hardware platform and a set of sensors to sense the surrounding environment and to collect measurement data and data for applications.Aside from conventional hardware platforms,such as those used in aircraft, ground, surface, and underwater vehicles,there has been increasing interest in expanding the functionality of unmanned systems by developing multimodal locomotion,such as land-water and land-air hybrids,as well as air-water unmanned vehicles, which can operate in more complicated environments. A vehicle’s perception and data-collecting sensor set might include the integration of light detection and ranging(LiDAR), vision, sonar, radar, and/or other sensors, together with sophisticated sensor-fusion techniques [5].

(2) Perception and localization.Localization is a process that determines where an unmanned vehicle is located with respect to its environment,and is one of the most fundamental and essential components required for vehicle control and high-level decision-making. In a typical localization scenario [6], the unmanned vehicle is equipped with sensors that observe the environment and monitor the vehicle’s own motion. The localization problem is to estimate the position and orientation of the vehicle through the information gathered from the sensor set. Perception is also necessary in many applications in which an unmanned vehicle must be operated in a complicated environment while performing tasks such as object detection, environment representation, scene understanding, activity recognition, and semantic place classification [7].

(3) Dynamical control system.The dynamical control system is essential for an unmanned vehicle to be operated with little or no interference from human operators.Although many researchers have attempted to design control systems without dynamical models,this author believes that a good control system design requires a clear understanding of the hardware platform dynamics, as well as a fairly accurate mathematical model. The dynamical control system of an unmanned system might involve multiple layers,such as an inner-loop control system and an outer-loop controller.In general,the inner-loop control is employed to stabilize the attitude of the unmanned vehicle, while the outer-loop system controls the position and velocity of the vehicle [8]. In some unconventional unmanned platforms, more advanced control systems may be needed to yield a good performance.

Fig. 1. Illustration of an unmanned systems framework. Comm.: communication.

Fig. 2. Technical structure of an intelligent autonomous unmanned system. HCI: human-computer interaction; HMI: human-machine interaction.

In general, any physical system equipped with the abovementioned three functionalities can be called an ‘‘unmanned system.”It is projected that more research works will be carried out on UUV systems, which will include the development of advanced underwater vehicles [9], underwater image processing [10], and underwater simultaneous localization and mapping (SLAM).

(4)Motion planning.Motion planning entails breaking down a desired movement task (generated from a task-planning module)into discrete segments that satisfy movement constraints, such as obstacle avoidance in cluttered environments, constraints in space, and constraints associated with the velocity and acceleration of the vehicle. It generally optimizes specific aspects of the movement, such as traveling time or energy consumption. A motion-planning algorithm produces an appropriate trajectory containing position, velocity, acceleration, and/or jerk reference signals for the lower-level dynamical control systems. Collisionfree specification and dynamical feasibility are the two major constraints in motion planning; these involve the geometric constraints induced by obstacles in the environment (path planning), and the dynamic constraints of the vehicle (trajectory generation) [11].

(5) Task planning.Many real-life applications require more than just the movement of an unmanned vehicle from one place to another while avoiding obstacles.Guiding an unmanned vehicle to achieve a complicated mission often requires additional planning at the task level.Task planning involves translating a mission into discrete operations that can allow the successful execution of the mission; these operations may include gross-motion planning with collision avoidance, sensory operations planning, and finemotion planning with specific tasks. In contrast to its planning counterpart—that is, motion planning, which focuses on the dynamic constraints of the vehicle and the geometric constraints of the environment in the continuous domain—task planning works with a highly abstract model,such as planning domain definition language (PDDL), in the discrete domain and focuses on generating a coarse task plan. Task system dynamics are typically modeled as a discrete transition system and are solved by discrete planning techniques [11].

In general, when unmanned systems are integrated with additional task- and motion-planning capabilities, they can be called‘‘a(chǎn)utonomous unmanned systems” (or ‘‘a(chǎn)utonomous systems” in short),as they can perform certain relatively simple tasks and missions autonomously.Current research in motion planning is focusing on enhancing the speed of their navigation process. Attempts have also been made to integrate the task-planning and motionplanning processes to improve the efficacy of the overall system.

To make it an ‘‘intelligent machine,” an unmanned system should be further equipped with advanced AI capabilities, such as an expert system with specific domain knowledge that can be utilized to decide and assign missions automatically. In many applications,a human-machine interaction(HMI)system[12]that allows human operators in the loop to interact with the unmanned vehicle would become necessary in order for the system to make fast and sensible decisions.

To integrate an autonomous unmanned system intelligently and effectively into the real world, a clear understanding of its surrounding environment and the use of mathematical models of the environment would be highly desirable to provide interdisciplinary analyses for effective decision-making. The subject of environmental modeling deals with the description of processes that occur in the physical world in space and time. Spatial interactions and topological rules are commonly characterized by geographic information systems (GIS) [13]. The processes that transform the environment over time are described by sets of differential equations, similar to the dynamical modeling of a physical system.

An autonomous unmanned system that possesses advanced AI capabilities is a so-called ‘‘intelligent autonomous unmanned system” (or simply ‘‘intelligent autonomous system”). As mentioned earlier, research in this direction is still very premature. A great deal of work remains to be done, along with the actual utilization of multi-agent unmanned systems. Fig. 3 depicts our understanding of a practical framework for the cooperation of multi-agent autonomous unmanned systems,which can be coordinated among each other at the dynamical-control or motion-planning level—such as through formation and consensus control(these have been heavily studied in the past decade or so; e.g., see Ref. [14] and the references therein)—or at the task and mission level, which would align the coordination of different types of unmanned systems in a more logical fashion [15].

In conclusion, it is the author’s belief that research on autonomous unmanned systems is a never-ending topic. Given the development of powerful techniques in AI and deep learning for data processing, the subject of intelligent autonomous unmanned systems will be one of the major technological and economic stories of the years to come, due to the wide range of applications of and value added by this potential technology.Finally, it is worth noting that, for some applications, the detection and catching of intruding unmanned systems are crucial issues to be investigated [16]. Advanced sensing and perception techniques using bio-inspired and event-based neuromorphic vision sensors [17] are another interesting and promising area for future research.

Acknowledgment

This work is supported in part by the Research Grants Council of Hong Kong SAR (14209020).

Fig. 3. Cooperation of multi-agent autonomous unmanned systems.