Cooperative Persistent Surveillance with a Multi-Ugv System based on Reinforcement Learning

This paper investigates a persistent surveillance problem using a group of unmanned ground vehicles (UGVs) in a cooperative manner. The primary objective is to achieve continuous and frequent coverage of the entire target area through cooperation of the multi-UGV system. To this end, we model the persistent surveillance problem as a decentralized partially observable Markov decision process, where a knowability map is introduced for the target area and employed in the design of reward functions. Due to the limited sensing range of each UGV, the knowability map cannot be directly available. Thus, a consensus-based estimation method is designed for each UGV for estimation, and the issue of partial observability is resolved by fully exploiting observations from neighboring UGVs. Furthermore, we propose a deep reinforcement learningbased algorithm with the architecture of centralized training and distributed execution, which derives efficient cooperative surveillance policies for the UGVs. Extensive simulations demonstrate the effectiveness and robustness of the proposed algorithm for the persistent surveillance.