AUV Pipeline Following using Reinforcement Learning

Abstract

This paper analyzes the application of several reinforcement learning techniques for continuous state and action spaces to pipeline following for an autonomous underwater vehicle (AUV). Continuous space SARSA is compared to the actor-critic CACLA algorithm, and is also extended into a supervised reinforcement learning architecture. A novel exploration method using the skew-normal stochastic distribution is proposed, and evidence towards advantages in the case of tabula rasa exploration is presented. Results are validated on a realistic simulator of the AUV, and confirm the applicability of reinforcement learning to optimize pipeline following behavior.