This paper analyzes the application of several reinforcement learning techniques for continuous state and action spaces to pipeline following for an autonomous underwater vehicle (AUV). Continuous space SARSA is compared ...