Design of deep Q-networks for transfer time prediction of spacecraft orbit-raising

Abstract

Recently, there has been a surge in use of electric propulsion to transfer satellites to the geostationary Earth orbit (GEO). Traditionally, the transfer times to reach GEO using all electric propulsion are obtained by solving challenging trajectory optimization problems, whose solution rely on numerical schemes that are not only computationally intensive, but also lack automated implementation capabilities. This naturally creates a hindrance to their incorporation within Deep Reinforcement Learning (DRL) framework, which combines Reinforcement Learning (RL) and Deep Learning to solve trajectory planning problems in near real-time. The operation of DRL, as typically used in trajectory planning, relies on a Q-value. In the electric orbit-raising problem under consideration in this thesis, this Q-Value requires computation of transfer time in near real-time to have practical DRL training times. In our work, this Q-value is predicted by a set of deep neural networks (DNNs) instead of solving traditional optimization problems. This thesis aims at designing a set of DNNs that can serve as a Q-value (transfer time) predictor for different orbit-raising mission scenarios. To this end, we investigate different architectures for DNNs to determine the optimal DNN configuration that can predict the transfer time for each of the mission scenarios. Experimental results indicate that our designed DNNs can predict the transfer time for different scenarios with an accuracy of over 99%. We also compare the results from our designed DNNs with the contemporary Machine Learning (ML) algorithms, such as Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT) for regression. Experimental results indicate that our best-performing DNNs can provide an improvement in mean error of transfer time prediction of up to 14.05× for non-planar transfers and up to 254× for planar transfers.