In this paper, the recently developed machine learning (ML) approach to improve orbit prediction accuracy is systematically investigated using three ML algorithms, including support vector machine (SVM), artificial neural network (ANN), and Gaussian processes (GPs). In a simulation environment consisting of orbit propagation, measurement, estimation, and prediction processes, totally 12 resident space objects (RSOs) in solar-synchronous orbit (SSO), low Earth orbit (LEO), and medium Earth orbit (MEO) are simulated to compare the performance of three ML algorithms. The results in this paper show that ANN usually has the best approximation capability but is easiest to overfit data; SVM is the leastlikely to overfit but the performance usually cannot surpass ANN and GPs. Additionally, the ML approach with all the three algorithms is observed to be robust with respect to the measurement noise.