AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Chinese Journal of Aeronautics Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Full Length Article | Open Access

Efficient and fair PPO-based integrated scheduling method for multiple tasks of SATech-01 satellite

Qi SHIa, Lu LIb( ), Ziruo FANGb,c, Xingzi BIb, Huaqiu LIUb, Xiaofeng ZHANGb, Wen CHENb,c, Jinpei YUb,c
Shanghai Satellite Network Research Institute CO., LTD, Shanghai 201210, China
Innovation Academy for Microsatellites of Chinese Academy of Sciences, Shanghai 201306, China
University of Chinese Academy of Sciences, Beijing 100039, China
Show Author Information

Abstract

SATech-01 is an experimental satellite for space science exploration and on-orbit demonstration of advanced technologies. The satellite is equipped with 16 experimental payloads and supports multiple working modes to meet the observation requirements of various payloads. Due to the limitation of platform power supply and data storage systems, proposing reasonable mission planning schemes to improve scientific revenue of the payloads becomes a critical issue. In this article, we formulate the integrated task scheduling of SATech-01 as a multi-objective optimization problem and propose a novel Fair Integrated Scheduling with Proximal Policy Optimization (FIS-PPO) algorithm to solve it. We use multiple decision heads to generate decisions for each task and design the action mask to ensure the schedule meeting the platform constraints. Experimental results show that FIS-PPO could push the capability of the platform to the limit and improve the overall observation efficiency by 31.5% compared to rule-based plans currently used. Moreover, fairness is considered in the reward design and our method achieves much better performance in terms of equal task opportunities. Because of its low computational complexity, our task scheduling algorithm has the potential to be directly deployed on board for real-time task scheduling in future space projects.

Keywords

Reinforcement learningSatellite observatoriesSATech-01Multi-modes platformScheduling algorithmsProximal Policy Optimization (PPO)

References

1
An ZH, Antier S, Bi XZ, et al. Insight-HXMT and GECAM-C observations of the brightest-of-all-time GRB 221009A. arXiv preprint: 2303.01203; 2023.
2

Bai XY, Tian H, Deng YY, et al. The solar upper transition region imager (SUTRI) onboard the SATech-01 satellite. Res Astron Astrophys 2023;23(6):065014.
Crossref Google Scholar
3
Ling ZX, Liu Y, Zhang C, et al. LEIA discovered an X-ray transient LXT 221107A. Available from: https://nadc.china-vo.org/ep/cms/article/view?id=96.
4

Zhang C, Ling ZX, Sun XJ, et al. First wide field-of-view X-ray observations by a lobster-eye focusing telescope in orbit. ApJL 2022;941(1):L2.
Crossref Google Scholar
5
China’s first self-developed quantum magnetometer payload realizes global magnetic field measurement [Internet]. National Space Science Center, CAS. [updated 2022 Nov 14]. Available from: http://www.nssc.ac.cn/xwdt2015/xwsd2015/202211/t20221114_6548153.html [Chinese].
6
Bensana E, Verfaillie G, Agnese JC, et al. Exact and inexact methods for the daily management of an earth observation satellite. Proceedings of the fourth international symposium space mission operations and ground data systems-SpaceOps’ 96.1996.p. 507–14.
7

Wang MC, Dai GM, Vasile M. Heuristic scheduling algorithm oriented dynamic tasks for imaging satellites. Math Probl Eng 2014;2014:1–11.
Crossref Google Scholar
8

Chen YN, Lu J, He RJ, et al. An efficient local search heuristic for earth observation satellite integrated scheduling. Appl Sci 2020;10(16):5616.
Crossref Google Scholar
9

Chen H, Wu JJ, Shi WY, et al. Coordinate scheduling approach for EDS observation tasks and data transmission jobs. J Syst Eng Electron 2016;27(4):822–35.
Crossref Google Scholar
10

Baek SW, Han SM, Cho KR, et al. Development of a scheduling algorithm and GUI for autonomous satellite missions. Acta Astronaut 2011;68(7–8):1396–402.
Crossref Google Scholar
11

Zhang JW, Xing LN. An improved genetic algorithm for the integrated satellite imaging and data transmission scheduling problem. Comput Oper Res 2022;139:105626.
Crossref Google Scholar
12

He YM, Xing LN, Chen YW, et al. A generic Markov decision process model and reinforcement learning method for scheduling agile earth observation satellites. IEEE Trans Syst Man Cybern 2022;52(3):1463–74.
Crossref Google Scholar
13
Hadj-Salah A, Verdier R, Caron C, et al. Schedule earth observation satellites with deep reinforcement learning. arXiv preprint: 1911.05696; 2019.
14

Wen ZJ, Li L, Song JK, et al. Scheduling single-satellite observation and transmission tasks by using hybrid Actor-Critic reinforcement learning. Adv Space Res 2023;71(9):3883–96.
Crossref Google Scholar
15

Li GL, Xing LN, Chen YW. A hybrid online scheduling mechanism with revision and progressive techniques for autonomous Earth observation satellite. Acta Astronaut 2017;140:308–21.
Crossref Google Scholar
16

Li DL, Gu YF, Jaubert J, et al. A robust scheduling algorithm for space telescopes with unpredictable tasks. Sci China Technol Sci 2021;64(3):571–84.
Crossref Google Scholar
17
Maillard A, Verfaillie G, Pralet C, et al. Postponing decision-making to deal with resource uncertainty on earth-observation satellites. Proceedings of the 9th international workshop on planning and scheduling for space (IWPSS 2015). 2015.
18
Sherwood R, Chien S, Tran D, et al. Intelligent systems in space: The EO-1 Autonomous Sciencecraft. Proceedings of the Infotech@Aerospace. Reston: AIAA; 2005.
Crossref
19

Gleyzes MA, Perret L, Kubik P. Pleiades system architecture and main performances. Int Arch Photogramm Remote Sens Spatial Inf Sci 2012;XXXIX-B1:537–42.
Crossref Google Scholar
20
Bucher S, Williams B, Pendexter M, et al. Chandra mission scheduling on-orbit experience. Observatory operations: Strategies, processes, and systems Ⅱ 2008.
Crossref
21
Bello I, Pham H, Le QV, et al. Neural combinatorial optimization with reinforcement learning. arXiv preprint: 1611.09940; 2016.
22
Nazari M, Oroojlooy A, Snyder LV, et al. Reinforcement learning for solving the vehicle routing problem. arXiv preprint: 1802.04240; 2018.
23
Peng B, Wang JH, Zhang ZZ. A deep reinforcement learning algorithm using dynamic attention model for vehicle routing problems. International symposium on intelligence computation and applications. 2020. p. 636-50.
Crossref
24

Wang HJ, Yang Z, Zhou WG, et al. Online scheduling of image satellites based on neural networks and deep reinforcement learning. Chin J Aeronaut 2019;32(4):1011–9.
Crossref Google Scholar
25

Wei LN, Chen YN, Chen M, et al. Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem. Appl Soft Comput 2021;110:107607.
Crossref Google Scholar
26

Zhao XX, Wang ZK, Zheng GT. Two-phase neural combinatorial optimization with reinforcement learning for agile satellite scheduling. J Aerosp Inf Syst 2020;17(7):346–57.
Crossref Google Scholar
27

Huang YX, Mu ZC, Wu SF, et al. Revising the observation satellite scheduling problem based on deep reinforcement learning. Remote Sens 2021;13(12):2377.
Crossref Google Scholar
28
Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms. arXiv preprint: 1707.06347; 2017.
29
Schulman J, Moritz P, Levine S, et al. High-dimensional continuous control using generalized advantage estimation. arXiv preprint: 1506.02438; 2015.
30

Lee J, Kim H, Chung H, et al. Schedule optimization of imaging missions for multiple satellites and ground stations using genetic algorithm. JASS 2018;19(1):139–52.
Crossref Google Scholar
31

Niu XN, Tang H, Wu LX. Satellite scheduling of large areal tasks for rapid response to natural disaster using a multi-objective genetic algorithm. Int J Disaster Risk Reduct 2018;28:813–25.
Crossref Google Scholar
32

Wu J, Song BY, Zhang GT, et al. A data-driven improved genetic algorithm for agile earth observation satellite scheduling with time-dependent transition time. Comput Ind Eng 2022;174:108823.
Crossref Google Scholar
Chinese Journal of Aeronautics
Volume 37 Issue 2,
February 2024
Pages 417-430
DOI: 10.1016/j.cja.2023.10.011
Cite this article:
SHI Q, LI L, FANG Z, et al. Efficient and fair PPO-based integrated scheduling method for multiple tasks of SATech-01 satellite. Chinese Journal of Aeronautics, 2024, 37(2): 417-430. https://doi.org/10.1016/j.cja.2023.10.011

106

Views

1

Crossref

1

Web of Science

1

Scopus

Altmetrics
Received: 13 February 2023
Revised: 06 May 2023
Accepted: 02 July 2023
Published: 21 October 2023
© 2023 Chinese Society of Aeronautics and Astronautics.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Return