AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning

Key Laboratory of Data Engineering and Knowledge Engineering of Ministry of Education Renmin University of China, Beijing 100872, China
School of Information, Renmin University of China, Beijing 100872, China
College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan 030027, China
Show Author Information

Abstract

Configuration tuning is essential to optimize the performance of systems (e.g., databases, key-value stores). High performance usually indicates high throughput and low latency. At present, most of the tuning tasks of systems are performed artificially (e.g., by database administrators), but it is hard for them to achieve high performance through tuning in various types of systems and in various environments. In recent years, there have been some studies on tuning traditional database systems, but all these methods have some limitations. In this article, we put forward a tuning system based on attention-based deep reinforcement learning named WATuning, which can adapt to the changes of workload characteristics and optimize the system performance efficiently and effectively. Firstly, we design the core algorithm named ATT-Tune for WATuning to achieve the tuning task of systems. The algorithm uses workload characteristics to generate a weight matrix and acts on the internal metrics of systems, and then ATT-Tune uses the internal metrics with weight values assigned to select the appropriate configuration. Secondly, WATuning can generate multiple instance models according to the change of the workload so that it can complete targeted recommendation services for different types of workloads. Finally, WATuning can also dynamically fine-tune itself according to the constantly changing workload in practical applications so that it can better fit to the actual environment to make recommendations. The experimental results show that the throughput and the latency of WATuning are improved by 52.6% and decreased by 31%, respectively, compared with the throughput and the latency of CDBTune which is an existing optimal tuning method.

Keywords

attention mechanismauto-tuning systemreinforcement learning (RL)workload-aware

Electronic Supplementary Material

Download File(s)
jcst-36-4-741-Highlights.pdf (253.8 KB)

References

[1]

O’Neil P, Cheng E, Gawlick D, O’Neil E. The log-structured merge-tree (LSM-tree). Acta Informatica, 1996, 33(4): 351-385. DOI: 10.1007/s002360050048.
Crossref Google Scholar
[2]
Dong S Y, Callaghan M, Galanis L, Borthakur D, Savor T, Stumm M. Optimizing space amplification in RocksDB. In Proc. the 8th Biennial Conference on Innovative Data Systems Research, Jan. 2017.
[3]
Chai Y P, Chai Y F, Wang X, Wei H C, Bao N, Liang Y S. LDC: A lower-level driven compaction method to optimize SSD-oriented key-value stores. In Proc. the 35th IEEE International Conference on Data Engineering, April 2019, pp.722-733. DOI: 10.1109/ICDE.2019.00070.
Crossref
[4]

Chai Y P, Chai Y F, Wang X, Wei H C, Wang Y Y. Adaptive lower-level driven compaction to optimize LSM-Tree key-value stores. IEEE Transactions on Knowledge Data Engineering. DOI: 10.1109/TKDE.2020.3019264.
Crossref Google Scholar
[5]
Zhu Y Q, Liu J X, Guo M Y, Bao Y G, Ma W L, Liu Z Y, Song K P, Yang Y C. BestConfig: Tapping the performance potential of systems via automatic configuration tuning. In Proc. ACM Symposium on Cloud Computing, Sept. 2017, pp.338-350. DOI: 10.1145/3127479.3128605.
Crossref
[6]
Van Aken D, Pavlo A, Gordon G J, Zhang B H. Automatic database management system tuning through large-scale machine learning. In Proc. the 2017 ACM International Conference on Management of Data, May 2017, pp.1009-1024. DOI: 10.1145/3035918.3064029.
Crossref
[7]
Zhang J, Liu L, Ran M, Li Z K, Liu Y, Zhou K, Li G L, Xiao Z L, Cheng B, Xing J S, Wang Y T, Cheng T H. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proc. the 2019 International Conference on Management of Data, June 2019, pp.415-432. DOI: 10.1145/3299869.3300085.
Crossref
[8]

Li G L, Zhou X H, Li S F, Gao B. QTune: A query-aware database tuning system with deep reinforcement learning. Proc. the VLDB Endowment, 2019, 12(12): 2118-2130. DOI: 10.14778/3352063.3352129.
Crossref Google Scholar
[9]
Lillicrap T P, Hunt J J, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. arXiv: 1509.02971, 2015. https://arxiv.org/abs/1509.02971, Jun. 2021.
[10]
Van Hasselt H. Double Q-learning. In Proc. the 24th Annual Conference on Neural Information Processing Systems, Dec. 2010, pp.2613-2621.
[11]
Kingma D, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2015.
[12]

Munos R, Moore A. Variable resolution discretization in optimal control. Machine Learning, 2002, 49(2/3): 291-323. DOI: 10.1023/A:1017992615625.
Crossref Google Scholar
[13]

Mnih V, Kavukcuoglu K, Silver D et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529-533. DOI: 10.1038/nature14236.
Crossref Google Scholar
[14]

Ban T W. An autonomous transmission scheme using dueling DQN for D2D communication networks. IEEE Transactions on Vehicular Technology, 2020, 69(12): 16348-16352. DOI: 10.1109/TVT.2020.3041458.
Crossref Google Scholar
[15]

Chen L, Hu X M, Tang B, Cheng Y. Conditional DQN-based motion planning with fuzzy logic for autonomous driving. IEEE Transactions on Intelligent Transportation Systems. DOI: 10.1109/TITS.2020.3025671.
Crossref Google Scholar
[16]

Huang H J, Yang Y C, Wang H, Ding Z G, Sari H, Adachi F. Deep reinforcement learning for UAV navigation through massive MIMO technique. IEEE Transactions on Vehicular Technology, 2020, 69(1): 1117-1121. DOI: 10.1109/TVT.2019.2952549.
Crossref Google Scholar
[17]

Li J X, Yao L, Xu X, Cheng B, Ren J K. Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving. Information Sciences, 2020, 532: 110-124. DOI: 10.1016/j.ins.2020.03.105.
Crossref Google Scholar
[18]

Yoo H, Kim B, Kim J W, Lee J H. Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation. Computers & Chemical Engineering, 2021, 144: Article No. 107133. DOI: 10.1016/j.compchemeng.2020.107133.
Crossref Google Scholar
[19]

He X M, Lu H D, Du M, Mao Y C, Wang K. QoE-based task offloading with deep reinforcement learning in edge-enabled Internet of vehicles. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(4): 2252-2261. DOI: 10.1109/TITS.2020.3016002.
Crossref Google Scholar
[20]

Li L Y, Xu H, Ma J, Zhou A Z. Joint EH time and transmit power optimization based on DDPG for EH communications. IEEE Communications Letters, 2020, 24(9): 2043-2046. DOI: 10.1109/LCOMM.2020.2999914.
Crossref Google Scholar
[21]

Nguyen D Q, Vien N A, Dang V H, Chung T. Asynchronous framework with Reptile+ algorithm to meta learn partially observable Markov decision process. Applied Intelligence, 2020, 50(11): 4050-4062. DOI: 10.1007/s10489-020-01748-7.
Crossref Google Scholar
[22]

Gheisarnejad M, Khooban M H. IoT-based DC/DC deep learning power converter control: Real-time implementation. IEEE Transactions on Power Electronics, 2020, 35(12): 13621-13630. DOI: 10.1109/TPEL.2020.2993635.
Crossref Google Scholar
[23]

Tang Z T, Shao K, Zhao D B, Zhu Y H. Recent progress of deep reinforcement learning: From AlphaGo to AlphaGoZero. Control Theory & Applications, 2017, 34(12): 1529-1546. DOI: 10.7641/CTA.2017.70808. (in Chinese)
Crossref Google Scholar
[24]

Silver D, Schrittwieser J, Simonyan K et al. Mastering the game of Go without human knowledge. Nature, 2017, 550(7676): 354-359. DOI: 10.1038/nature24270.
Crossref Google Scholar
[25]
Ye D H, Chen G B, Zhang W et al. Towards playing full MOBA games with deep reinforcement learning. arXiv: 2011.12692, 2020. https://arxiv.org/abs/2011.12692, Dec. 2020.
[26]

Li G L. Human-in-the-loop data integration. Proceedings of the VLDB Endowment, 2017, 10(12): 2006-2017. DOI: 10.14778/3137765.3137833.
Crossref Google Scholar
[27]

Li G L, Zhou X H, Li S H. XuanYuan: An AI-native database. IEEE Data Engineering Bulletin, 2019, 42(2): 70-81.
Google Scholar
[28]
Basu D, Lin Q, Chen W, Vo H T, Yuan Z, Senellart P, Bressan S. Regularized cost-model oblivious database tuning with reinforcement learning. In Transactions on Large-Scale Data- and Knowledge-Centered Systems XXVIII, Hameurlain A, Küng J, Wagner R, Chen Q (eds.), Springer, 2016, pp.96-132. DOI: 10.1007/978-3-662-53455-7_5.
Crossref
[29]

Sun J, Li G L. An end-to-end learning-based cost estimator. Proceedings of the VLDB Endowment, 2019, 13(3): 307-319. DOI: 10.14778/3368289.3368296.
Crossref Google Scholar
[30]
Kraska T, Alizadeh M, Beutel A et al. SageDB: A learned database system. In Proc. the 9th Biennial Conference on Innovative Data Systems Research, Jan. 2019.
[31]

Duan S Y, Thummala V, Babu S. Tuning database configuration parameters with iTuned. Proceedings of the VLDB Endowment, 2009, 2(1): 1246-1257. DOI: 10.14778/1687627.1687767.
Crossref Google Scholar
[32]
Wei Z J, Ding Z H, Hu J L. Self-tuning performance of database systems based on fuzzy rules. In Proc. the 11th International Conference on Fuzzy Systems and Knowledge Discovery, Aug. 2014, pp.194-198. DOI: 10.1109/FSKD.2014.6980831.
Crossref
[33]
Zheng C H, Ding Z H, Hu J L. Self-tuning performance of database systems with neural network. In Proc. the 10th International Conference on Natural Computation, Aug. 2014, pp.1-12. DOI: 10.1007/978-3-319-09333-8_1.
Crossref
Journal of Computer Science and Technology
Volume 36 Issue 4,
July 2021
Pages 741-761
DOI: 10.1007/s11390-021-1350-8
Cite this article:
Ge J-K, Chai Y-F, Chai Y-P. WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning. Journal of Computer Science and Technology, 2021, 36(4): 741-761. https://doi.org/10.1007/s11390-021-1350-8

446

Views

20

Crossref

9

Web of Science

16

Scopus

1

CSCD

Altmetrics
Received: 01 February 2021
Accepted: 24 June 2021
Published: 05 July 2021
©Institute of Computing Technology, Chinese Academy of Sciences 2021
Return