AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (3.7 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Structure Optimization for Echo State Network Based on Contribution

College of Communication Engineering, Jilin University, Changchun 130022, China.
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China.
Department of Information Technology, Beijing Vocational College of Agriculture, Beijing 100124, China.
Show Author Information

Abstract

Echo State Network (ESN) is a recurrent neural network with a large, randomly generated recurrent part called the dynamic reservoir. Only the output weights are modified during training. However, proper balancing of the trade-off between the structure and performance for ESN remains a difficult task. In this paper, a structure optimized method for ESN based on contribution is proposed to simplify its network structure and improve its performance. First, we evaluate the contribution of reservoir neurons. Second, we present a pruning mechanism to remove the unimportant connection weights of reservoir neurons with low contribution. Finally, the new output weights are learned with the pseudo inverse method. The novel optimized ESN, named C-ESN, is tested on a Lorenz chaotic time-series prediction and an actual municipal sewage treatment system. The simulation results show that the C-ESN can have better prediction and generalization performance than ESN.

Keywords

neural networkstructural designtime-series prediction

References

[1]
Jaeger H. and Haas H., Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science, vol. 304, no. 5667, pp. 78-80, 2004.
Crossref Google Scholar
[2]
Jaeger H., The “echo state” approach to analysing and training recurrent neural networks, GMD Report, German National Research Center for Information Technology, Sankt Augustin, Germany, 2001.
[3]
Tong M. H., Bickett A. D., Christiansen E. M., and Cottrell G. W., Learning grammatical structure with Echo State Networks, Neural Networks, vol. 20, no. 3, pp. 424-432, 2007.
Crossref Google Scholar
[4]
Gideon K., Nyirenda C., and Nyah C. T., Echo state network-based radio signal strength prediction for wireless communication in Northern Namibia, IET Communications, vol. 11, no. 12, pp. 1920-1926, 2017.
Crossref Google Scholar
[5]
Coulibaly P., Reservoir computing approach to Great Lakes water level forecasting, Journal of Hydrology, vol. 381, nos. 1&2, pp. 76-88, 2010.
Crossref Google Scholar
[6]
Meftah B., Lézoray O., and Benyettou A., Novel approach using echo state networks for microscopic cellular image segmentation, Cognitive Computation, vol. 8, no. 2, pp. 237-245, 2016.
Crossref Google Scholar
[7]
Salmen M. and Ploger P. G., Echo state networks used for motor control, in Proc. IEEE International Conference on Robotics and Automation, Barcelona, Spain, 2006, pp. 1953-1958.
[8]
Wang H. S. and Yan X. F., Improved simple deterministically constructed Cycle Reservoir Network with Sensitive Iterative Pruning Algorithm, Neurocomputing, vol. 145, pp. 353-362, 2014.
Crossref Google Scholar
[9]
Rodan A. and Tino P., Minimum complexity echo state network, IEEE Transactions on Neural Networks, vol. 22, no. 1, pp. 131-144, 2011.
Crossref Google Scholar
[10]
Scardapane S., Nocco G., Comminiello D., Scarpiniti M., and Uncini A., An effective criterion for pruning reservoir’s connections in Echo State Networks, in Proc. International Joint Conference on Neural Networks, Beijing, China, 2015, pp. 1205-1212.
Crossref
[11]
Dutoit X., Schrauwen B., Campenhout J. V., Stoobandt D., Van Brussel H., and Nuttion M., Pruning and regularization in reservoir computing, Neurocomputing, vol. 72, nos. 7–9, pp. 1534-1546, 2009.
Crossref Google Scholar
[12]
Chouikhi N., Ammar B., Rokbani N., Alimi A. M., and Abraham A., A hybrid approach based on particle swarm optimization for echo state network initialization, in Proc. IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, China, 2016, pp. 2896-2901.
Crossref
[13]
Ferreira A. A. and Ludermir T. B., Comparing evolutionary methods for reservoir computing pre-training, in Proc. International Joint Conference on Neural Networks, San Jose, CA, USA, 2011, pp. 283-290.
Crossref
[14]
Kobialka H. U. and Kayani U., Echo state networks with sparse output connections, in Proc. 20th Int. Conference on Artificial Neural Networks: Part I, Thessaloniki, Greece, 2010, pp. 356-361.
Crossref
[15]
Shannon C. E., A mathematical theory of communication, ACM SIGMOBILE Mobile Computing and Communications Review, vol. 5, no. 1, pp. 3-55, 2001.
Crossref Google Scholar
[16]
Gharbi A. H., Ravier P., Harba R., and Mohamadi T., Low bias histogram-based estimation of mutual information for feature selection, Pattern Recognition Letters, vol. 33, no. 10, pp. 1302-1308, 2012.
Crossref Google Scholar
Tsinghua Science and Technology
Volume 24 Issue 1,
February 2019
Pages 97-105
DOI: 10.26599/TST.2018.9010049
Cite this article:
Li D, Liu F, Qiao J, et al. Structure Optimization for Echo State Network Based on Contribution. Tsinghua Science and Technology, 2019, 24(1): 97-105. https://doi.org/10.26599/TST.2018.9010049

569

Views

18

Downloads

15

Crossref

N/A

Web of Science

12

Scopus

1

CSCD

Altmetrics
Received: 03 January 2018
Accepted: 10 January 2018
Published: 08 November 2018
© The author(s) 2019
Return