PDF (268.2 KB)
Collect
Submit Manuscript
Show Outline
Figures (3)

Tables (3)
Table. I
Table. II
Table. III
Research paper | Open Access

Collaborative filtering recommendation algorithm based on variational inference

Kai ZhengXianjun YangYilei Wang()Yingjie WuXianghan Zheng
College of Mathematics Computer Science/College of Software, Fuzhou University, Fuzhou, China
Show Author Information

Abstract

Purpose

The purpose of this paper is to alleviate the problem of poor robustness and over-fitting caused by large-scale data in collaborative filtering recommendation algorithms.

Design/methodology/approach

Interpreting user behavior from the probabilistic perspective of hidden variables is helpful to improve robustness and over-fitting problems. Constructing a recommendation network by variational inference can effectively solve the complex distribution calculation in the probabilistic recommendation model. Based on the aforementioned analysis, this paper uses variational auto-encoder to construct a generating network, which can restore user-rating data to solve the problem of poor robustness and over-fitting caused by large-scale data. Meanwhile, for the existing KL-vanishing problem in the variational inference deep learning model, this paper optimizes the model by the KL annealing and Free Bits methods.

Findings

The effect of the basic model is considerably improved after using the KL annealing or Free Bits method to solve KL vanishing. The proposed models evidently perform worse than competitors on small data sets, such as MovieLens 1 M. By contrast, they have better effects on large data sets such as MovieLens 10 M and MovieLens 20 M.

Originality/value

This paper presents the usage of the variational inference model for collaborative filtering recommendation and introduces the KL annealing and Free Bits methods to improve the basic model effect. Because the variational inference training denotes the probability distribution of the hidden vector, the problem of poor robustness and overfitting is alleviated. When the amount of data is relatively large in the actual application scenario, the probability distribution of the fitted actual data can better represent the user and the item. Therefore, using variational inference for collaborative filtering recommendation is of practical value.

References

 

Adomavicius, G. and Tuzhilin, A. (2005), “Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions”, IEEE Transactions on Knowledge and Data Engineering, Vol. 17 No. 6, pp. 734-749.

 
Bowman, S.R. Vilnis, L. Vinyals, O. Dai, A.M. Jozefowicz, R. and Bengio, S. (2015), “Generating sentences from a continuous space”, Computer Science.https://doi.org/10.18653/v1/K16-1002
 
Cao, S., Yang, N. and Liu, Z. (2017), “Online news recommender based on stacked au-to-encoder”, IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), IEEE, pp. 721-726.https://doi.org/10.1109/ICIS.2017.7960088
 
Chen, X. Kingma, D.P. Salimans, T. Duan, Y. Dhariwal, P. Schulman, J. Sutskever, I. and Abbeel, P. (2016), “Variational lossy autoencoder”.
 

Daunizeau, J., Adam, V. and Rigoux, L. (2014), “VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data”, PLoS Computational Biology, Vol. 10 No. 1, p. e1003441, doi: 10.1371/journal.pcbi.1003441.

 

Deldjoo, Y., Dacrema, M.F., Constantin, M.G., Eghbal-Zadeh, H., Cereda, S., Schedl, M., Ionescu, B. and Cremonesi, P. (2019), “Movie genome: alleviating new item cold start in movie recommendation”, User Modeling and User-Adapted Interaction, Vol. 2019 No. 5, pp. 1-53, doi: 10.1007/s11257-019-09221-y.

 

Duh, K. (2018), “Bayesian analysis in natural language processing”, Computational Linguistics, Vol. 44 No. 1, pp. 187-189, doi: 10.1162/COLI_r_00310.

 

Gershman, S.J., Blei, D.M., Norman, K.A. and Sederberg, P.B. (2014), “Decomposing spatiotemporal brain patterns into topographic latent sources”, NeuroImage, Vol. 98, pp. 91-102, doi: 10.1016/j.neuroimage.2014.04.055.

 
Hu, K. and O’Connor, P. (2018), “Learning a representation map for robot navigation using deep variational autoencoder”.
 
Kingma, D.P. Salimans, T. Jozefowicz, R. Chen, X. Sutskever, I. and Welling, M. (2016), “Improving variational inference with inverse autoregressive flow”, NIPS.
 
Kingma, D.P. and Welling, M. (2013), “Auto-encoding variational bayes”, arXiv preprint arXiv:1312.6114.
 
Krishnan, R. Subedar, M. and Tickoo, O. (2018), “BAR: Bayesian activity recognition using variational inference”, arXiv preprint arXiv:1811.03305.
 
Ning, X. and Karypis, G. (2011), “SLIM: sparse linear methods for top-N recommender systems”, 2011 IEEE 11th International Conference on Data Mining, IEEE, pp. 497-506.https://doi.org/10.1109/ICDM.2011.134
 
Ning, X. and Karypis, G. (2012), “Sparse linear methods with side information for top-n recommendations”, Proceedings of the sixth ACM Conference on Recommender systems, ACM, pp. 155-162.https://doi.org/10.1145/2365952.2365983
 

Schedl, M., Zamani, H., Chen, C.W., Deldjoo, Y. and Elahi, M. (2018), “Current challenges and visions in music recommender systems research”, International Journal of Multimedia Information Retrieval, Vol. 7 No. 2, pp. 95-116, doi: 10.1007/s13735-018-0154-2.

 
Sedhain, S., Menon, A.K., Sanner, S. and Xie, L. (2015), “Autorec: autoencoders meet collaborative filtering”, Proceedings of the 24th International Conference on World Wide Web, ACM, pp. 111-112.https://doi.org/10.1145/2740908.2742726
 
Shen, X. Su, H. Niu, S. and Demberg, V. (2018), “Improving variational encoder-decoders in dialogue generation”.
 

Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y. and Manzagol, P.A. (2010), “Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion”, Journal of Machine Learning Research, Vol. 11 No. 12, pp. 3371-3408, doi: 10.1016/j.mechatronics.2010.09.004.

 
Wang, C. and Blei, D.M. (2011), “Collaborative topic modeling for recommending scientific articles”, Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 448-456.https://doi.org/10.1145/2020408.2020480
 
Wang, H., Wang, N. and Yeung, D.Y. (2015), “Collaborative deep learning for recommender systems”, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pp. 1235-1244.https://doi.org/10.1145/2783258.2783273
 
Wang, P. and Blunsom, P. (2015), “Stochastic collapsed variational inference for hidden markov models”, Statistics.
 
Wu, Y., Dubois, C., Zheng, A.X. and Ester, M. (2016), “Collaborative denoising auto-encoders for top-n recommender systems”, Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, ACM, pp. 153-162.https://doi.org/10.1145/2835776.2835837
 
Yan, C., Yan, H., Zhang, Q. and Wan, Y. (2018), “NSPD: an N-stage purchase decision model for e-commerce recommendation”, Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data, Springer, Cham, pp. 149-164.https://doi.org/10.1007/978-3-319-96890-2_13
International Journal of Crowd Science
Pages 31-44
Cite this article:
Zheng K, Yang X, Wang Y, et al. Collaborative filtering recommendation algorithm based on variational inference. International Journal of Crowd Science, 2020, 4(1): 31-44. https://doi.org/10.1108/IJCS-10-2019-0030
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return