Home Big Data Mining and Analytics Article
PDF (1.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Open Access

Robust Non-Negative Matrix Tri-Factorization with Dual Hyper-Graph Regularization

School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China
College of Electronic and Information Engineering, Southwest University, Chongqing 400715, China, and also with Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, Chongqing 400715, China
School of Computing and Information Science, Faculty of Science and Engineering, Anglia Ruskin University, Cambridge CB1 1PT, UK
Department of Computer Science, Shantou University, Shantou 515063, China
College of Electronics and Information Engineering, Shenzhen University, Shenzhen 518060, China
Australia Artificial Intelligence Institute, University of Technology Sydney, Sydney 2007, Australia
Show Author Information

Abstract

Non-negative Matrix Factorization (NMF) has been an ideal tool for machine learning. Non-negative Matrix Tri-Factorization (NMTF) is a generalization of NMF that incorporates a third non-negative factorization matrix, and has shown impressive clustering performance by imposing simultaneous orthogonality constraints on both sample and feature spaces. However, the performance of NMTF dramatically degrades if the data are contaminated with noises and outliers. Furthermore, the high-order geometric information is rarely considered. In this paper, a Robust NMTF with Dual Hyper-graph regularization (namely RDHNMTF) is introduced. Firstly, to enhance the robustness of NMTF, an improvement is made by utilizing the l2,1-norm to evaluate the reconstruction error. Secondly, a dual hyper-graph is established to uncover the higher-order inherent information within sample space and feature spaces for clustering. Furthermore, an alternating iteration algorithm is devised, and its convergence is thoroughly analyzed. Additionally, computational complexity is analyzed among comparison algorithms. The effectiveness of RDHNMTF is verified by benchmarking against ten cutting-edge algorithms across seven datasets corrupted with four types of noise.

Keywords

Non-negative Matrix Tri-Factorization (NMTF)l2,1-normdual hyper-graph regularizationco-clustering

References

[1]
D. Seung and L. Lee, Algorithms for non-negative matrix factorization, Advances in Neural Information Processing Systems, vol. 13, pp. 556–562, 2001.
[2]
P. Padilla, M. Lopez, J. M. Gorriz, J. Ramirez, D. Salas-Gonzalez, and I. Alvarez, NMF-SVM based CAD tool applied to functional brain images for the diagnosis of Alzheimer’s disease, IEEE Trans. Med. Imag., vol. 31, no. 2, pp. 207–216, 2012.
Crossref
[3]

J. J.-Y. Wang, X. Wang, and X. Gao, Non-negative matrix factorization by maximizing correntropy for cancer clustering, BMC Bioinform., vol. 14, no. 1, p. 107, 2013.

Crossref Google Scholar
[4]

M. Nagayama, T. Aritake, H. Hino, T. Kanda, T. Miyazaki, M. Yanagisawa, S. Akaho, and N. Murata, Detecting cell assemblies by NMF-based clustering from calcium imaging data, Neural Networks, vol. 149, p. 29–39, 2022.

Crossref Google Scholar
[5]
A. Saha and V. Sindhwani, Learning evolving and emerging topics in social media: A dynamic NMF approach with temporal regularization, in Proc. fifth ACM Int. Conf. Web Search and Data Mining, Seattle, WA, USA, 2012, p. 693–702.
Crossref
[6]

A. Alfajri, D. Richasdy, and M. A. Bijaksana, Topic modelling using non-negative matrix factorization (NMF) for telkom university entry selection from instagram comments, J. Comput. Syst. Inform. Josyc, vol. 3, no. 4, pp. 485–492, 2022.

Crossref Google Scholar
[7]

N. Yu, Y.-L. Gao, J.-X. Liu, J. Wang, and J. Shang, Robust hypergraph regularized non-negative matrix factorization for sample clustering and feature selection in multi-view gene expression data, Hum. Genom., vol. 13, no. 1, p. 46, 2019.

Crossref Google Scholar
[8]

F. Wang, T. Li, X. Wang, S. Zhu, and C. Ding, Community discovery using nonnegative matrix factorization, Data Min. Knowl. Discov., vol. 22, no. 3, pp. 493–521, 2011.

Crossref Google Scholar
[9]

C. He, X. Fei, Q. Cheng, H. Li, Z. Hu, and Y. Tang, A survey of community detection in complex networks using nonnegative matrix factorization, IEEE Trans. Comput. Soc. Syst., vol. 9, no. 2, pp. 440–457, 2022.

Crossref Google Scholar
[10]
I. Psorakis, S. Roberts, and B. Sheldon, Efficient Bayesian community detection using non-negative matrix factorisation, arXiv preprint, arXiv: 1009.2646, 2010.
[11]

J. Cao, W. Xu, D. Jin, X. Zhang, A. Miller, L. Liu, and W. Ding, A network embedding-enhanced NMF method for finding communities in attributed networks, IEEE Access, vol. 10, pp. 118141–118155, 2022.

Crossref Google Scholar
[12]

W. Wu, S. Kwong, Y. Zhou, Y. Jia, and W. Gao, Nonnegative matrix factorization with mixed hypergraph regularization for community detection, Inf. Sci., vol. 435, pp. 263–281, 2018.

Crossref Google Scholar
[13]

N. Yu, M.-J. Wu, J.-X. Liu, C.-H. Zheng, and Y. Xu, Correntropy-based hypergraph regularized NMF for clustering and feature selection on multi-cancer integrated data, IEEE Trans. Cybern., vol. 51, no. 8, pp. 3952–3963, 2021.

Crossref Google Scholar
[14]
N. Yu, Y.-L. Gao, J.-X. Liu, J. Wang, and J. Shang, Hypergraph regularized NMF by L2, 1-norm for clustering and com-abnormal expression genes selection, in Proc. IEEE Int. Conf. Bioinformatics and Biomedicine, Madrid, Spain, 2018, pp. 578–582.
Crossref
[15]

M. Venkatasubramanian, K. Chetal, D. J. Schnell, G. Atluri, and N. Salomonis, Resolving single-cell heterogeneity from hundreds of thousands of cells through sequential hybrid clustering and NMF, Bioinformatics, vol. 36, no. 12, pp. 3773–3780, 2020.

Crossref Google Scholar
[16]

Y.-J. Hao, Y.-L. Gao, M.-X. Hou, L.-Y. Dai, and J.-X. Liu, Hypergraph regularized discriminative nonnegative matrix factorization on sample classification and co-differentially expressed gene selection, Complexity, vol. 2019, p. 12, 2019.

Crossref Google Scholar
[17]

M. H. Aghdam and M. D. Zanjani, A novel regularized asymmetric non-negative matrix factorization for text clustering, Inf. Process. Manag., vol. 58, no. 6, p. 102694, 2021.

Crossref Google Scholar
[18]

M. Bansal and D. Sharma, A novel multi-view clustering approach via proximity-based factorization targeting structural maintenance and sparsity challenges for text and image categorization, Inf. Process. Manag., vol. 58, no. 4, p. 102546, 2021.

Crossref Google Scholar
[19]

H. Che and J. Wang, A nonnegative matrix factorization algorithm based on a discrete-time projection neural network, Neural Netw., vol. 103, pp. 63–71, 2018.

Crossref Google Scholar
[20]

H. Che, J. Wang, and A. Cichocki, Bicriteria sparse nonnegative matrix factorization via two-timescale duplex neurodynamic optimization, IEEE Trans. Neural Netw. Learning Syst., vol. 34, no. 8, pp. 4881–4891, 2023.

Crossref Google Scholar
[21]
D. Kong, C. Ding, and H. Huang, Robust nonnegative matrix factorization using L21-norm, in Proc. 20th ACM Int. Conf. Information and knowledge management, Glasgow Scotland, UK, 2011, pp. 673–682.
Crossref
[22]
H. Gao, F. Nie, W. Cai, and H. Huang, Robust capped norm nonnegative matrix factorization: Capped norm NMF, in Proc. 24th ACM Int. on Conf. Information and Knowledge Management, Melbourne, Australia, 2015, pp. 871–880.
Crossref
[23]

N. Guan, T. Liu, Y. Zhang, D. Tao, and L. S. Davis, Truncated cauchy non-negative matrix factorization, IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 1, pp. 246–259, 2019.

Crossref Google Scholar
[24]
N. Guan, D. Tao, Z. Luo, and J. Shawe-Taylor, MahNMF: Manhattan non-negative matrix factorization, arXiv preprint, arXiv:1207.3438, 2012.
[25]

Z. Lin, C. Xu, and H. Zha, Robust matrix factorization by majorization minimization, IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 1, pp. 208–220, 2018.

Crossref Google Scholar
[26]

B. Yang, X. Zhang, B. Chen, F. Nie, Z. Lin, and Z. Nan, Efficient correntropy-based multi-view clustering with anchor graph embedding, Neural Networks, vol. 146, pp. 290–302, 2022.

Crossref Google Scholar
[27]

L. Li, J. Yang, K. Zhao, Y. Xu, H. Zhang, and Z. Fan, Graph regularized non-negative matrix factorization by maximizing correntropy, J. Comput., vol. 9, pp. 2570–2579, 2014.

Crossref Google Scholar
[28]

X. Yang, H. Che, M.-F. Leung, and C. Liu, Adaptive graph nonnegative matrix factorization with the self-paced regularization, Appl. Intell., vol. 53, no. 12, pp. 15818–15835, 2023.

Crossref Google Scholar
[29]

D. Cai, X. He, J. Han, and T. S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548–1560, 2011.

Crossref Google Scholar
[30]

F. Shang, L. C. Jiao, and F. Wang, Graph dual regularization non-negative matrix factorization for co-clustering, Pattern Recognit., vol. 45, no. 6, pp. 2237–2250, 2012.

Crossref Google Scholar
[31]

D. Tolić, N. Antulov-Fantulin, and I. Kopriva, A nonlinear orthogonal non-negative matrix factorization approach to subspace clustering, Pattern Recognit., vol. 82, pp. 40–55, 2018.

Crossref Google Scholar
[32]

J. Yu, D. Tao, and M. Wang, Adaptive hypergraph learning and its application in image classification, IEEE Trans. Image Process., vol. 21, no. 7, pp. 3262–3272, 2012.

Crossref Google Scholar
[33]

Y. Feng, H. You, Z. Zhang, R. Ji, and Y. Gao, Hypergraph neural networks, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 3558–3565, 2019.

Crossref Google Scholar
[34]

W. Wang, Y. Qian, and Y. Y. Tang, Hypergraph-regularized sparse NMF for hyperspectral unmixing, IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens., vol. 9, no. 2, pp. 681–694, 2016.

Crossref Google Scholar
[35]

K. Zeng, J. Yu, C. Li, J. You, and T. Jin, Image clustering by hyper-graph regularized non-negative matrix factorization, Neurocomputing, vol. 138, pp. 209–217, 2014.

Crossref Google Scholar
[36]
A. Salah, M. Ailem, and M. Nadif, Word co-occurrence regularized non-negative matrix tri-factorization for text data co-clustering, in Proc. AAAI Conf. Artif. Intell., https://doi.org/10.1609/aaai.v32i1.11659, 2018.
Crossref
[37]

T. Hwang, G. Atluri, M. Xie, S. Dey, C. Hong, V. Kumar, and R. Kuang, Co-clustering phenome-genome for phenotype classification and disease gene discovery, Nucleic Acids Res., vol. 40, no. 19, p. e146, 2012.

Crossref Google Scholar
[38]

P. Deng, T. Li, H. Wang, S.-J. Horng, Z. Yu, and X. Wang, Tri-regularized nonnegative matrix tri-factorization for co-clustering, Knowl. Based Syst., vol. 226, p. 107101, 2021.

Crossref Google Scholar
[39]
C. Ding, T. Li, W. Peng, and H. Park, Orthogonal nonnegative matrix t-factorizations for clustering, in Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Philadelphia, PA, USA, 2006, pp. 126–135.
Crossref
[40]

S. Wang and A. Huang, Penalized nonnegative matrix tri-factorization for co-clustering, Expert Syst. Appl., vol. 78, pp. 64–73, 2017.

Crossref Google Scholar
[41]

S. Peng, W. Ser, B. Chen, and Z. Lin, Robust orthogonal nonnegative matrix tri-factorization for data representation, Knowl. Based Syst., vol. 201, p. 106054, 2020.

Crossref Google Scholar
[42]

Y. Xu, L. Lu, Q. Liu, and Z. Chen, Hypergraph-regularized Lp smooth nonnegative matrix factorization for data representation, Mathematics, vol. 11, no. 13, p. 2821, 2023.

Crossref Google Scholar
[43]
C.-N. Jiao, J.-X. Liu, Y.-L. Gao, X.-Z. Kong, C.-H. Zheng, and X. Yu, Sparse hyper-graph non-negative matrix factorization by maximizing correntropy, in Proc. IEEE Int. Conf. Bioinformatics and Biomedicine, Houston, TX, USA, 2021, pp. 418–423.
Crossref
[44]

S. Li, W. Li, J. Hu, and Y. Li, Semi-supervised bi-orthogonal constraints dual-graph regularized NMF for subspace clustering, Appl. Intell., vol. 52, no. 3, pp. 3227–3248, 2022.

Crossref Google Scholar
[45]

G. Lu, C. Leng, B. Li, L. Jiao, and A. Basu, Robust dual-graph discriminative NMF for data classification, Knowl. Based Syst., vol. 268, p. 110465, 2023.

Crossref Google Scholar
[46]
L. Lovász and M. D. Plummer, Matching Theory. Rhode Island, USA: AMS Chelsea Publishing, 2009.
Crossref
[47]
D. Lee and H. S. Seung, Algorithms for non-negative matrix factorization, in Proc. Advances in Neural Information Processing Systems, Denver, CO, USA, 2000, pp. 535–541.
Big Data Mining and Analytics
Volume 8 Issue 1,
February 2025
Pages 214-232
DOI: 10.26599/BDMA.2024.9020055
Cite this article:
Yu J, Che H, Leung M-F, et al. Robust Non-Negative Matrix Tri-Factorization with Dual Hyper-Graph Regularization. Big Data Mining and Analytics, 2025, 8(1): 214-232. https://doi.org/10.26599/BDMA.2024.9020055
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return