LTSA-LE: A Local Tangent Space Alignment Label Enhancement Algorithm

Chao Tan; Genlin Ji; Richen Liu; Yanqiu Cao

doi:10.26599/TST.2019.9010052

| Sign up

PDF (521.6 KB)

Cite

EndNote(RIS) BibTeX

Collect

Submit Manuscript

Show Outline

Tables (15)

Table 1

Table 2

Table 3

Table 4

Table 5

Open Access

LTSA-LE: A Local Tangent Space Alignment Label Enhancement Algorithm

Chao Tan(), Genlin Ji, Richen Liu, Yanqiu Cao

School of Computer Science and Engineering, Southeast University, Nanjing 210096.

School of Computer Science and Technology, Nanjing Normal University, Nanjing 210023, China.

Show Author Information

Abstract

According to smoothness assumption, local topological structure can be shared between feature and label manifolds. This study proposes a new algorithm based on Local Tangent Space Alignment (LTSA) to implement the label enhancement process. In general, we first establish a learning model for feature extraction in label space and use a feature extraction method of LTSA to guide the reconstruction of label manifolds. Then, we establish an unconstrained optimization model based on the optimal theory presented in this paper. The model is suitable for solving problems with a large number of sample points. Finally, the experiment results show that the algorithm can effectively improve the training speed and multilabel dataset prediction accuracy.

Keywords

smoothness assumption feature manifold label manifold unconstrained optimization

References

[1]

Geng

, N.

, and R.

Shao

, Label enhancement for label distribution learning, Journal of Computer Research and Development, vol. 54, no. 6, pp. 1171-1184, 2017.

Google Scholar

[2]

Wang

, J.

Xin

, H.

Yang

, S.

Tian

, G.

, C.

, and Y.

Yao

, Distributed and weighted extreme learning machine for imbalanced big data learning, Tsinghua Science and Technology, vol. 22, no. 2, pp. 160-173, 2017.

Crossref Google Scholar

[3]

Wang

, M.

Yang

, W.

Yang

, and Y.

Yin

, Deep cross-view label embedding with correlation and structure preserved for multi-label classification, in Proc. of International Conference on Tools with Artificial Intelligence, Volos, Greece, 2018, pp. 12-19.

Crossref

[4]

Tai

and H.

Lin

, Multilabel classification with principal label space transformation, Neural Computation, vol. 24, no. 9, pp. 2508-2542, 2012.

Crossref Google Scholar

[5]

Sun

, S.

, and J.

, Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 33, no. 1, pp. 194-200, 2011.

Crossref Google Scholar

[6]

Tuia

, J.

Verrelst

, L.

Alonso

, F.

Prez-Cruz

, and G.

Camps-Valls

, Multioutput support vector regression for remote sensing biophysical parameter estimation, Geoscience and Remote Sensing Letters, vol. 8, no. 4, pp. 804-808, 2011.

Crossref Google Scholar

[7]

Geng

, Label distribution learning, IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 7, pp. 1734-1748, 2016.

Crossref Google Scholar

[8]

Hou

, X.

Geng

, and M.

Zhang

, Multi-label manifold learning, in Proc. of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA, 2016, pp. 1680-1686.

[9]

Zhu

, J.

Lafferty

, and R.

Rosenfeld

, Semi-supervised learning with graphs, PhD dissertation, Dept. Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA, 2005.

[10]

, A.

Tao

, and X.

Geng

, Label enhancement for label distribution learning, in Proc. of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018, pp. 2926-2932.

Crossref

[11]

Peng

, A.

Tao

, and X.

Geng

, Label embedding based on multi-scale locality preservation, in Proc. of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018, pp. 2623-2629.

Crossref

[12]

Shao

, N.

, and X.

Geng

, Multi-label learning with label enhancement, in Proc. of IEEE International Conference on Data Mining, Singapore, 2018, pp. 437-446.

Crossref

[13]

Zhang

and H.

Zha

, Principal manifolds and nonlinear dimensionality reduction via tangent space alignment, SIAM Journal of Scientific Computing, vol. 26, no. 1, pp. 313-338, 2004.

Crossref Google Scholar

[14]

Jiang

, Z.

Deng

, J.

Wang

, P.

Qian

, and S.

Wang

, Collaborative partition multi-view fuzzy clustering algorithm using entropy weighting, Journal of Software, vol. 25, no. 10, pp. 2293-2311, 2014.

Google Scholar

[15]

Zhang

and Z.

Zhou

, Multi-label neural networks with applications to functional genomics and text categorization, IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 10, pp. 1338-1351, 2006.

Crossref Google Scholar

[16]

Geng

and P.

Hou

, Pre-release prediction of crowd opinion on movies by label distribution learning, in Proc. of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 2015, pp. 3511-3517.

[17]

Geng

, C.

Yin

, and Z.

Zhou

, Facial age estimation by learning from label distributions, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 10, pp. 2401-2412, 2013.

Crossref Google Scholar

[18]

Zhang

and Z.

Zhou

, A review on multi-label learning algorithms, IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819-1837, 2014.

Crossref Google Scholar

[19]

Demsar

, Statistical comparisons of classifiers over multiple datasets, Journal of Machine Learning Research, vol. 7, no. 1, pp. 1-30, 2006.

Google Scholar

Tsinghua Science and Technology

Volume 26 Issue 2,
April 2021

Pages 135-145

DOI: 10.26599/TST.2019.9010052

Cite this article:

Tan C, Ji G, Liu R, et al. LTSA-LE: A Local Tangent Space Alignment Label Enhancement Algorithm. Tsinghua Science and Technology, 2021, 26(2): 135-145. https://doi.org/10.26599/TST.2019.9010052

Return

10.26599/TST.2019.9010052.T1Table 1Characteristic of multilabel datasets.

Dataset	$S$	$T$	$dim (S)$	$L (S)$	$LCard (S)$	$LDen (S)$	$DL (S)$	$F (S)$
Emotions	415	178	72	6	1.869	0.311	27	numeric
Medical	645	333	1449	45	1.245	0.028	94	nominal
Cal500	250	252	68	174	26.044	0.150	502	numeric
Birds	320	325	260	19	1.014	0.053	133	numeric
Enron	1123	579	1001	53	3.378	0.064	753	nominal
Yeast	1200	1217	103	14	4.237	0.303	198	numeric
Image	1000	1000	294	5	1.236	0.247	20	numeric
Scene	1211	1196	294	6	1.074	0.179	15	numeric
Corel5k	2500	2500	499	374	3.522	0.009	3175	nominal
Bibtex	3700	3695	1836	159	2.402	0.015	2856	nominal

10.26599/TST.2019.9010052.T2Table 2Comparison of MLL and multilabel distribution algorithms on Hamming loss $↓$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	0.4500	0.2987	0.0290	0.1472	0.0683	0.3056	0.2904	0.0682	0.0094	0.0160	3.000
ML $^{2}$	0.2267	0.2865	0.3405	0.3701	0.1330	0.2450	0.1803	0.2049	0.3922	0.0747	3.600
LDSVR	0.3037	0.2996	0.9721	0.1488	0.0517	0.7516	0.1810	0.0677	0.9907	0.0149	3.450
CPNN	0.6964	0.7097	0.9732	0.8522	0.9491	0.7522	0.8194	0.9339	0.9907	0.9853	5.950
AA-KNN	0.2297	0.3006	0.0184	0.1814	0.0748	0.2158	0.1134	0.0705	0.0114	0.0165	3.100
LTSA-LE	0.1958	0.3024	0.0145	0.1400	0.1705	0.1656	0.0962	0.0558	0.0096	0.0127	1.900

10.26599/TST.2019.9010052.T3Table 3Comparison of MLL and multilabel distribution algorithms on ranking loss $↓$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	0.4450	0.4803	0.2445	0.1996	0.3964	0.7956	0.5992	0.3738	0.2695	0.4764	2.800
ML $^{2}$	0.3325	0.3579	0.4984	0.4981	0.4051	0.2470	0.1915	0.4493	0.5101	0.2199	2.900
LDSVR	0.4974	0.5899	0.5000	0.5005	0.4374	0.5000	0.6556	0.4741	0.5000	0.5012	4.300
CPNN	0.9708	0.8511	0.8982	0.8621	0.3132	0.8892	0.8609	0.9621	0.4990	0.6954	5.100
AA-KNN	0.5054	0.4283	0.5039	0.7750	0.7335	0.3139	0.1838	0.8563	0.9444	0.7416	4.600
LTSA-LE	0.3244	0.2426	0.1453	0.4648	0.3158	0.1406	0.0713	0.3509	0.4393	0.1052	1.300

10.26599/TST.2019.9010052.T4Table 4Comparison of MLL and multilabel distribution algorithms on one error $↓$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	0.7034	0.7022	0.4024	0.1071	0.7989	0.6710	0.8269	0.2642	0.9716	0.4547	4.300
ML $^{2}$	0.4286	0.6667	0.4737	0.0827	0.9474	0.2000	0.6667	0.8846	0.9564	0.6792	4.000
LDSVR	0.4286	0.6667	0.5000	0.8563	0.4990	0.5000	0.4999	0.9615	0.4890	0.9497	4.500
CPNN	0.0714	0.3333	0.4290	0.3333	0.8421	0.5470	0.3333	0.5050	0.4400	0.9874	3.400
AA-KNN	0.4999	0.4899	0.1579	0.5862	0.4737	0.4990	0.5000	0.4808	0.4419	0.7688	3.100
LTSA-LE	0.0000	0.5000	0.3947	0.7816	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	1.700

10.26599/TST.2019.9010052.T5Table 5Comparison of MLL and multilabel distribution algorithms on coverage $↓$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	0.8990	0.3089	0.2955	1.3386	0.4415	2.1460	2.0761	0.2369	0.1980	0.7356	5.100
ML $^{2}$	0.8950	0.1723	0.7684	0.2313	0.3031	0.9962	1.0617	0.5029	0.1912	0.3513	4.200
LDSVR	0.8982	0.1568	0.2087	0.2284	0.3014	0.9608	1.0843	0.4936	1.5023	0.3382	3.250
CPNN	0.8845	0.1703	0.2081	0.2316	0.2309	0.9648	1.0773	0.5028	1.5023	0.3598	3.650
AA-KNN	0.8794	0.1661	0.1374	0.2315	0.2899	0.9644	1.0505	0.4956	1.5126	0.3585	3.300
LTSA-LE	0.8846	0.1586	0.0598	0.2275	0.2523	0.9538	0.1048	0.4456	0.1502	0.2596	1.500

10.26599/TST.2019.9010052.T6Table 6Comparison of MLL and multilabel distribution algorithms on average precision $↑$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	0.4297	0.5161	0.2081	0.4783	0.2460	0.5111	0.4200	0.2057	0.2012	0.0659	2.600
ML $^{2}$	0.4366	0.4442	0.4683	0.1644	0.1407	0.4905	0.3078	0.1234	0.2930	0.3872	3.200
LDSVR	0.3965	0.4900	0.0480	0.1676	0.0759	0.2729	0.7859	0.0747	0.0141	0.0226	4.750
CPNN	0.3064	0.3123	0.0467	0.1598	0.1013	0.2645	0.2954	0.0828	0.0141	0.0182	5.750
AA-KNN	0.4779	0.4926	0.3692	0.1705	0.1131	0.5954	0.7649	0.1201	0.0252	0.1111	3.100
LTSA-LE	0.4825	0.5567	0.5631	0.1829	0.1602	0.7105	0.7957	0.1843	0.0278	0.3678	1.600

10.26599/TST.2019.9010052.T7Table 7Comparison of MLL and multilabel distribution algorithms on time (s) $↓$ .

Algorithm	Dataset										Average rank
Algorithm	Yeast	Emotions	Medical	Cal500	Birds	Image	Scene	Enron	Corel5k	Bibtex	Average rank
BP-MLL	4.6563	0.7020	6.7080	10.2337	1.5625	1.5288	1.6380	13.2757	218.3234	122.6948	5.600
ML $^{2}$	0.4836	0.0005	0.1716	0.1404	0.0469	0.2500	0.3438	0.1875	1.5444	4.5625	3.300
LDSVR	0.1406	0.0001	0.1872	0.0780	0.0001	0.2496	0.2652	0.1716	1.0156	3.5781	2.300
CPNN	0.1092	0.0001	0.5616	0.2496	0.0468	0.1404	0.0780	0.9048	15.0385	25.6466	3.100
AA-KNN	3.9936	0.2340	9.8593	0.2184	0.5460	3.9624	5.7876	14.7109	21.4345	107.28	5.300
LTSA-LE	0.0938	0.0001	0.0781	0.0469	0.1404	0.0938	0.0624	0.1406	0.7969	2.9531	1.400

10.26599/TST.2019.9010052.T8Table 8Friedman statistics $F_{𝐅}$ in terms of each evaluation metric, the critical value is 2.42 when the significance level is 0.05 (the number of comparing algorithms, k, is 6 and the number of datasets, N, is 10).

Evaluation metric	$F_{F}$	Evaluation metric	$F_{F}$
Hamming loss	9.50	Coverage	6.25
Ranking loss	12.29	Average precision	16.38%
One error	3.91	Time	2.75 s

10.26599/TST.2019.9010052.T9Table 9Critical values for post-hoc tests after the Friedman test.

Number of classifiers	$q_{0.05}$	$q_{0.10}$	Number of classifiers	$q_{0.05}$	$q_{0.10}$
2	1.960	1.645	7	2.638	2.394
3	2.241	1.960	8	2.690	2.450
4	2.394	2.128	9	2.724	2.498
5	2.498	2.241	10	2.773	2.539
6	2.576	2.326

10.26599/TST.2019.9010052.T10Table 10Experimental results on the real-world datasets measured by the Cheb $↓$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.0080	0.0073	0.0090	0.0073
Yeast-cdc	0.0141	0.0138	0.0229	0.0083
Yeast-elu	0.0267	0.0262	0.0189	0.0078
Yeast-diau	0.0414	0.0505	0.0572	0.0194
Yeast-heat	0.0481	0.0504	0.0669	0.0118
Yeast-spo	0.1774	0.1969	0.1830	0.0268
Yeast-cold	0.0773	0.0727	0.0833	0.0172
Yeast-dtt	0.0417	0.0436	0.0543	0.0074
Yeast-spo5	0.1722	0.1751	0.2006	0.0491
Yeast-spoem	0.3369	0.2869	0.1512	0.0125
Human Gene	0.0179	0.0202	0.0140	0.0130
Natural Scene	0.4045	0.1877	0.2473	0.1291
Movie	0.1697	0.1420	0.1544	0.1359
s-JAFFE	0.1300	0.1416	0.1183	0.0576
s-BU_3DFE	0.1790	0.1745	0.2229	0.0794
Average rank	2.97	2.77	3.23	1.03

10.26599/TST.2019.9010052.T11Table 11Experimental results on real-world datasets measured by the Clark $↓$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.1713	0.1141	0.1262	0.1021
Yeast-cdc	0.2178	0.2005	0.2857	0.1186
Yeast-elu	0.2494	0.2351	0.2386	0.1024
Yeast-diau	0.2890	0.3287	0.3393	0.0927
Yeast-heat	0.2534	0.2492	0.3438	0.0545
Yeast-spo	0.4690	0.5093	0.4928	0.1204
Yeast-cold	0.1979	0.2077	0.2392	0.0452
Yeast-dtt	0.1244	0.1366	0.1363	0.0181
Yeast-spo5	0.4098	0.4233	0.4501	0.0948
Yeast-spoem	0.5671	0.5117	0.3180	0.0176
Human Gene	1.0551	1.0739	1.3913	0.9660
Natural Scene	2.3735	2.1469	2.1008	2.0813
Movie	0.7311	0.6558	0.5541	0.4177
s-JAFFE	0.3860	0.5163	0.3324	0.2401
s-BU_3DFE	0.4379	0.4457	0.5710	0.2573
Average rank	2.87	2.93	3.20	1.00

10.26599/TST.2019.9010052.T12Table 12Experimental results on real-world datasets measured by the Canber $↓$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.5686	0.3792	0.4057	0.3315
Yeast-cdc	0.7528	0.6780	0.9368	0.3230
Yeast-elu	0.7263	0.7003	0.6932	0.3064
Yeast-diau	0.6489	0.7181	0.7262	0.1648
Yeast-heat	0.5499	0.5282	0.7659	0.1114
Yeast-spo	0.9634	1.0667	0.9917	0.2340
Yeast-cold	0.3574	0.3990	0.4401	0.0723
Yeast-dtt	0.2436	0.2709	0.2184	0.0296
Yeast-spo5	0.5826	0.5843	0.6554	0.1505
Yeast-spoem	0.7600	0.6749	0.3972	0.0249
Human Gene	6.2419	6.1261	9.6774	6.1365
Natural Scene	6.8702	5.5629	5.2013	5.2951
Movie	1.2497	1.2530	0.9741	0.7652
s-JAFFE	0.7755	1.0884	0.6243	0.5023
s-BU_3DFE	0.9382	0.9765	1.1770	0.5483
Average rank	2.93	2.93	3	1.13

10.26599/TST.2019.9010052.T13Table 13Experimental results on real-world datasets measured by KL-div $↓$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.0030	0.0015	0.0018	0.0012
Yeast-cdc	0.0064	0.0054	0.0111	0.0018
Yeast-elu	0.0094	0.0085	0.0080	0.0015
Yeast-diau	0.0198	0.0258	0.0273	0.0023
Yeast-heat	0.0193	0.0187	0.0367	0.0009
Yeast-spo	0.0952	0.1182	0.1047	0.0048
Yeast-cold	0.0201	0.0219	0.0290	0.0009
Yeast-dtt	0.0077	0.0092	0.0096	0.0001
Yeast-spo5	0.0836	0.0934	0.1144	0.0057
Yeast-spoem	0.2484	0.1858	0.0538	0.0003
Human Gene	0.0355	0.0355	0.0594	0.0285
Natural Scene	1.4482	0.6021	0.6874	0.5503
Movie	0.0955	0.1190	0.0994	0.0688
s-JAFFE	0.0619	0.1073	0.0473	0.0206
s-BU_3DFE	0.0925	0.0809	0.1625	0.0252
Average rank	2.83	2.83	3.33	1.00

10.26599/TST.2019.9010052.T14Table 14Experimental results on real-world datasets measured by the Cosine $↑$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.9971	0.9985	0.9981	0.9988
Yeast-cdc	0.9935	0.9946	0.9893	0.9982
Yeast-elu	0.9904	0.9913	0.9921	0.9985
Yeast-diau	0.9832	0.9778	0.9766	0.9978
Yeast-heat	0.9821	0.9826	0.9648	0.9990
Yeast-spo	0.9032	0.8815	0.8944	0.9952
Yeast-cold	0.9803	0.9785	0.9710	0.9990
Yeast-dtt	0.9925	0.9909	0.9903	0.9998
Yeast-spo5	0.9310	0.9268	0.9075	0.9945
Yeast-spoem	0.8293	0.8808	0.9556	0.9997
Human Gene	0.9647	0.9639	0.9420	0.9717
Natural Scene	0.4386	0.7315	0.6838	0.7690
Movie	0.9371	0.9225	0.9205	0.9364
s-JAFFE	0.9360	0.8879	0.9482	0.9791
s-BU_3DFE	0.9018	0.9147	0.8432	0.9739
Average rank	2.73	2.80	3.40	1.07

10.26599/TST.2019.9010052.T15Table 15Experimental results on real-world datasets measured by Intersec $↑$ .

Dataset	LDSVR	CPNN	AA-KNN	LTSA-LE
Yeast-alpha	0.9694	0.9789	0.9771	0.9815
Yeast-cdc	0.9495	0.9546	0.9378	0.9786
Yeast-elu	0.9480	0.9488	0.9502	0.9780
Yeast-diau	0.9136	0.9037	0.9024	0.9769
Yeast-heat	0.9119	0.9154	0.8753	0.9814
Yeast-spo	0.8226	0.8031	0.8170	0.9610
Yeast-cold	0.9108	0.9001	0.8887	0.9819
Yeast-dtt	0.9394	0.9325	0.9444	0.9926
Yeast-spo5	0.8278	0.8249	0.7994	0.9509
Yeast-spoem	0.6631	0.7131	0.8488	0.9875
Human Gene	0.9071	0.9097	0.8567	0.9095
Natural Scene	0.3522	0.5683	0.5008	0.5791
Movie	0.8203	0.7973	0.8183	0.8479
s-JAFFE	0.8603	0.7990	0.8817	0.9102
s-BU_3DFE	0.8210	0.8255	0.7771	0.9048
Average rank	2.93	2.80	3.20	1.07