Sort:
Open Access Issue
Improving Few-Shot Named Entity Recognition with Causal Interventions
Big Data Mining and Analytics 2024, 7(4): 1375-1395
Published: 04 December 2024
Abstract PDF (1.2 MB) Collect
Downloads:8

Few-shot Named Entity Recognition (NER) systems are designed to identify new categories of entities with a limited number of labeled examples. A major challenge encountered by these systems is overfitting, particularly pronounced in comparison to tasks with ample samples. This overfitting predominantly stems from spurious correlations, a consequence of biases inherent in the selection of a small sample set. In response to this challenge, we introduce a novel approach in this paper: a causal intervention-based method for few-shot NER. Building upon the foundational structure of prototypical networks, our method strategically intervenes in the context to obstruct the indirect association between the context and the label. For scenarios restricted to 1-shot, where contextual intervention is not feasible, our method utilizes incremental learning to intervene at the prototype level. This not only counters overfitting but also aids in alleviating catastrophic forgetting. Additionally, to preliminarily classify entity types, we employ entity detection methods for coarse categorization. Considering the distinct characteristics of the source and target domains in few-shot tasks, we introduce sample reweighting to aid in model transfer and generalization. Through rigorous testing across multiple benchmark datasets, our approach consistently sets new state-of-the-art benchmarks, underscoring its efficacy in few-shot NER applications.

Open Access Issue
Restage: Relation Structure-Aware Hierarchical Heterogeneous Graph Embedding
Tsinghua Science and Technology 2025, 30(1): 198-214
Published: 11 September 2024
Abstract PDF (6.1 MB) Collect
Downloads:10

Heterogeneous graphs contain multiple types of entities and relations, which are capable of modeling complex interactions. Embedding on heterogeneous graphs has become an essential tool for analyzing and understanding such graphs. Although these meticulously designed methods make progress, they are limited by model design and computational resources, making it difficult to scale to large-scale heterogeneous graph data and hindering the application and promotion of these methods. In this paper, we propose Restage, a relation structure-aware hierarchical heterogeneous graph embedding framework. Under this framework, embedding only a smaller-scale graph with existing graph representation learning methods is sufficient to obtain node representations on the original heterogeneous graph. We consider two types of relation structures in heterogeneous graphs: interaction relations and affiliation relations. Firstly, we design a relation structure-aware coarsening method to successively coarsen the original graph to the top-level layer, resulting in a smaller-scale graph. Secondly, we allow any unsupervised representation learning methods to obtain node embeddings on the top-level graph. Finally, we design a relation structure-aware refinement method to successively refine the node embeddings from the top-level graph back to the original graph, obtaining node embeddings on the original graph. Experimental results on three public heterogeneous graph datasets demonstrate the enhanced scalability of representation learning methods by the proposed Restage. On another large-scale graph, the speed of existing representation learning methods is increased by up to eighteen times at most.

Issue
Overlapping community detection model based on a modularity-aware graph autoencoder
Journal of Tsinghua University (Science and Technology) 2024, 64(8): 1319-1329
Published: 15 August 2024
Abstract PDF (4.4 MB) Collect
Downloads:21
Objective

In the ever-expanding field of network science, the abstraction of complex entity relationships into network structures provides a foundation for understanding real-world interactions.The discovery of communities within these networks plays a pivotal role in identifying clusters of closely interconnected nodes.This process reveals latent patterns and functionalities inherent in the intricate fabric of reality, proving invaluable for tracking dynamic network behaviors and assessing community influences.These influences span a range of phenomena, from rumor propagation to virus outbreaks and tumor evolution.A notable characteristic of these communities is their overlapping nature, with participants often straddling multiple community boundaries.This characteristic adds an additional layer of complexity to the exploration of network structures, making the discovery of overlapping communities imperative for a comprehensive understanding of network structures and functional dynamics.

Methods

Within the realm of network science, network representation learning algorithms have significantly enriched the pursuit of community discovery.These algorithms adeptly transform complex network information into lower-dimensional vectors, effectively maintaining the underlying network structure and attribute information.Such representations prove invaluable for subsequent graph processing tasks, including but not limited to link prediction, node classification, and community discovery.Among these algorithms, the graph autoencoder model is a prominent representative, demonstrating efficiency in learning network embeddings and finding applications in diverse community discovery tasks.However, a limitation inherent in traditional graph autoencoder models is their predominant focus on local node-edge reconstruction.This focus often overlooks the crucial influence of community structure, particularly in scenarios featuring overlapping nodes across multiple communities.This inherent challenge makes it difficult to precisely determine node affiliations and community distributions.To address this issue, we introduce an innovative unsupervised modularity-aware graph autoencoder model (GAME) designed for overlapping community discovery.The model incorporates an efficient modularity maximization loss function into the graph autoencoder framework.This ensures the preservation of community structure throughout the network embedding process.The modularity-aware loss is meticulously reconstructed to facilitate the update of encoder parameters, thereby improving the model performance in overlapping community discovery tasks.We harness the resulting community membership matrix to probabilistically assign communities to nodes.

Results

The efficacy of the proposed GAME model was rigorously evaluated across six diverse social network datasets (Facebook 348, Facebook 414, Facebook 686, Facebook 698, Facebook 1684, and Facebook 1912), with node counts ranging from 60-800.Additionally, assessments were conducted on four collaborator network datasets (Computer Science, Engineering, Chemistry, and Medicine) featuring node counts ranging from 1.4×104 to 6.4×104.Comparative analyses with seven prevalent overlapping community discovery methods, encompassing both traditional and graph autoencoder-based algorithms, demonstrated a noteworthy 2.1% improvement under the normalized mutual information (NMI) evaluation index.This performance enhancement substantiated the tangible advantages and effectiveness of the proposed GAME model.

Conclusions

The integration of an efficient modularity maximization loss function into the graph autoencoder model, as demonstrated by the GAME model, successfully addresses the conventional limitations of graph autoencoders.These models often prioritize the reconstruction of local node connections during community discovery tasks, often overlooking the overarching structure of the community, particularly when confronted with overlapping nodes.The experimentally validated performance boost underscores the GAME model's efficacy in navigating the complexities of overlapping community discovery compared to mainstream methods.However, it is worth noting that the model's reliance on substantial memory resources can become a challenge when handling datasets that combine network structure and node attributes.This is especially apparent in scenarios with small attribute networks (N≤800), where the model exhibits insensitivity to the threshold ρ variation.Future work will focus on refining the model to mitigate these challenges and ensure optimal performance across a broader spectrum of real-world scenarios.

Open Access Issue
An Optimization Algorithm for Service Composition Based on an Improved FOA
Tsinghua Science and Technology 2015, 20(1): 90-99
Published: 12 February 2015
Abstract PDF (1.1 MB) Collect
Downloads:10

Large-scale service composition has become an important research topic in Service-Oriented Computing (SOC). Quality of Service (QoS) has been mostly applied to represent nonfunctional properties of web services and to differentiate those with the same functionality. Many studies for measuring service composition in terms of QoS have been completed. Among current popular optimization methods for service composition, the exhaustion method has some disadvantages such as requiring a large number of calculations and poor scalability. Similarly, the traditional evolutionary computation method has defects such as exhibiting slow convergence speed and falling easily into the local optimum. In order to solve these problems, an improved optimization algorithm, WS_FOA (Web Service composition based on Fruit Fly Optimization Algorithm) for service composition, was proposed, on the basis of the modeling of service composition and the FOA. Simulated experiments demonstrated that the algorithm is effective, feasible, stable, and possesses good global searching ability.

Open Access Issue
Multiple-Instance Learning with Instance Selection via Constructive Covering Algorithm
Tsinghua Science and Technology 2014, 19(3): 285-292
Published: 18 June 2014
Abstract PDF (461.6 KB) Collect
Downloads:19

Multiple-Instance Learning (MIL) is used to predict the unlabeled bags’ label by learning the labeled positive training bags and negative training bags. Each bag is made up of several unlabeled instances. A bag is labeled positive if at least one of its instances is positive, otherwise negative. Existing multiple-instance learning methods with instance selection ignore the representative degree of the selected instances. For example, if an instance has many similar instances with the same label around it, the instance should be more representative than others. Based on this idea, in this paper, a multiple-instance learning with instance selection via constructive covering algorithm (MilCa) is proposed. In MilCa, we firstly use maximal Hausdorff to select some initial positive instances from positive bags, then use a Constructive Covering Algorithm (CCA) to restructure the structure of the original instances of negative bags. Then an inverse testing process is employed to exclude the false positive instances from positive bags and to select the high representative degree instances ordered by the number of covered instances from training bags. Finally, a similarity measure function is used to convert the training bag into a single sample and CCA is again used to classification for the converted samples. Experimental results on synthetic data and standard benchmark datasets demonstrate that MilCa can decrease the number of the selected instances and it is competitive with the state-of-the-art MIL algorithms.

Total 5