Few-Shot Graph Classification with Structural-Enhanced Contrastive Learning for Graph Data Copyright Protection

Figure 2 illustrates the framework of our proposed method. Two complementary classification tasks are performed simultaneously to learn the main encoder ${ℱ}_{\theta }(\cdot )$ , which is a GNN for projecting a graph into an embedding. The first learning module is metric-based meta-learning, which utilizes explicit label information to generate the graph embedding and compute the similarity between the support set and query set. The second learning module is contrastive learning, which is a self-supervised instance-level classification task to improve the representation result. For self-supervised learning, we design a strategy to generate a pair of positive and negative augmentation views of the input graph automatically, which contributes to data copyright protection by mitigating the risk of unauthorized reproduction or misuse of the original data.

10.26599/TST.2023.9010071.F002 Fig. 2Overview of SE-GCL. The framework consists of two main processes: graph meta-learning and contrastive learning. Given a support set of input graphs, we use a graph encoder to extract robust feature representation and derive reliable prototypes for each class. The Wasserstein metric measures the similarity between the query graph and the prototype. Further, we impose the contrastive loss on the query set to improve the model’s generalizability. The complete workflow of all modules is an end-to-end solution. More details could be in the section of the proposed framework.

During meta-learning, the main encoder ${ℱ}_{\theta }(\cdot )$ maps each graph into a latent representation as its graph embedding $h_{G_i}={ℱ}_{\theta }(G_i)$ . Specifically, GNNs compute graph embedding via a message-passing framework:

(1) $$h_u^{(l+1)}=\text{COM}(h_u^{(l)},\left[\text{AGG}\left(\{h_{u^{\prime }}^{(l)}|\forall u^{\prime }\in U^{\prime }\}\right)\right])$$

(2) $$h_G=\text{READOUT}(h_u^{(l)}|\forall u\in U)$$

where $h_u^{(l)}$ denotes the embedding of node u at the l-th GNN layer; $U{}^{\prime }$ is the neighbor set of node u; AGG $(\cdot )$ is the neighbor aggregation function; COM $(\cdot )$ is the combination function; and READOUT $(\cdot )$ is the graph-level pooling function. Then all support graph embeddings in the same class $y_n$ are aggregated into one prototype representation $z_n$ by computing the average, which is formulated as

(3) $$z_n=\frac{1}{K}\underset{i=1}{\overset{K}{\sum }}{h}_{G_i}(G_i\in D_S(G,y_n),n\in [1,N])$$

To predict the label of the query graph, the similarity between query graph embedding and the prototype representation is measured by the p-th Wasserstein distance following the work in Ref. [ 36 ], which is the optional cost of moving mass between two graph embeddings. The classification loss ${ℒ}_{\mathrm{Meta}}$ is defined as the average cross entropy between true labels and predictions based on the similarity, which can be formulated as

(4) $${ℒ}_{\mathrm{Meta}}(D_S,D_Q,\theta )=$$ $$-{\displaystyle \frac{1}{M}}{\displaystyle \underset{(G,y)\in D_Q}{\sum }}\log {\displaystyle \frac{{\mathrm{e}}^{\text{sim}({ℱ}_{\theta }(G),z_y)}}{{\sum }_{i=1}^N{\mathrm{e}}^{\text{sim}({ℱ}_{\theta }(G),{\text{𝒛}}_i)}}}$$

where sim denotes the Wasserstein similarity metric.

Because contrastive learning can maximize the agreement between the input data and its positive view while minimizing the agreement with the negative view, two automatic augmentations are employed to generate a pair of differentiable views for the respective goals, which reduces the need to unauthorized operations of the original data. Expressly, the positive augmentation operation preserves the original topology of the sample graph $G_i$ and masks all the node features to form a positive view $G_i^{\mathrm{mask}}$ , which aims to mediate the overwhelming of the node features over the graph structure information in the representation learning. On the other hand, the negative augmentation operation generates a negative view $G_i^{\mathrm{neg}}$ by random node-dropping and edge-perturbation. Both operations follow an i.i.d. uniform distribution with node-dropping ratio $\eta$ and edge-perturbation ratio $1-\eta$ . For edge-perturbation, it randomly drops $1-\eta$ existing edges, then adds the same amount of random edges back into $G_i$ . To form $G_i^{\mathrm{neg}}$ as a small subgraph from $G_i$ with a few noisy edges, $\eta$ is set to 0.8 by default. Moreover, it is stated in Ref. [ 23 ] that the structural information of graph data consists of both local and global dimensions, which means some attributes of a graph depend on the substructure of the graph while some consider the global structure more. As generalization is the main challenge for meta-learning to test novel domains, randomly treating a small subgraph as the negative example helps predictive models generalize beyond the limited training data. It should be noted that the negative view of one sample graph is also treated as the negative view of the rest samples (i.e., for a query set containing M samples, there are M negative views for each sample graph). Introduced in Ref. [ 37 ], we apply a momentum encoder ${ℱ}_{\omega }(\cdot )$ for projecting the contrastive views, which behaves similarly as the main encoder as its parameter $\omega$ is a moving average of $\theta$ . Given ${ℱ}_{\theta }(G_i)$ , the contrastive loss aims to maximize its agreement with ${ℱ}_{\omega }(G_i^{\mathrm{mask}})$ while minimizing the agreement with all the negative views ${ℱ}_{\omega }(G_j^{\mathrm{neg}}),j\in M$ , which can be formulated as

(5) $${ℒ}_{\mathrm{con}}(D_S,D_Q,\theta ,\omega ,\eta )=$$ $$-{\displaystyle \frac{1}{M}}{\displaystyle \underset{G\in D_Q}{\sum }}\log {\displaystyle \frac{{\text{e}}^{\text{sim}({ℱ}_{\theta }(G),{ℱ}_{\omega }(G^{\text{mask}}))}}{{\sum }_{j=1}^M{\text{e}}^{\text{sim}({ℱ}_{\theta }(G),{ℱ}_{\omega }(G_j^{\text{neg}}))}}}$$

where M denotes the size of the query set, and $\eta$ is the perturbation ratio. By minimizing ${ℒ}_{\text{con}}$ w.r.t. $\theta$ , we force the main encoder ${ℱ}_{\theta }(\cdot )$ to maintain the complete structural information in the embedding and produce more generalized prototypical networks. Thus, the overall loss is the combination of the classification loss and the contrastive loss:

(6) $${ℒ}_{\text{total}}={ℒ}_{\mathrm{Meta}}(D_S,D_Q,\theta )+\beta {ℒ}_{\text{con}}(D_S,D_Q,\theta ,\omega ,\eta )$$

where $\beta$ is a hyper-parameter that balances two terms. The detailed learning process is described in Algorithm 1. And all notations used in this paper are listed in Table 1 .