Portable network resolving huge-graph isomorphism problem

We demonstrate the implementation framework for studying graph isomorphism problems using graph neural networks in figure 3. The framework consists of three parts. We firstly partition large graphs that cannot be processed by the neural network at once by using algorithm 1, and obtain the corresponding subgraph lists. And then, a lightweight graph neural network (LGNN) (upper part of figure 3) is used for feature extraction transformation to obtain the injective transformation of the network graph. Finally, the isomorphism results of the subgraph list can be further analyzed to obtain network graph similarity, subgraph isomorphism, and graph isomorphism.

Zoom In Zoom Out Reset image size

Algorithm 1
Input: Graph: $\mathcal{G} = (V,E)$, k: subgraph size threshold
Output: Ω: Subgraphs List
1: Initialize: st = 0, cur = 0
2: for d in Dord do:
3:  $cur + = d_n$; # The number of nodes corresponding to the current degree d
4:  if $cur-st \gt k$:
5:   $st = cur+1$;
6:   $\Omega_i = (Nodes_{(st, cur)}, Edges_{(st, cur)})$;
7:   $\Omega_i \longrightarrow \Omega$;
8:  end if
9: end for

We design a linear-complexity graph partition algorithm to reduce the complexity of GI. The subgraphs obtained from structural partition of isomorphic graphs are isomorphic, while nonisomorphic graphs have a higher probability to obtain different subgraphs. There are two primary reasons why the likelihood of obtaining different subgraphs from non-isomorphic graphs is greater. First, the inherent diversity in nonisomorphic graphs contributes to this probability. Since these graphs possess distinct structural features, the subgraphs extracted from them are more likely to exhibit different structures. Second, the independence of subgraphs plays a crucial role. As the differences between nonisomorphic graphs increase, the likelihood of extracting the same subgraph from both diminishes. This principle generally holds true for sets of graphs with varying structures, as the internal structural differences lead to a wider variety of subgraphs present within each graph. Notably, even when two non-isomorphic graphs share the same node degree sequence (i.e. each node has the same degree), significant structural differences may still exist, leading to the presence of different types of subgraphs within them.

In algorithm 1, we begin by sorting the nodes in the network graph based on their connectivity to derive Nodes and Edges, which represent the connections in the graph. Subsequently, we create a degree dictionary structured as $D\_ord: \{key: degree, value: {nodes}\}$. Each (key, value) pair in $D\_ord$ signifies nodes with a specific degree of connectivity. To simplify the process, we define the number of nodes with a particular degree of connectivity as $D\_ord: \{key: degree, value: d\_n\}$. These nodes are then segmented to form a list of subgraphs denoted as Ω. Starting from the node with the highest degree, we progressively add nodes until reaching the minimum required number of nodes in the subgraph. Nodes at the cut-off degree in the current subgraph are all included to prevent errors caused by random node truncation at that degree. The current subgraph $\Omega_i$ is composed of all nodes within it and their corresponding edges from the original graph. The limit on the number of nodes k may not be a fixed value; instead, it can be defined as a proportion of the total number of nodes in the original graph, such as one-third. This often reflects a trade-off between the accuracy and efficiency. While cutting off original edge connections to isolate a subgraph can lead to misjudgments in analysis, partitioning a large graph into smaller subgraphs can significantly accelerate computation time.

The key of this classification method lies in the principles of Preferred Attachment and Assortativity. Nodes with high degrees tend to connect with other high-degree nodes, while low-degree nodes tend to connect with other low-degree nodes. Dividing subgraphs based on the maximum degree can minimize edge cuts, reducing the probability of misclassification due to isomorphism. A higher threshold for the number of nodes in each subgraph enhances the accuracy of the overall network graph assessment. By disregarding node and edge characteristics, the time complexity of algorithm 1 depends primarily on the sorting algorithm, with a proper sorting algorithm achieving linear time complexity.

LGNN updates the feature vectors of each node to learn the network structure and features of other nodes around. The representation of node v in the mth hidden layer Npooling is given by

$$\begin{equation} h_{v}^{\left(m\right)} = C^{\left(m\right)}\left(h_{v}^{\left(m-1\right)}, A_{v}^{\left(m\right)}\left(\left\{h_{u}^{\left(m-1\right)} \mid u \in N\left(v\right)\right\}\right)\right),\end{equation} \tag{ 1 }$$

where N(v) denotes the set of all nodes in graph $\mathcal{G}$ except for the node v, $C^{(m)}$ denotes the combining function used by the node v in the mth and $(m-1)$th layers, which can be a concatenation or simple operation. Moreover, denote $A_{v}^{(m)}$ as the aggregation function used by the node v in the mth layer. We choose simple summation instead of excessive redundant operations to ensure the injectivity and efficiency of the aggregation and combination functions as shown in figure 4.

Zoom In Zoom Out Reset image size

Note that equation (1) requires more layers to obtain sufficient depth of expression when facing large network graphs, which is a limitation on both time and memory. So, we introduce an AntiWeightConv (AWConv) layer (M defaults to 1) after NPooling Layer to obtain a new node representation as (see figure 4)

$$\begin{eqnarray} x_{u}^{\ell}& = &x_{u}^{\ell-1}+\sigma\left(\left(W-W^{T}-\gamma I\right) x_{u}^{\ell-1}+\Phi\left(X^{\ell-1}, N_{u}\right)+b\right),\end{eqnarray} \tag{ 2 }$$

where σ is a monotonically non-decreasing activation function, γ is an intensity moderating factor, default to 0.1, $\Phi(X,N_{u})$ denotes the aggregation function of node in the u neighborhood, and $W-W^T\in R^{d\times d}$ is the antisymmetric weight matrix. The inverse weight matrix ensures the long-term dependency relationship between graph network in propagation and nodes, and prevent gradient explosion or disappearance.

Our model does not necessitate a deep architecture beyond A-DGNs [41]. However, the large-scale nodes can still impact the interdependencies among nodes, particularly in scenarios with a high density of neighboring nodes, where the influence of the nodes themselves will be excessively squeezed.

For the graph level representation, the graph level readout function GPooling is defined by

$$\begin{equation} h_{\mathrm{G}} = R^{l}\left(\left\{h_{v}^{l} \mid v \in V\right\}\right),\end{equation} \tag{ 3 }$$

where R is a simple permutation invariant function, such as the summation, or a graph level pooling function [27, 42]. We rank the expression ability of sum, mean, and max aggregators on multiple sets [10], and select sum aggregators that can capture complete multiple sets. In each update module, a full graph representation will be obtained. As the number of iterations increases, the node representation corresponding to the structure become more refined and global. The iterative information from all depths (N defaults to 3) of the model will be used by using an architecture similar to skip connection.

LGNN uses an isomorphism discriminator module named I-D to make judgments on the output of the model, which is defined by

$$\begin{equation} P_t = \frac{1}{\Delta}\textrm{RELU}\left(\Delta-\sum_{r_1,r_2}|r_1-r_2|\right),\end{equation} \tag{ 4 }$$

where r is the embedded representation of a network graph after it being processed by LGNN, Δ is a positive small number, i.e. $P_t = 1$ if $r_1 = r_2$, 0 otherwise. Different from previous graph classifications that only required processing a single network graph [10], LGNN can distinguish the embedding relationship between two network graphs and perform backpropagation based on isomorphic labels. The embedding results from GNN are differentiable for the calculation of isomorphism loss. LGNN is then regarded as an injective transformation of a graph, performing different operations after distinguishing the unique representations of two graphs.

Model convergence. Figure 5(a) demonstrates the efficacy of our model's recognition capabilities for network structures. By utilizing a single discriminator in the Siamese network structure [44], our test model achieved convergence within a remarkably short duration of only 7 epochs, where the two network outputs are concatenated and subsequently processed by a linear layer discriminator (L-D),

Zoom In Zoom Out Reset image size

Furthermore, upon introducing the isomorphism discriminator (I-D) with a predefined structure, our model exhibited exceptional performance by attaining a 100% accuracy rate across all graph pairs within the I-M dataset [43]. This achievement underscores the robust recognition capabilities of our model when tasked with identifying intricate network structures.

Training time. Figure 5(b) shows the average training time of the present model. On the I-M dataset, LGNN costs one half the time of GIN [10] and SAGE [18], and even 1/5 of GCN [36]. As the dataset increases, it only takes 1/100 to 1/10 of the time of other models on the R-B, R-M, and CLAB.

Recognition rate. Figure 5(c) shows superior performance of the present model in terms of isomorphism recognition rates across all four datasets compared to existing models such as GIN and GIUNet [12] within the framework [10], or GCN [36] in recognition accuracy on all datasets. Specifically, our model exhibits a 2% improvement over GCN on the I-M dataset and achieves performance enhancements over GIUNet, GIN, and GCN on the CLAB dataset by varying margins. The disparities in recognition rates observed in GATs [37, 38] and SAGE [38] may stem from the distinct weighting mechanisms employed for aggregating neighboring nodes within their architectures.

From these results, we exclude GATs and SAGE models in further experiments due to their performance discrepancies in the isomorphism recognition task. Our model's robust performance against baseline models underscores its efficacy and superiority in graph structure identification across diverse datasets.

Due to device limitations, we obtained up to L4 level datasets in experiments. Combining with Algorithm 1, the present model shows good performance on network graphs of various scales, related to its ability to recognize structures and generalizations.

Model parameters. Figure 5(d) showcases the relationship between parameter quantity and recognition rate for various neural network models on the L2 dataset. Our model stands out by considering both parameter quantity and recognition rate metrics, outperforming other models. Despite GIN potentially approaching our model's accuracy levels when employing 7 layers, the former significantly exceeds our model in terms of parameter count by almost 10k, as indicated in figure 5(d). On the other hand, GCN achieves a 21% accuracy rate with 9 layers; however, continued increases in layer depth result in complete unrecognizability, possibly attributed to excessive smoothness phenomena [45]. In the case of GATs, their multi-head mechanism necessitates a larger parameter count, with even a 3-layer network requiring 80k parameters. In contrast, our model achieves full detection of L2 isomorphism while maintaining a lower parameter count, highlighting its efficiency and effectiveness in comparison to these competing architectures.

Isomorphism performance. Table 3 shows that the present model has the shortest time on all datasets, requiring only about one-third of the time of other methods, especially on the L2 dataset, where accuracy and time exceed those of other methods. Figures 5(e) and (f) provide the more intuitive display of the model's performances. As shown in figure 5(e), GIN and GIUNet exhibit a sharp decline (by 6% and 33% respectively) in performance when dealing with dense L2 dataset (100 average degree of dataset). Instead, our model can maintain a stable level. In figure 5(f), as the size of the network graph increases from L1 to L4, the time complexity calculated by our model is approximately linear.

We used multiple metrics to evaluate the performance of the model on the dataset, such as recall, precision, F1 score (as a percentage), and accuracy (1-error). We present the performance of each model on I-M and CLA in respective tables 4 and 5.

The effectiveness of our model is denoted by bold numbers, which appear to indicate superior performance in several key aspects. Specifically, on the I-M dataset, our model outperforms others in recall, precision, F1 score, and 1-error evaluations. This underscores its comprehensive superiority in capturing various metrics of model performance on this dataset.

While on the CLAB dataset, SAGE and GAT models exhibit higher recall rates, there are concerns regarding their tendency to misclassify a substantial number of non-isomorphic network graphs as isomorphic. In contrast, our model has improved the accuracy and consistency on the CLAB dataset, showcasing a balanced and robust performance that minimizes misclassifications and ensures reliable identification of graph structures.

We conducted a comparison with models incorporating different pooling layers (pooling ratio of 0.1) on an isomorphic test dataset (EXPWL1) to demonstrate the simplicity and effectiveness of our model.

The detailed results presented in table 6 highlight the remarkable performance of the present model without the need for additional pooling operations. It not only boasts the shortest processing time but also attains the highest accuracy in testing graph isomorphism, achieving an impressive recognition rate of 95.3% across the entire dataset.

Moreover, the incorporation of common pooling operations only marginally enhances performance on the isomorphism task. This suggests that additional pooling operations are not necessary for optimal performance in graph isomorphism tasks, underscoring the efficiency and effectiveness of the present model in accurately identifying graph structures without the need for cumbersome additional operations.

We further compared the accuracy of our partitioning algorithm with other model methods to verify that partitioning methods can increase the reliability of judgments.

The analysis conducted on the enhancement effect of the partition algorithm based on GraphSAGE, GAT, and GCN models on the I-M and L1 datasets reveals promising outcomes. By utilizing algorithm 1 to augment the model results, significant improvements in accuracy (see table 7) were observed across different models and datasets: (1) GCN's accuracy on the I-M dataset improved from 98.68% to 99.99%, while achieving 98% accuracy on the L1 dataset. (2) GraphSAGE exhibited an increase in accuracy from 12.22% to 17.86% on the I-M dataset. (3) GAT's accuracy on the I-M dataset saw an improvement from 12.22% to 12.77%. (4) GATv2 achieved a recognition rate of 33% on the L1 dataset with the enhancement from the partition algorithm.

Note that although partitioning was applied to the L1 dataset, GraphSAGE and GAT did not show improvement initially due to limitations in analyzing isomorphism of larger network graphs. However, adjusting the threshold of the partitioning algorithm on the L1 dataset to 30 nodes led to GraphSAGE achieving a recognition rate of 39%.

These results highlight the effectiveness and rationality of algorithm 1 in tackling graph isomorphism challenges and enhancing the performance of neural network models in analyzing complex network structures. By leveraging the partition algorithm, improvements in accuracy were observed across different models and datasets, showcasing the potential of this approach in boosting the capabilities of graph analysis models.

We finally evaluated the model for two downstream tasks: graph classification and node classification in tables 8 and 9 respectively.

The baseline for graph classification in table 8 includes the following three different classes: (1) WL subtree kernel [47], where C-SVM [53] is used as the classifier; (2) Deep learning architecture including diffusion convolutional neural network (DCNN) [48], patch san [54], and deep graph CNN (DGCNN) [42]; (3) Anonymous Walk Embedding (AWL) [49]. The present model performs similarly to classical network models and their variants in graph classification tasks, especially with impressive accuracy on the PTC dataset (see figure 6). Our model's accuracy on the PTC dataset is over 20% higher than the benchmark, 25% higher than GIN, 20% higher than GCNs, and 25% higher than GATs.

Zoom In Zoom Out Reset image size

In the node classification task, we use three citation graph tasks, i.e. Cora, Citeseer and Pubmed from Planetoid [55] and Arixv [56]. We compared the present model in table 9 with other popular GNNs and graph kernels, including Graph Convolutional Network (GCN) [36], GraphHomomorphic Network (GIN) [10], Graph Attention Network (GAT) [37], GraphSAGE [18]. We also compared two methods that combine GNN and dt: BGNN [51] and XGraphBoost [50]. Our model performs similarly to the optimal model in the node classification task mentioned above.