Privacy-preserving graph machine learning

Abstract

Most real-world data can be represented as graphs, capturing intricate relationships and dependencies among entities. This unique characteristics of graphs makes them applicable in various domains. A special family of machine learning models called graph neural networks (GNNs) are specially designed to handle graph data. In recent years, the widespread adoption of GNNs has revolutionized various analytical tasks involving graph data, such as node classification and link prediction.

However, concerns regarding the privacy vulnerabilities of these models have emerged, particularly in sensitive domains like healthcare, finance and recommender systems. This thesis explores the privacy implications of GNNs through a multi-faceted analysis encompassing several attacks, defense strategies and privacy-preserving frameworks.

The first investigation focuses on the susceptibility of GNNs to membership inference attacks. We propose several attacks and defenses to effectively mitigate these attacks while minimizing the impact on model performance. Our findings reveal that structural information rather than overfitting is the primary contributor to information leakage.

Subsequently, we propose a novel privacy-preserving framework, PrivGnn, leveraging knowledge distillation and two noise mechanisms, random subsampling, and noisy labeling to privately release GNN models while providing rigorous privacy guarantees. The theoretical analysis within the Rényi differential privacy framework is accompanied by empirical validation against baseline methods. We also show that our privately released GNN model is robust to membership inference attacks.

Furthermore, since model explanations have become a desirable outcome of modern machine learning models, we explore the privacy risks involved in releasing model explanations from GNNs. Specifically, we study the interplay between privacy and interpretability in GNNs through graph reconstruction attacks. We demonstrate how model explanations can facilitate the reconstruction of sensitive graph structures.

Various attack strategies are evaluated based on auxiliary information available to adversaries, with a proposed defense employing randomized response mechanisms to mitigate privacy leakage. Lastly, we develop attacks to systematically study the information leakage from latent representation in graph and tabular input data domains. We reveal the susceptibility of latent space representation learning to privacy attacks that reconstruct original input with high accuracy. Furthermore, we utilize these attacks as privacy auditors to evaluate the privacy guarantees of differentially private models on both graph and tabular data, providing valuable insights into the privacy risks associated with releasing latent space representations.

By comprehensively addressing these privacy challenges, this thesis contributes to a deeper understanding of the privacy implications of GNNs and provides practical insights into enhancing their privacy-preserving capabilities in real-world applications.