Explainability of Graph Neural Networks 王翔 University of Science and Technology of China 2023.03.02

Why Graph & Graph Neural Networks (GNNs)? Background Protein Structure • Graph data are everywhere. Social Network Knowledge Graph Transaction Network • Graph neural networks (GNNs) are popular. • Powerful representation learning: • incorporate graph structure with node/edge features in an end-to-end fashion; • Impressive performance: • graph classification, node classification, link prediction, graph matching … ……

Why Explainability? Motivation • GNNs work as a black box. Input Output • Call For Explainability! • Given a GNN model, how can we interpret to users the model outcome? • ”What knowledge should/does the model use to make decisions?” • Knowledge: insights for a particular audience into a specific problem. “The black box is an algorithm that tasks data & turns it into something. The issue that black boxes often find patterns without being able to explain their methodology.” It cannot be fully trusted, especially in applications on safety, security!

Why Explainability? Type I: Post-hoc Explainability GNN Model 𝒇 Output Prediction " 𝒚 Input Graph 𝑮 Core of GNNs Post-hoc Explainability Which fraction of the input graph is most influential to the model’s decision? Explanatory Subgraph 𝑮𝒔 Input Graph 𝑮 Output Prediction " 𝒚 Use an additional explainer method to exhibit what a model learns as a black box • Emulate the decision-making process of the target model

Post-hoc Explainability Our Work 1: Explaining Graph Neural Networks Post-hoc Explainability Which fraction of the input graph is most influential to the model’s decision? Explanatory Subgraph 𝑮𝒔 Input Graph 𝑮 Output Prediction " 𝒚 Screen Graphs of an Image Subgraphs with Top Gradient & Attention Scores Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

Post-hoc Explainability Our Work 1: Drawbacks of Existing GNN Explainers Drawbacks of Gradient- & Attention-based Explainers Our Goal • Causation: • We would like to identify the subgraph that may plausibly be the causal determinants of the model outcome • e.g., (standing, on, surfboard). • Conciseness: • We need concise explanations to avoid redundancy, considering the dependencies of interpretability across edges. • Spurious correlation: • Due to the confounding associations (human-related objects), some edges are wrongly highlighted; • e.g., (shorts, on, man), (man, has, hand). • Redundancy: • As the edge dependencies within the subgraph are ignored, edges might have no unique information; • e.g., (man, on, ocean) vs (man, ridding, waves). Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

Post-hoc Explainability Our Work 1: Causal Screening • Key idea • Screening à a sequential decision process • Start from an empty set as the explanatory subgraph • Incrementally add one edge to the subgraph • One edge at each step. • Causal Attribution à do-intervention • Treatment: feed one edge with the previously selected edges into the model • Control: feed the previous selected edges into the model • Difference: the causal effect caused by the edge Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

Post-hoc Explainability Our Work 1: Reinforced Causal Screening Our Goal • Learning to Perform Causal Screening: • Train a reinforcement learning agent, which learns to do intervene & explain individual predictions • Action à Do intervention • Reward à Causal Effect of Action Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

Post-hoc Explainability Our Work 1: Evaluation of Explanations Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

Post-hoc Explainability Our Work 2: Out-of-distribution (OOD) Issue A Causal Look At Out-of-Distribution • Feature Removal: • Given a subgraph of interest 𝑮𝐬 • Remove the complement 𝑮# 𝒔 • Quantify the mutual information between the subgraph & the prediction on the full graph; • One unobserved variable O is the confounder of 𝑮𝐬 & 𝒀 • Open backdoor path 𝑮𝐬 ← 𝑫 → 𝒀 • Introduce spurious correlations Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.

Post-hoc Explainability Our Work 2: Front-door Adjustment Deconfounding • Feature Removal & In-filling: • Given a subgraph of interest • Remove the complement à make the subgraph off the data manifold • Imagine possible complements instead à make the “subgraph + complements” on the data manifold • Quantify the mutual information between the “in-filled surrogate” & the target prediction; Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.

Post-hoc Explainability Our Work 2: Deconfounded Subgraph Evaluator (DSE) Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks. • Post-hoc explanations may not be faithful to the original GNNs • Post-hoc explanations often do not make sense, or do not provide enough details to understand what a black-box model is doing • “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead” —— [Rudin, Nature Machine Intelligence’19]

Why Explainability? Type II: Intrinsic Interpretability GNN Model 𝒇 Output Prediction " 𝒚 Input Graph 𝑮 Core of GNNs Intrinsic Interpretability Prediction " 𝒚 with Rationale Subgraph 𝑮𝒔 Input Graph 𝑮 Incorporate a rationalization module into the model design to make the predictions transparent • Intrinsically reason about causes and effect observable within a model

Intrinsic Interpretability Our Work 3: Rationale Discovery Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022 • Deep learning models like GNNs generally • Fail to exhibit interpretability • Fail to generalize out of distribution Solution: • Find causal feature 𝐶!

Intrinsic Interpretability Our Work 3: Invariant Learning for Rationale Discovery Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022 In general, only the pairs of input 𝐺 and label 𝑌 are observed during training, while neither causal feature 𝑪 nor shortcut feature 𝑺 is available.

Intrinsic Interpretability Our Work 3: Invariant Learning for Rationale Discovery Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

Intrinsic Interpretability Our Work 3: Invariance Condition Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

Intrinsic Interpretability Our Work 3: Discovering Invariant Rationale (DIR) Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

Intrinsic Interpretability Our Work 3: Discovering Invariant Rationale (DIR) Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

Intrinsic Interpretability Our Work 3: Empirical Results of DIR Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

Intrinsic Interpretability Our Work 4: Graph Contrastive Learning Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022. Graph Augmentation Contrastive Learning Invariance Look If augmentations are too aggressive? Instance discrimination may fail …

Intrinsic Interpretability Our Work 4: Let IRD inspire Graph Pretraining Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022. Graph Augmentation Contrastive Learning Invariance Look If augmentations are too aggressive? Instance discrimination may fail … Rationale Discovery Invariance Look Sufficiency & Independence Principles Rationale captures discriminative info. 𝒚 is the label of “instance discrimination”

Intrinsic Interpretability Our Work 4: Let IRD inspire Graph Pretraining Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.

Intrinsic Interpretability Our Work 2: Let IRD inspire Graph Pretraining Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.

Summary Explainability of GNNS • Post-hoc explainability • Using an additional explainer method to explain a black-box model post hoc • Explanations could be unfaithful to the decision-making process of model • Intrinsic Interpretability • Incorporating a rationalization module into the model design, so as to transform a black-box to a white-box. • Causal theory is one promising solution! • Interpretability & Generalization

Summary Explainability of GNNS Ø Towards Multi-grained Explainability for Graph Neural Networks (NeurIPS’2021) • https://github.com/Wuyxin/ReFine Ø Reinforced Causal Explainer for Graph Neural Networks (TPAMI’2022) • https://github.com/xiangwang1223/reinforced_causal_explainer Ø Discovering invariant rationales for graph neural networks (ICLR’2022) • https://github.com/Wuyxin/DIR-GNN Ø Let Invariant Rationale Discovery inspire Graph Contrastive Learning (ICML’2022) • https://github.com/lsh0520/RGCL Ø Causal Attention for Interpretable and Generalizable Graph Classification (KDD’2022) • https://github.com/yongduosui/CAL Ø Invariant Grounding for Video Question Answering (CVPR’2022, oral & best paper final list) • https://github.com/yl3800/IGV Ø Equivariant and Invariant Grounding for Video Question Answering (MM’2022) • https://github.com/yl3800/EIGV

Thanks!