Explainability of Graph Neural Networks

Graph Neural Networks (GNNs) are powerful models to exploit the high-order relationship between entities on graphs. Despite the superior performance, we have little knowledge about the explainability of GNNs. In this talk, we will introduce two themes of explainability, (1) Post-hoc explainability: Using an additional explainer method to explain a black-box model post hoc, but explanations could be unfaithful to the decision-making process of a model; (2) Intrinsic Interpretability: Incorporating a rationalization module into the model design, so as to transform a black-box to a white-box. We find causal theory is one promising solution and we will discuss interpretability and generalization.

Speaker's bio: Dr. Wang Xiang is a Professor in University of Science and Technology of China, where he is a member of Lab of Data Science. With his colleagues, students, and collaborators, he strives to develop trustworthy deep learning and artificial intelligence algorithms with better interpretability, generalization, and robustness. His research is motivated by, and contributes to, graph-structured applications in information retrieval (e.g., personalized recommendation), data mining (e.g., graph pre-training), security (e.g., fraud detection in fintech, information security in system), and multimedia (e.g., video question answering). His work has over 50 publications in top-tier conferences and journals. Over 10 papers have been featured in the most cited and influential list (e.g., KDD 2019, SIGIR 2019, SIGIR 2020, SIGIR 2021) and best paper finalist (e.g., WWW 2021, CVPR 2022). Moreover, He has served as the PC member for top-tier conferences including NeurIPS, ICLR, SIGIR and KDD, and the invited reviewer for prestigious journals including JMLR, TKDE, and TOIS.

Transcript

1. Explainability of Graph Neural Networks
   王翔
   University of Science and Technology of China
   2023.03.02
   

   View Slide

2. Why Graph & Graph Neural Networks (GNNs)?
   Background
   Protein Structure
   • Graph data are everywhere.
   Social Network Knowledge Graph Transaction Network
   • Graph neural networks (GNNs) are popular.
   • Powerful representation learning:
   • incorporate graph structure with node/edge features in an end-to-end fashion;
   • Impressive performance:
   • graph classification, node classification, link prediction, graph matching …
   ……
   

   View Slide

3. Why Explainability?
   Motivation
   • GNNs work as a black box.
   Input
   Output
   • Call For Explainability!
   • Given a GNN model, how can we interpret to users the model outcome?
   • ”What knowledge should/does the model use to make decisions?”
   • Knowledge: insights for a particular audience into a specific problem.
   “The black box is an algorithm that tasks data & turns it into
   something. The issue that black boxes often find patterns without
   being able to explain their methodology.”
   It cannot be fully trusted, especially in applications on safety, security!
   

   View Slide

4. Why Explainability?
   Type I: Post-hoc Explainability
   GNN Model 𝒇 Output Prediction "
   𝒚
   Input Graph 𝑮
   Core of GNNs
   Post-hoc Explainability
   Which fraction of the input graph is
   most influential to the model’s
   decision?
   Explanatory Subgraph 𝑮𝒔
   Input Graph 𝑮
   Output Prediction "
   𝒚
   Use an additional explainer method to exhibit what a model learns as a black box
   • Emulate the decision-making process of the target model
   

   View Slide

5. Post-hoc Explainability
   Our Work 1: Explaining Graph Neural Networks
   Post-hoc Explainability
   Which fraction of the input graph is
   most influential to the model’s
   decision?
   Explanatory Subgraph 𝑮𝒔
   Input Graph 𝑮
   Output Prediction "
   𝒚
   Screen Graphs of an Image Subgraphs with Top Gradient & Attention Scores
   Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022
   

   View Slide

6. Post-hoc Explainability
   Our Work 1: Drawbacks of Existing GNN Explainers
   Drawbacks of Gradient- & Attention-based Explainers
   Our Goal
   • Causation:
   • We would like to identify the subgraph that may plausibly be the causal determinants of the model outcome
   • e.g., (standing, on, surfboard).
   • Conciseness:
   • We need concise explanations to avoid redundancy, considering the dependencies of interpretability across edges.
   • Spurious correlation:
   • Due to the confounding associations (human-related objects),
   some edges are wrongly highlighted;
   • e.g., (shorts, on, man), (man, has, hand).
   • Redundancy:
   • As the edge dependencies within the subgraph are
   ignored, edges might have no unique information;
   • e.g., (man, on, ocean) vs (man, ridding, waves).
   Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022
   

   View Slide

7. Post-hoc Explainability
   Our Work 1: Causal Screening
   • Key idea
   • Screening à a sequential decision process
   • Start from an empty set as the explanatory subgraph
   • Incrementally add one edge to the subgraph
   • One edge at each step.
   • Causal Attribution à do-intervention
   • Treatment: feed one edge with the previously selected edges into the model
   • Control: feed the previous selected edges into the model
   • Difference: the causal effect caused by the edge
   Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022
   

   View Slide

8. Post-hoc Explainability
   Our Work 1: Reinforced Causal Screening
   Our Goal
   • Learning to Perform Causal Screening:
   • Train a reinforcement learning agent, which learns to do intervene & explain individual predictions
   • Action à Do intervention
   • Reward à Causal Effect of Action
   Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022
   

   View Slide

9. Post-hoc Explainability
   Our Work 1: Evaluation of Explanations
   Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022
   

   View Slide

10. Post-hoc Explainability
    Our Work 2: Out-of-distribution (OOD) Issue
    A Causal Look At Out-of-Distribution
    • Feature Removal:
    • Given a subgraph of interest 𝑮𝐬
    • Remove the complement 𝑮#
    𝒔
    • Quantify the mutual information between the subgraph & the
    prediction on the full graph;
    • One unobserved variable O is the confounder of 𝑮𝐬
    & 𝒀
    • Open backdoor path 𝑮𝐬
    ← 𝑫 → 𝒀
    • Introduce spurious correlations
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.
    

    View Slide

11. Post-hoc Explainability
    Our Work 2: Front-door Adjustment
    Deconfounding
    • Feature Removal & In-filling:
    • Given a subgraph of interest
    • Remove the complement à make the subgraph off the data manifold
    • Imagine possible complements instead à make the “subgraph + complements” on the data manifold
    • Quantify the mutual information between the “in-filled surrogate” & the target prediction;
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.
    

    View Slide

12. Post-hoc Explainability
    Our Work 2: Deconfounded Subgraph Evaluator (DSE)
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.
    • Post-hoc explanations may not be faithful to the original GNNs
    • Post-hoc explanations often do not make sense, or do not provide enough details to
    understand what a black-box model is doing
    • “Stop explaining black box machine learning models for high stakes decisions and use
    interpretable models instead” —— [Rudin, Nature Machine Intelligence’19]
    

    View Slide

13. Why Explainability?
    Type II: Intrinsic Interpretability
    GNN Model 𝒇 Output Prediction "
    𝒚
    Input Graph 𝑮
    Core of GNNs
    Intrinsic Interpretability
    Prediction "
    𝒚 with Rationale Subgraph 𝑮𝒔
    Input Graph 𝑮
    Incorporate a rationalization module into the model design to make the predictions transparent
    • Intrinsically reason about causes and effect observable within a model
    

    View Slide

14. Intrinsic Interpretability
    Our Work 3: Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    • Deep learning models like GNNs generally
    • Fail to exhibit interpretability
    • Fail to generalize out of distribution
    Solution:
    • Find causal feature 𝐶!
    

    View Slide

15. Intrinsic Interpretability
    Our Work 3: Invariant Learning for Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    In general, only the pairs of input 𝐺 and label 𝑌 are observed during training, while
    neither causal feature 𝑪 nor shortcut feature 𝑺 is available.
    

    View Slide

16. Intrinsic Interpretability
    Our Work 3: Invariant Learning for Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    

    View Slide

17. Intrinsic Interpretability
    Our Work 3: Invariance Condition
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    

    View Slide

18. Intrinsic Interpretability
    Our Work 3: Discovering Invariant Rationale (DIR)
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    

    View Slide

19. Intrinsic Interpretability
    Our Work 3: Discovering Invariant Rationale (DIR)
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    

    View Slide

20. Intrinsic Interpretability
    Our Work 3: Empirical Results of DIR
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    

    View Slide

21. Intrinsic Interpretability
    Our Work 4: Graph Contrastive Learning
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    Graph Augmentation
    Contrastive Learning
    Invariance Look
    If augmentations are
    too aggressive?
    Instance discrimination
    may fail …
    

    View Slide

22. Intrinsic Interpretability
    Our Work 4: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    Graph Augmentation
    Contrastive Learning
    Invariance Look
    If augmentations are
    too aggressive?
    Instance discrimination
    may fail …
    Rationale Discovery
    Invariance Look
    Sufficiency & Independence Principles
    Rationale captures
    discriminative info.
    𝒚 is the label of “instance
    discrimination”
    

    View Slide

23. Intrinsic Interpretability
    Our Work 4: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    

    View Slide

24. Intrinsic Interpretability
    Our Work 2: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    

    View Slide

25. Summary
    Explainability of GNNS
    • Post-hoc explainability
    • Using an additional explainer method to explain a black-box model post hoc
    • Explanations could be unfaithful to the decision-making process of model
    • Intrinsic Interpretability
    • Incorporating a rationalization module into the model design, so as to transform a
    black-box to a white-box.
    • Causal theory is one promising solution!
    • Interpretability & Generalization
    

    View Slide

26. Summary
    Explainability of GNNS
    Ø Towards Multi-grained Explainability for Graph Neural Networks (NeurIPS’2021)
    • https://github.com/Wuyxin/ReFine
    Ø Reinforced Causal Explainer for Graph Neural Networks (TPAMI’2022)
    • https://github.com/xiangwang1223/reinforced_causal_explainer
    Ø Discovering invariant rationales for graph neural networks (ICLR’2022)
    • https://github.com/Wuyxin/DIR-GNN
    Ø Let Invariant Rationale Discovery inspire Graph Contrastive Learning (ICML’2022)
    • https://github.com/lsh0520/RGCL
    Ø Causal Attention for Interpretable and Generalizable Graph Classification (KDD’2022)
    • https://github.com/yongduosui/CAL
    Ø Invariant Grounding for Video Question Answering (CVPR’2022, oral & best paper final list)
    • https://github.com/yl3800/IGV
    Ø Equivariant and Invariant Grounding for Video Question Answering (MM’2022)
    • https://github.com/yl3800/EIGV
    

    View Slide

27. Thanks!
    

    View Slide