$30 off During Our Annual Pro Sale. View Details »

Explainability of Graph Neural Networks

wing.nus
March 17, 2023
210

Explainability of Graph Neural Networks

Graph Neural Networks (GNNs) are powerful models to exploit the high-order relationship between entities on graphs. Despite the superior performance, we have little knowledge about the explainability of GNNs. In this talk, we will introduce two themes of explainability, (1) Post-hoc explainability: Using an additional explainer method to explain a black-box model post hoc, but explanations could be unfaithful to the decision-making process of a model; (2) Intrinsic Interpretability: Incorporating a rationalization module into the model design, so as to transform a black-box to a white-box. We find causal theory is one promising solution and we will discuss interpretability and generalization.

Speaker's bio: Dr. Wang Xiang is a Professor in University of Science and Technology of China, where he is a member of Lab of Data Science. With his colleagues, students, and collaborators, he strives to develop trustworthy deep learning and artificial intelligence algorithms with better interpretability, generalization, and robustness. His research is motivated by, and contributes to, graph-structured applications in information retrieval (e.g., personalized recommendation), data mining (e.g., graph pre-training), security (e.g., fraud detection in fintech, information security in system), and multimedia (e.g., video question answering). His work has over 50 publications in top-tier conferences and journals. Over 10 papers have been featured in the most cited and influential list (e.g., KDD 2019, SIGIR 2019, SIGIR 2020, SIGIR 2021) and best paper finalist (e.g., WWW 2021, CVPR 2022). Moreover, He has served as the PC member for top-tier conferences including NeurIPS, ICLR, SIGIR and KDD, and the invited reviewer for prestigious journals including JMLR, TKDE, and TOIS.

wing.nus

March 17, 2023
Tweet

More Decks by wing.nus

Transcript

  1. Explainability of Graph Neural Networks
    王翔
    University of Science and Technology of China
    2023.03.02

    View Slide

  2. Why Graph & Graph Neural Networks (GNNs)?
    Background
    Protein Structure
    • Graph data are everywhere.
    Social Network Knowledge Graph Transaction Network
    • Graph neural networks (GNNs) are popular.
    • Powerful representation learning:
    • incorporate graph structure with node/edge features in an end-to-end fashion;
    • Impressive performance:
    • graph classification, node classification, link prediction, graph matching …
    ……

    View Slide

  3. Why Explainability?
    Motivation
    • GNNs work as a black box.
    Input
    Output
    • Call For Explainability!
    • Given a GNN model, how can we interpret to users the model outcome?
    • ”What knowledge should/does the model use to make decisions?”
    • Knowledge: insights for a particular audience into a specific problem.
    “The black box is an algorithm that tasks data & turns it into
    something. The issue that black boxes often find patterns without
    being able to explain their methodology.”
    It cannot be fully trusted, especially in applications on safety, security!

    View Slide

  4. Why Explainability?
    Type I: Post-hoc Explainability
    GNN Model 𝒇 Output Prediction "
    𝒚
    Input Graph 𝑮
    Core of GNNs
    Post-hoc Explainability
    Which fraction of the input graph is
    most influential to the model’s
    decision?
    Explanatory Subgraph 𝑮𝒔
    Input Graph 𝑮
    Output Prediction "
    𝒚
    Use an additional explainer method to exhibit what a model learns as a black box
    • Emulate the decision-making process of the target model

    View Slide

  5. Post-hoc Explainability
    Our Work 1: Explaining Graph Neural Networks
    Post-hoc Explainability
    Which fraction of the input graph is
    most influential to the model’s
    decision?
    Explanatory Subgraph 𝑮𝒔
    Input Graph 𝑮
    Output Prediction "
    𝒚
    Screen Graphs of an Image Subgraphs with Top Gradient & Attention Scores
    Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

    View Slide

  6. Post-hoc Explainability
    Our Work 1: Drawbacks of Existing GNN Explainers
    Drawbacks of Gradient- & Attention-based Explainers
    Our Goal
    • Causation:
    • We would like to identify the subgraph that may plausibly be the causal determinants of the model outcome
    • e.g., (standing, on, surfboard).
    • Conciseness:
    • We need concise explanations to avoid redundancy, considering the dependencies of interpretability across edges.
    • Spurious correlation:
    • Due to the confounding associations (human-related objects),
    some edges are wrongly highlighted;
    • e.g., (shorts, on, man), (man, has, hand).
    • Redundancy:
    • As the edge dependencies within the subgraph are
    ignored, edges might have no unique information;
    • e.g., (man, on, ocean) vs (man, ridding, waves).
    Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

    View Slide

  7. Post-hoc Explainability
    Our Work 1: Causal Screening
    • Key idea
    • Screening à a sequential decision process
    • Start from an empty set as the explanatory subgraph
    • Incrementally add one edge to the subgraph
    • One edge at each step.
    • Causal Attribution à do-intervention
    • Treatment: feed one edge with the previously selected edges into the model
    • Control: feed the previous selected edges into the model
    • Difference: the causal effect caused by the edge
    Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

    View Slide

  8. Post-hoc Explainability
    Our Work 1: Reinforced Causal Screening
    Our Goal
    • Learning to Perform Causal Screening:
    • Train a reinforcement learning agent, which learns to do intervene & explain individual predictions
    • Action à Do intervention
    • Reward à Causal Effect of Action
    Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

    View Slide

  9. Post-hoc Explainability
    Our Work 1: Evaluation of Explanations
    Wang et al. Reinforced Causal Explainer for Graph Neural Networks. TPAMI’2022

    View Slide

  10. Post-hoc Explainability
    Our Work 2: Out-of-distribution (OOD) Issue
    A Causal Look At Out-of-Distribution
    • Feature Removal:
    • Given a subgraph of interest 𝑮𝐬
    • Remove the complement 𝑮#
    𝒔
    • Quantify the mutual information between the subgraph & the
    prediction on the full graph;
    • One unobserved variable O is the confounder of 𝑮𝐬
    & 𝒀
    • Open backdoor path 𝑮𝐬
    ← 𝑫 → 𝒀
    • Introduce spurious correlations
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.

    View Slide

  11. Post-hoc Explainability
    Our Work 2: Front-door Adjustment
    Deconfounding
    • Feature Removal & In-filling:
    • Given a subgraph of interest
    • Remove the complement à make the subgraph off the data manifold
    • Imagine possible complements instead à make the “subgraph + complements” on the data manifold
    • Quantify the mutual information between the “in-filled surrogate” & the target prediction;
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.

    View Slide

  12. Post-hoc Explainability
    Our Work 2: Deconfounded Subgraph Evaluator (DSE)
    Wu et al. Deconfounding to Explanation Evaluation in Graph Neural Networks.
    • Post-hoc explanations may not be faithful to the original GNNs
    • Post-hoc explanations often do not make sense, or do not provide enough details to
    understand what a black-box model is doing
    • “Stop explaining black box machine learning models for high stakes decisions and use
    interpretable models instead” —— [Rudin, Nature Machine Intelligence’19]

    View Slide

  13. Why Explainability?
    Type II: Intrinsic Interpretability
    GNN Model 𝒇 Output Prediction "
    𝒚
    Input Graph 𝑮
    Core of GNNs
    Intrinsic Interpretability
    Prediction "
    𝒚 with Rationale Subgraph 𝑮𝒔
    Input Graph 𝑮
    Incorporate a rationalization module into the model design to make the predictions transparent
    • Intrinsically reason about causes and effect observable within a model

    View Slide

  14. Intrinsic Interpretability
    Our Work 3: Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    • Deep learning models like GNNs generally
    • Fail to exhibit interpretability
    • Fail to generalize out of distribution
    Solution:
    • Find causal feature 𝐶!

    View Slide

  15. Intrinsic Interpretability
    Our Work 3: Invariant Learning for Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022
    In general, only the pairs of input 𝐺 and label 𝑌 are observed during training, while
    neither causal feature 𝑪 nor shortcut feature 𝑺 is available.

    View Slide

  16. Intrinsic Interpretability
    Our Work 3: Invariant Learning for Rationale Discovery
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

    View Slide

  17. Intrinsic Interpretability
    Our Work 3: Invariance Condition
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

    View Slide

  18. Intrinsic Interpretability
    Our Work 3: Discovering Invariant Rationale (DIR)
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

    View Slide

  19. Intrinsic Interpretability
    Our Work 3: Discovering Invariant Rationale (DIR)
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

    View Slide

  20. Intrinsic Interpretability
    Our Work 3: Empirical Results of DIR
    Wu et al. Discovering Invariant Rationales For Graph Neural Networks. ICLR’2022

    View Slide

  21. Intrinsic Interpretability
    Our Work 4: Graph Contrastive Learning
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    Graph Augmentation
    Contrastive Learning
    Invariance Look
    If augmentations are
    too aggressive?
    Instance discrimination
    may fail …

    View Slide

  22. Intrinsic Interpretability
    Our Work 4: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.
    Graph Augmentation
    Contrastive Learning
    Invariance Look
    If augmentations are
    too aggressive?
    Instance discrimination
    may fail …
    Rationale Discovery
    Invariance Look
    Sufficiency & Independence Principles
    Rationale captures
    discriminative info.
    𝒚 is the label of “instance
    discrimination”

    View Slide

  23. Intrinsic Interpretability
    Our Work 4: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.

    View Slide

  24. Intrinsic Interpretability
    Our Work 2: Let IRD inspire Graph Pretraining
    Li et al. Let Invariant Rationale Discovery inspire Graph Contrastive Learning. ICML 2022.

    View Slide

  25. Summary
    Explainability of GNNS
    • Post-hoc explainability
    • Using an additional explainer method to explain a black-box model post hoc
    • Explanations could be unfaithful to the decision-making process of model
    • Intrinsic Interpretability
    • Incorporating a rationalization module into the model design, so as to transform a
    black-box to a white-box.
    • Causal theory is one promising solution!
    • Interpretability & Generalization

    View Slide

  26. Summary
    Explainability of GNNS
    Ø Towards Multi-grained Explainability for Graph Neural Networks (NeurIPS’2021)
    • https://github.com/Wuyxin/ReFine
    Ø Reinforced Causal Explainer for Graph Neural Networks (TPAMI’2022)
    • https://github.com/xiangwang1223/reinforced_causal_explainer
    Ø Discovering invariant rationales for graph neural networks (ICLR’2022)
    • https://github.com/Wuyxin/DIR-GNN
    Ø Let Invariant Rationale Discovery inspire Graph Contrastive Learning (ICML’2022)
    • https://github.com/lsh0520/RGCL
    Ø Causal Attention for Interpretable and Generalizable Graph Classification (KDD’2022)
    • https://github.com/yongduosui/CAL
    Ø Invariant Grounding for Video Question Answering (CVPR’2022, oral & best paper final list)
    • https://github.com/yl3800/IGV
    Ø Equivariant and Invariant Grounding for Video Question Answering (MM’2022)
    • https://github.com/yl3800/EIGV

    View Slide

  27. Thanks!

    View Slide