Do machines dream of atoms? Crippen’s logP as a quantitative molecular benchmark for explainable AI heatmaps

M2D2 on-line seminar 1.11.2022

Transcript

1. Do machines dream of atoms? Crippen’s logP as a quantitative molecular benchmark for explainable AI heatmaps Maria H. Rasmussen, Diana S. Christensen, and Jan H. Jensen Department of Chemistry, University of Copenhagen @janhjensen 1 M2D2 2022.11.01 Preprint: 10.26434/chemrxiv-2022-gnq3w
2. 2 Heatmaps: An idea from image classification arXiv:1703.01365
3. Molecular Heatmaps: atomic contributions to ML predictions Predicted activity decreases when “removed” Predicted activity increases when “removed” 3
4. 4 Uses of heatmaps Understand the ML model Right answer for the right reasons? Understand errors and improve models Build trust in model Understand the chemistry and guide discovery
5. Do expert chemist agree/find them useful? (non-numerical and imprecise) (hard to automate) Toy models where the ground truth heatmap is known (mainly for classification) (how to compare heatmaps?) But how to validate the heatmaps?
6. 6 Crippen logP atomic contributions as heatmap ground truth?
7. 7 Crippen logP atomic contributions as heatmap ground truth? Similar idea in
8. 8 2048 bit-ECFP4/Random Forest Trained on 150K molecules from ZINC / 5K test set XAI: “atom attribution from finger prints”-method developed by Riniker and Landrum* *DOI:10.1186/1758-2946-5-43
9. 9 2048 bit-ECFP4/Random Forest The approach seems to work
10. 10 2048 bit-ECFP4/Random Forest Finger print-adapted group truth: The sum of the logP contributions of each of the ten fragments is assigned to the atom
11. 11 2048 bit-ECFP4/Random Forest Finger print-adapted group truth improves agreement
12. 12 2048 bit-ECFP4/Random Forest Right answer for the wrong reason
13. 13 Comparing XAI methods Sheridan’s dummy atom approach is worse (dummy atoms introduce new on-bits)
14. 14 Comparing XAI methods Graph-convolution NN + remove atom approach Overall worse, but better for some molecules
15. 15 Comparing Heatmaps GCNN RF Ground Truth Overlap 0.20 logPpred = 3.47 Overlap 0.32 logPpred = 3.79 logP = 3.47 Overlap 0.76 logPpred = 1.21 Overlap 0.48 logPpred = 1.06 logP = 1.07
16. 16 Conclusions / Outlook Good heatmaps are possible for regression (especially if oracle heatmap is adapted to reflect molecular representation) A very simple model + XAI method gives good heatmaps (That doesn’t mean logP is “too easy”) (“atoms” don’t come natural to ML) New models/methods must beat this Heatmaps can guide adversarial models DOI: 10.26434/chemrxiv-2022-gnq3w