Intro to Deep Learning Class

Introduction to Artificial Neural Networks and Deep Learning.

Deep Learning class as part of the MSc in Medical Statistics and Health Data Science, University of Bristol. Mar, 2023

Transcript

1. Deep Learning for Health and Life Sciences
2. Deep Learning for the Health and Life Sciences Valerio Maggio, PhD @l er i o m a g g i o v ma g g i o@a n a c o n d a .c o m
3. m e p u n Who? B a c k g r o un d i n C S P h D i n Ma c h i n e L ea r n i n g R e s e a r c h : M L /D L f o r B i o Me d i c i n e P y t h o n G e e k 
 D e v e l o p e r A d v o c a te _a t _ An a c o n da a l s o m e
4. Machine Learning “M a c h i n e l e a r n i n g i s t h e s ci e nc e (a nd a r t) o f p r o g ra m m i n g c o m p ut e rs so th e y c a n l e a rn fr o m d a t a” A u r él ie n G ér o n , Ha n ds -o n M a c h i n e L e a r n i n g w i t h Sc i ki t-L e a r n a nd T e n s o r Fl o w S o u r ce : bi t.l y /m l -s i m p l e -d ef i ni t i o n “(m l ) fo c us e s o n t e a c h i n g c o m p ut e rs h o w t o l e a r n w i t h o u t t h e n ee d t o b e p r o g r a m m e d f o r s p ec i f i c t a sk s” S. P a l & A . G u l l i , D e e p L e a r ni n g w it h K e ra s
5. Machine Learning M a c h i n e l e a r n i n g t e a c h es ma c h i n es h o w t o c a rr y o u t t a s k s b y t h e m s e l ve s. I t i s t h a t s i mp l e . T h e c o m p l e x i t y c o m e s w it h t h e d et a il s L o u i s P ed r o C o e l h o , B u i l d i ng Ma c h i n e L ea r ni n g S y s t e m s w i t h P y t h o n (a n d th a t ’s p r o b a bl y o n e o f t h e re a so n w h y y o u ’r e h e r e :)
6. The (Machine) Learning is about DATA D a t a a r e o ne o f th e m o s t i m p o r t a n t p a rt o f a m l s o l u ti o n I m p o rt a n c e: Da t a >> Mo d e l ? L e a r ni n g b y e xa m p l e s D a t a P r e p a ra t i o n i s c r uc i a l ! d a t a a l g o r i t h m s
7. BioMedicine: another data case ? C o n t e m p o r a ry Li f e S c i e n c e i s a b o u t d a t a r ec e nt a d v a n c e s i n s e q u e n ci n g t ec h s a nd i n s t r u m e n t s (e.g. “b io -i m a g e s ”) h u g e d a ta s e t s g e ne r a t e d a t i n c r e d i bl e p a ce f ro m h u m a n o b se r v a t i o n t o d a ta a n a l y s is c h e m o -i nf o rm a ti c s (d r u g d i sc o v e r y ) R e s e a r c h I m p a ct —> S o c ia l a n d H u m a n I mp a c t
8. Why Deep Learning, btw? S u b s et o f M L w / v e r y s pe c if i c m o d e l : (D e e p?) N e u ra l N e tw o r k s S ta t e o f t h e a r t T h e o r y ’50 / ’80 h w a cc e l e r a t i o n t o tr a i n (~n ew ) l e a rn i ng st r u c t ur e + c o m po s a b i l i t y (2018/23)
9. Ml / Dl basics in a NutShell
10. f e a t ur e s l a b e l s (r a w ) d a t a M L /D L 
 M o d e l T r a i n i n g T ra i ne d 
 M o d e l (u n s e e n ) d a t a T e s t P r e di c ti o n s Supervised learning s up e rv i si o n
11. f e a t ur e s l a b e l s (r a w ) d a t a M L /D L 
 M o d e l T r a i n i n g T ra i ne d 
 M o d e l (u n s e e n ) d a t a T e s t S i m i l a r i t i e s /l i k e l i h o o d UnSupervised learning
12. l a b e l s (r a w ) d a t a D L 
 M o d e l T r a i n i n g T ra i ne d 
 M o d e l (u n s e e n ) d a t a T e s t P r e di c ti o n s s up e rv i si o n f e a t ur e s Deep 
 Supervised learning
13. What Deep Learning is about…
14. Neural Networks A m u l t i-l a y e r f e e d-f o r w a r d n e ur a l n et w o r k c o mp o se d b y m u l t i pl e 
 d en s e (a.k.a. f u l l y -c o n n ec t ed ) h i d d e n l a y e r s a nd no n-l in e a r t r a n s f o rm a t i o ns
16. More details…
17. More details… ReLu | sigmoid | tanh
18. More details… Repeat for each layer…
19. More details… Image Classification Task
20. More details… S u m m a r y: 
 
 A (b a s i c ) D e e p N e u r a l N e t w o r k i s : • C o m p o s e d o f m u l t i p l e l a y e rs ; • E a c h L a y e r i s f (W x +b): • L in e a r m o d e l -> W x +b • f() is a n o n -l i n e a r f u n c t i o n G o a l o f t h e l ea r ni n g t a s k: fi n d i n g t h e va l ue s o f a l l 
 W s a n d bs (i.e. m o d e l p a r a m e te r s ) s o t h a t t h e c l a s s i f ic a ti o n e rr o r w o u l d b e m in i m u m.
21. Supervised Learning Loop l a b e l s (r a w ) d a t a M o d e l P a r a m e t e r s L o s s l o s s p r e d ic t i o n s
22. Supervised Training Loop breakdown & terminology (r a w ) D a t a - a .k.a. O b s e r v a t i o n s / I n p u t 
 I te m s a b o u t w h i c h w e w a n t t o p r e d i c t s o m e t h i n g. W e u s u a l l y w i l l d en o te o b s e r v a t i o n w i t h x . L a b e l s - a .k.a. T a r g e t s (i.e. G r o u n d T r u t h) 
 L a b el s c o rr e s p o n d i n g t o o b s e r v a t i o n s. T h e s e a r e u s u a l l y t h e t h i n g s b e i n g p re d ic t e d . F o l l o w i n g s t a n d a r d n o t a t i o n s i n M L /D L , w e w i l l u s e y t o r e f e r t o t h e s e . M o d e l f(x) = ˆy 
 A m a th e m a t i c a e x p r e s s i o n o r a f u n c t i o n t h a t t a k e s a n o b s e r v a t i o n x a n d p re d ic t s t h e v a l u e o f i t s t a r g e t l a b e l . P r e d i c t i o n s - a .k.a. Es t i m a t e s: V a l u e s o f t h e T a r g e t s g e n e r a t e d b y t h e m o d e l - u s u a l l y r e f e r r e d t o a s ˆy P a r a m e t e r s - a .k.a. W e i g h t s (i n D L t e r m i n o l o g y ) 
 P a r a m e t e r s o f t h e M o d e l . W e w i l l r e f e r t o t h e m u s i n g t h e w. L o s s F u n c t i o n L (y, ˆy): 
 F u n c t i o n t h a t c o m p a r e s h o w f a r o f f a p r e d i c t i o n i s f ro m i t s t a r g e t f o r o bs e r v a t i o n s i n t h e t r a i n i n g d a t a . T h e l o s s f u n c t i o n a s s i g n s a s c a l a r r e a l v a l u e c a l l e d t h e l o s s . T h e l o w e r t h e v a l u e o f t h e l o s s , t h e b e t t e r t h e m o d e l i s p r e d i c t i n g. T h e L o s s i s u s u a l l y r e f e r r e d t o a s L S o u r c e :D. R a o e t a l. - N a t ur a l L a n g u a g e P r o c e s s i n g w i th Py T o r c h , O ’R e i l l y 2019
24. Deep Learning in 
 Bioinformatics and BioMedicine A f e w e xa m p l e s
25. Diabetic Retinopathy from fundus retinal images DeepDR
26. DL for Digital Pathology Digital Pathology (Pipeline) Detect Tissue Region Transfer Learning
27. DL for Digital Pathology Transfer Learning
28. BioMedical Image Segmentation U-Net Note: AutoEncoder Model (Unsupervised Learning)
29. Toxicity Prediction using Deep Learning ToX21 Dataset DeepTox Pipeline
30. Now it’s time to switch to notebooks…
31. Materials and Practical R e p o si t o r y: 
 h t tp s://g i t h ub .c o m /l er i o m a gg i o /d ee p-l ea r ni n g -c l a s s m y b i nd e r