Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intro to Deep Learning Class

Intro to Deep Learning Class

Introduction to Artificial Neural Networks and Deep Learning.

Deep Learning class as part of the MSc in Medical Statistics and Health Data Science, University of Bristol. Mar, 2023

Valerio Maggio

March 15, 2023
Tweet

More Decks by Valerio Maggio

Other Decks in Education

Transcript

  1. Deep Learning for
    Health and Life Sciences

    View Slide

  2. Deep Learning for the
    Health and Life Sciences
    Valerio Maggio, PhD
    @l
    er
    i
    o
    m
    a
    g g
    i
    o
    v
    ma
    g
    g
    i
    o@a
    n
    a
    c
    o
    n
    d
    a
    .c
    o
    m

    View Slide

  3. m
    e
    p
    u n
    Who?
    B
    a
    c
    k
    g r
    o
    un
    d
    i
    n
    C
    S


    P
    h
    D i
    n Ma
    c
    h
    i
    n
    e
    L
    ea
    r
    n i
    n
    g

    R
    e
    s
    e
    a
    r
    c
    h
    :
    M
    L
    /D
    L
    f
    o
    r B
    i
    o
    Me
    d
    i
    c
    i
    n
    e

    P
    y
    t
    h
    o
    n G
    e
    e
    k 

    D
    e v
    e
    l
    o
    p
    e
    r A d
    v
    o
    c
    a
    te
    _a
    t
    _
    An
    a
    c o
    n
    da
    a
    l
    s
    o
    m
    e

    View Slide

  4. Machine Learning
    “M
    a
    c
    h
    i n
    e
    l
    e
    a
    r
    n
    i n
    g
    i
    s
    t
    h
    e
    s
    ci
    e
    nc
    e (a
    nd a
    r
    t)
    o
    f
    p
    r
    o
    g
    ra
    m
    m
    i
    n
    g
    c
    o
    m
    p
    ut
    e
    rs so th
    e
    y
    c
    a
    n l
    e
    a
    rn fr
    o
    m
    d
    a
    t
    a”
    A
    u r
    él
    ie
    n
    G
    ér
    o
    n
    ,
    Ha
    n
    ds
    -o
    n
    M
    a
    c
    h
    i
    n
    e
    L
    e
    a
    r
    n
    i
    n
    g
    w
    i
    t
    h Sc
    i
    ki
    t-L
    e
    a
    r
    n
    a
    nd T
    e
    n
    s
    o
    r
    Fl
    o
    w
    S
    o
    u
    r
    ce
    :
    bi
    t.l
    y
    /m
    l
    -s
    i m
    p
    l
    e
    -d
    ef
    i
    ni
    t
    i o
    n
    “(m
    l
    )
    fo
    c
    us
    e
    s
    o
    n
    t
    e
    a
    c
    h
    i
    n
    g
    c
    o
    m
    p
    ut
    e
    rs h
    o
    w
    t
    o
    l
    e
    a
    r
    n
    w
    i
    t
    h
    o
    u
    t
    t
    h
    e
    n
    ee
    d
    t
    o
    b
    e
    p
    r
    o
    g
    r
    a
    m
    m
    e
    d f
    o
    r
    s
    p
    ec
    i
    f
    i
    c
    t
    a
    sk
    s”
    S.
    P a
    l &
    A
    .
    G
    u
    l
    l
    i
    ,
    D
    e
    e
    p
    L
    e
    a
    r
    ni
    n
    g
    w
    it
    h
    K
    e
    ra
    s

    View Slide

  5. Machine Learning
    M
    a
    c
    h
    i n
    e
    l
    e
    a
    r
    n
    i
    n
    g
    t
    e
    a
    c
    h
    es ma
    c
    h
    i
    n
    es h
    o
    w
    t
    o
    c
    a
    rr
    y
    o
    u
    t
    t
    a
    s
    k
    s
    b
    y
    t
    h
    e
    m
    s
    e
    l
    ve
    s.


    I
    t
    i
    s
    t
    h
    a
    t
    s
    i
    mp
    l
    e
    .


    T
    h
    e c
    o
    m
    p
    l
    e
    x i
    t
    y
    c
    o
    m
    e
    s
    w
    it
    h
    t
    h
    e
    d
    et
    a
    il
    s
    L
    o
    u
    i
    s
    P
    ed
    r
    o
    C
    o
    e
    l
    h
    o
    ,
    B
    u
    i
    l
    d
    i
    ng Ma
    c
    h
    i
    n
    e
    L
    ea
    r
    ni
    n
    g
    S
    y
    s
    t
    e
    m
    s
    w
    i
    t
    h P y
    t
    h
    o
    n
    (a
    n d th
    a
    t
    ’s
    p
    r
    o
    b
    a
    bl
    y
    o
    n
    e
    o
    f
    t
    h
    e re
    a
    so
    n
    w
    h
    y
    y
    o
    u
    ’r
    e
    h
    e r
    e :)

    View Slide

  6. The (Machine) Learning is about DATA
    D
    a
    t
    a
    a
    r
    e
    o
    ne o
    f th
    e
    m
    o
    s t i
    m
    p
    o
    r
    t
    a
    n
    t
    p
    a
    rt o
    f a
    m
    l
    s
    o
    l
    u
    ti
    o
    n

    I
    m
    p
    o
    rt
    a
    n
    c
    e:
    Da
    t
    a
    >>
    Mo
    d
    e l
    ?


    L
    e a
    r
    ni
    n
    g
    b
    y
    e
    xa
    m
    p
    l
    e
    s

    D
    a
    t
    a
    P
    r
    e
    p
    a
    ra
    t
    i o
    n
    i
    s
    c
    r
    uc
    i
    a
    l
    !
    d
    a
    t
    a
    a
    l
    g
    o
    r i
    t
    h
    m
    s

    View Slide

  7. BioMedicine: another data case ?
    C
    o
    n
    t
    e
    m
    p
    o
    r
    a
    ry Li
    f
    e
    S
    c i
    e
    n c
    e
    i
    s
    a
    b
    o
    u
    t
    d
    a
    t
    a


    r
    ec
    e
    nt a
    d
    v
    a
    n
    c
    e s i
    n s e
    q
    u e
    n
    ci
    n
    g
    t
    ec
    h
    s
    a
    nd i
    n
    s
    t
    r
    u
    m
    e
    n
    t
    s
    (e.g. “b
    io
    -i
    m a
    g
    e
    s
    ”)


    h
    u g
    e
    d
    a
    ta
    s
    e t
    s
    g
    e
    ne
    r
    a
    t
    e
    d
    a
    t
    i
    n
    c
    r
    e
    d
    i
    bl
    e
    p
    a
    ce

    f
    ro
    m
    h
    u
    m a
    n
    o
    b
    se
    r
    v
    a
    t
    i
    o
    n
    t
    o
    d
    a
    ta a
    n
    a
    l
    y
    s
    is

    c
    h
    e
    m
    o
    -i
    nf
    o
    rm
    a
    ti
    c
    s
    (d
    r u
    g
    d
    i
    sc
    o
    v e
    r
    y
    )


    R
    e
    s
    e
    a
    r
    c
    h
    I
    m
    p
    a
    ct
    —>
    S
    o
    c
    ia
    l
    a
    n
    d
    H
    u
    m
    a
    n
    I
    mp
    a
    c
    t

    View Slide

  8. Why Deep Learning, btw?
    S
    u b
    s
    et o
    f M
    L w
    /
    v
    e
    r
    y
    s
    pe
    c
    if
    i
    c
    m
    o
    d
    e
    l
    :
    (D
    e
    e
    p?)
    N
    e
    u
    ra
    l
    N
    e
    tw
    o
    r
    k
    s


    S
    ta
    t
    e
    o
    f
    t
    h
    e a
    r
    t


    T
    h
    e
    o
    r y
    ’50 / ’80


    h
    w
    a
    cc
    e
    l
    e
    r
    a
    t
    i
    o
    n t
    o tr
    a
    i
    n

    (~n
    ew
    )
    l
    e
    a
    rn
    i
    ng st
    r
    u
    c
    t
    ur
    e +
    c
    o
    m
    po
    s
    a
    b
    i
    l
    i
    t
    y
    (2018/23)

    View Slide

  9. Ml / Dl basics in a NutShell

    View Slide

  10. f
    e a
    t
    ur
    e
    s
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    L
    /D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    P r
    e
    di
    c
    ti
    o
    n
    s
    Supervised


    learning
    s
    up
    e
    rv
    i
    si
    o
    n

    View Slide

  11. f
    e a
    t
    ur
    e
    s
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    L
    /D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    S
    i
    m
    i
    l
    a
    r
    i
    t
    i
    e s
    /l
    i
    k
    e
    l
    i
    h
    o
    o
    d
    UnSupervised


    learning

    View Slide

  12. l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    P r
    e
    di
    c
    ti
    o
    n
    s
    s
    up
    e
    rv
    i
    si
    o
    n
    f
    e
    a
    t
    ur
    e
    s
    Deep

    Supervised


    learning

    View Slide

  13. What Deep Learning is about…

    View Slide

  14. Neural Networks
    A
    m
    u
    l
    t
    i-l
    a
    y
    e
    r
    f
    e e
    d-f
    o
    r
    w
    a
    r
    d
    n
    e
    ur
    a
    l
    n
    et
    w
    o
    r
    k
    c
    o
    mp
    o
    se
    d
    b
    y
    m
    u
    l
    t
    i
    pl
    e

    d
    en
    s
    e
    (a.k.a.
    f
    u
    l
    l
    y
    -c
    o
    n
    n
    ec
    t
    ed
    )
    h
    i
    d
    d
    e
    n
    l
    a
    y
    e
    r
    s
    a
    nd no
    n-l
    in
    e
    a
    r
    t
    r a
    n
    s f
    o
    rm
    a
    t
    i
    o
    ns

    View Slide

  15. View Slide

  16. More details…

    View Slide

  17. More details…
    ReLu |


    sigmoid |


    tanh

    View Slide

  18. More details…
    Repeat for each layer…

    View Slide

  19. More details…
    Image Classification Task

    View Slide

  20. More details…
    S
    u m
    m
    a
    r
    y:


    A (b
    a
    s
    i
    c
    )
    D e
    e
    p
    N
    e
    u
    r
    a
    l N
    e
    t
    w o
    r
    k
    i
    s
    :


    • C
    o
    m
    p
    o
    s
    e
    d
    o
    f
    m
    u
    l
    t
    i
    p
    l
    e
    l
    a
    y
    e
    rs
    ;


    • E
    a
    c
    h
    L
    a
    y
    e
    r
    i
    s
    f
    (W
    x
    +b):


    • L
    in
    e
    a
    r m
    o
    d
    e
    l
    ->
    W
    x
    +b


    • f()
    is a
    n
    o
    n
    -l
    i
    n
    e
    a
    r f
    u
    n
    c t
    i
    o
    n
    G
    o
    a
    l
    o
    f
    t
    h
    e
    l
    ea
    r
    ni
    n
    g
    t
    a
    s
    k:
    fi
    n
    d
    i
    n
    g
    t
    h
    e va
    l
    ue
    s
    o
    f
    a
    l
    l 

    W
    s
    a
    n d bs
    (i.e.
    m o
    d
    e l p
    a
    r
    a
    m
    e
    te
    r
    s
    )
    s o t h
    a
    t
    t
    h
    e
    c
    l
    a
    s
    s i
    f
    ic
    a
    ti
    o
    n
    e
    rr
    o
    r
    w
    o
    u
    l
    d
    b
    e
    m
    in
    i
    m
    u
    m.

    View Slide

  21. Supervised Learning Loop
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    o
    d
    e
    l
    P
    a
    r
    a
    m e
    t
    e r
    s
    L
    o
    s
    s
    l
    o
    s
    s
    p
    r e
    d
    ic
    t
    i
    o
    n
    s

    View Slide

  22. Supervised


    Training Loop


    breakdown & terminology
    (r
    a
    w
    )
    D
    a
    t
    a
    -
    a
    .k.a.
    O
    b
    s
    e r
    v
    a
    t
    i
    o
    n
    s /
    I
    n
    p
    u t

    I
    te
    m
    s
    a
    b o
    u
    t
    w
    h
    i
    c
    h
    w
    e
    w
    a
    n
    t
    t
    o
    p
    r
    e d
    i
    c t s
    o
    m
    e
    t
    h
    i n
    g.
    W
    e u s
    u
    a
    l
    l
    y
    w
    i l
    l
    d
    en
    o
    te o
    b
    s
    e
    r
    v
    a
    t
    i
    o
    n w i
    t
    h x
    .


    L
    a
    b
    e
    l
    s
    -
    a
    .k.a.
    T
    a
    r
    g
    e
    t
    s
    (i.e.
    G
    r
    o
    u
    n
    d
    T
    r
    u
    t
    h)

    L a
    b
    el
    s
    c
    o
    rr
    e
    s p
    o
    n d
    i
    n
    g t
    o o
    b
    s
    e
    r
    v
    a
    t
    i
    o
    n
    s.
    T
    h
    e
    s
    e a
    r
    e
    u
    s
    u
    a
    l
    l
    y t
    h
    e
    t
    h
    i n
    g
    s
    b
    e i
    n
    g
    p
    re
    d
    ic
    t
    e
    d
    .
    F
    o
    l
    l
    o
    w
    i n
    g
    s
    t
    a
    n
    d
    a
    r
    d
    n
    o
    t a
    t
    i o
    n
    s
    i
    n
    M
    L
    /D
    L
    ,
    w
    e w
    i
    l
    l
    u
    s
    e y
    t
    o
    r
    e f
    e
    r
    t
    o
    t
    h
    e
    s
    e
    .


    M
    o
    d
    e
    l
    f(x) = ˆy

    A
    m
    a
    th
    e
    m
    a
    t
    i
    c
    a
    e
    x
    p
    r
    e
    s
    s
    i
    o
    n o
    r a
    f
    u
    n
    c
    t i
    o
    n
    t
    h
    a
    t
    t
    a
    k e
    s
    a
    n
    o
    b
    s
    e
    r
    v a
    t
    i
    o
    n
    x a
    n
    d
    p
    re
    d
    ic
    t
    s
    t
    h
    e v a
    l
    u
    e o
    f i
    t
    s
    t
    a
    r g
    e
    t
    l
    a
    b
    e
    l
    .


    P
    r e
    d
    i
    c
    t
    i o
    n
    s
    -
    a
    .k.a.
    Es
    t
    i m
    a
    t
    e
    s:
    V
    a
    l
    u e
    s
    o
    f
    t
    h
    e
    T
    a
    r
    g
    e
    t
    s
    g
    e
    n
    e
    r
    a
    t
    e
    d
    b
    y
    t
    h
    e
    m
    o
    d
    e
    l
    -
    u
    s
    u
    a
    l
    l
    y
    r
    e
    f
    e
    r
    r
    e
    d
    t
    o
    a
    s
    ˆy


    P
    a
    r
    a
    m
    e
    t
    e r
    s -
    a
    .k.a.
    W
    e
    i
    g
    h
    t
    s
    (i
    n
    D
    L
    t
    e r
    m
    i
    n
    o
    l
    o
    g
    y
    )

    P
    a
    r
    a
    m
    e
    t
    e r
    s
    o
    f
    t
    h
    e
    M
    o
    d
    e
    l
    .
    W
    e w i
    l
    l
    r
    e
    f
    e
    r
    t
    o
    t
    h
    e
    m
    u
    s
    i
    n
    g
    t
    h
    e
    w.


    L
    o
    s
    s
    F
    u
    n
    c
    t
    i
    o
    n
    L
    (y, ˆy):

    F
    u
    n
    c
    t i
    o
    n
    t
    h
    a
    t
    c
    o
    m
    p
    a
    r
    e
    s
    h
    o
    w
    f
    a
    r o
    f
    f
    a p r
    e
    d i
    c
    t i
    o
    n
    i
    s
    f
    ro
    m
    i
    t
    s
    t
    a
    r
    g
    e t f
    o
    r
    o
    bs
    e
    r v
    a
    t i
    o
    n
    s i
    n t
    h
    e
    t
    r
    a
    i
    n
    i
    n
    g
    d
    a
    t a
    .
    T
    h
    e
    l
    o
    s
    s f u
    n
    c
    t
    i
    o
    n a
    s
    s
    i
    g
    n
    s
    a
    s
    c
    a
    l
    a
    r
    r
    e a
    l
    v
    a
    l
    u
    e
    c
    a
    l
    l
    e d t h
    e
    l
    o
    s
    s
    .
    T
    h
    e
    l
    o
    w
    e
    r
    t
    h
    e
    v
    a
    l
    u
    e
    o
    f
    t
    h
    e l
    o
    s
    s
    ,
    t h
    e
    b
    e
    t t
    e
    r
    t
    h
    e
    m
    o
    d
    e
    l
    i
    s
    p
    r e
    d
    i
    c
    t
    i
    n
    g.
    T
    h
    e
    L
    o
    s
    s i
    s u
    s
    u
    a
    l
    l
    y
    r
    e f
    e
    r r
    e
    d
    t
    o
    a
    s
    L
    S
    o
    u
    r
    c
    e
    :D.
    R
    a
    o
    e
    t
    a
    l. -
    N
    a
    t
    ur
    a
    l
    L
    a
    n
    g
    u a
    g
    e
    P
    r
    o
    c
    e s
    s
    i n
    g
    w
    i
    th Py
    T
    o
    r
    c
    h
    ,
    O
    ’R
    e i
    l
    l
    y
    2019

    View Slide

  23. View Slide

  24. Deep Learning in

    Bioinformatics and BioMedicine
    A
    f
    e
    w
    e
    xa
    m
    p
    l
    e
    s

    View Slide

  25. Diabetic Retinopathy from
    fundus retinal images
    DeepDR

    View Slide

  26. DL for Digital Pathology
    Digital Pathology (Pipeline)
    Detect


    Tissue Region
    Transfer Learning

    View Slide

  27. DL for Digital Pathology
    Transfer Learning

    View Slide

  28. BioMedical Image Segmentation
    U-Net
    Note: AutoEncoder Model
    (Unsupervised Learning)

    View Slide

  29. Toxicity Prediction
    using Deep Learning


    ToX21 Dataset
    DeepTox Pipeline

    View Slide

  30. Now it’s time to switch to
    notebooks…

    View Slide

  31. Materials and Practical
    R
    e
    p
    o
    si
    t
    o
    r
    y:

    h
    t
    tp
    s://g
    i t
    h
    ub
    .c
    o
    m
    /l
    er
    i
    o
    m
    a
    gg
    i
    o
    /d
    ee
    p-l
    ea
    r
    ni
    n
    g
    -c
    l
    a
    s
    s

    m
    y
    b
    i
    nd
    e
    r

    View Slide