Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intro to Deep Learning Class

Intro to Deep Learning Class

Introduction to Artificial Neural Networks and Deep Learning.

Deep Learning class as part of the MSc in Medical Statistics and Health Data Science, University of Bristol. Mar, 2023

Valerio Maggio

March 15, 2023
Tweet

More Decks by Valerio Maggio

Other Decks in Education

Transcript

  1. Deep Learning for
    Health and Life Sciences

    View full-size slide

  2. Deep Learning for the
    Health and Life Sciences
    Valerio Maggio, PhD
    @l
    er
    i
    o
    m
    a
    g g
    i
    o
    v
    ma
    g
    g
    i
    o@a
    n
    a
    c
    o
    n
    d
    a
    .c
    o
    m

    View full-size slide

  3. m
    e
    p
    u n
    Who?
    B
    a
    c
    k
    g r
    o
    un
    d
    i
    n
    C
    S


    P
    h
    D i
    n Ma
    c
    h
    i
    n
    e
    L
    ea
    r
    n i
    n
    g

    R
    e
    s
    e
    a
    r
    c
    h
    :
    M
    L
    /D
    L
    f
    o
    r B
    i
    o
    Me
    d
    i
    c
    i
    n
    e

    P
    y
    t
    h
    o
    n G
    e
    e
    k 

    D
    e v
    e
    l
    o
    p
    e
    r A d
    v
    o
    c
    a
    te
    _a
    t
    _
    An
    a
    c o
    n
    da
    a
    l
    s
    o
    m
    e

    View full-size slide

  4. Machine Learning
    “M
    a
    c
    h
    i n
    e
    l
    e
    a
    r
    n
    i n
    g
    i
    s
    t
    h
    e
    s
    ci
    e
    nc
    e (a
    nd a
    r
    t)
    o
    f
    p
    r
    o
    g
    ra
    m
    m
    i
    n
    g
    c
    o
    m
    p
    ut
    e
    rs so th
    e
    y
    c
    a
    n l
    e
    a
    rn fr
    o
    m
    d
    a
    t
    a”
    A
    u r
    él
    ie
    n
    G
    ér
    o
    n
    ,
    Ha
    n
    ds
    -o
    n
    M
    a
    c
    h
    i
    n
    e
    L
    e
    a
    r
    n
    i
    n
    g
    w
    i
    t
    h Sc
    i
    ki
    t-L
    e
    a
    r
    n
    a
    nd T
    e
    n
    s
    o
    r
    Fl
    o
    w
    S
    o
    u
    r
    ce
    :
    bi
    t.l
    y
    /m
    l
    -s
    i m
    p
    l
    e
    -d
    ef
    i
    ni
    t
    i o
    n
    “(m
    l
    )
    fo
    c
    us
    e
    s
    o
    n
    t
    e
    a
    c
    h
    i
    n
    g
    c
    o
    m
    p
    ut
    e
    rs h
    o
    w
    t
    o
    l
    e
    a
    r
    n
    w
    i
    t
    h
    o
    u
    t
    t
    h
    e
    n
    ee
    d
    t
    o
    b
    e
    p
    r
    o
    g
    r
    a
    m
    m
    e
    d f
    o
    r
    s
    p
    ec
    i
    f
    i
    c
    t
    a
    sk
    s”
    S.
    P a
    l &
    A
    .
    G
    u
    l
    l
    i
    ,
    D
    e
    e
    p
    L
    e
    a
    r
    ni
    n
    g
    w
    it
    h
    K
    e
    ra
    s

    View full-size slide

  5. Machine Learning
    M
    a
    c
    h
    i n
    e
    l
    e
    a
    r
    n
    i
    n
    g
    t
    e
    a
    c
    h
    es ma
    c
    h
    i
    n
    es h
    o
    w
    t
    o
    c
    a
    rr
    y
    o
    u
    t
    t
    a
    s
    k
    s
    b
    y
    t
    h
    e
    m
    s
    e
    l
    ve
    s.


    I
    t
    i
    s
    t
    h
    a
    t
    s
    i
    mp
    l
    e
    .


    T
    h
    e c
    o
    m
    p
    l
    e
    x i
    t
    y
    c
    o
    m
    e
    s
    w
    it
    h
    t
    h
    e
    d
    et
    a
    il
    s
    L
    o
    u
    i
    s
    P
    ed
    r
    o
    C
    o
    e
    l
    h
    o
    ,
    B
    u
    i
    l
    d
    i
    ng Ma
    c
    h
    i
    n
    e
    L
    ea
    r
    ni
    n
    g
    S
    y
    s
    t
    e
    m
    s
    w
    i
    t
    h P y
    t
    h
    o
    n
    (a
    n d th
    a
    t
    ’s
    p
    r
    o
    b
    a
    bl
    y
    o
    n
    e
    o
    f
    t
    h
    e re
    a
    so
    n
    w
    h
    y
    y
    o
    u
    ’r
    e
    h
    e r
    e :)

    View full-size slide

  6. The (Machine) Learning is about DATA
    D
    a
    t
    a
    a
    r
    e
    o
    ne o
    f th
    e
    m
    o
    s t i
    m
    p
    o
    r
    t
    a
    n
    t
    p
    a
    rt o
    f a
    m
    l
    s
    o
    l
    u
    ti
    o
    n

    I
    m
    p
    o
    rt
    a
    n
    c
    e:
    Da
    t
    a
    >>
    Mo
    d
    e l
    ?


    L
    e a
    r
    ni
    n
    g
    b
    y
    e
    xa
    m
    p
    l
    e
    s

    D
    a
    t
    a
    P
    r
    e
    p
    a
    ra
    t
    i o
    n
    i
    s
    c
    r
    uc
    i
    a
    l
    !
    d
    a
    t
    a
    a
    l
    g
    o
    r i
    t
    h
    m
    s

    View full-size slide

  7. BioMedicine: another data case ?
    C
    o
    n
    t
    e
    m
    p
    o
    r
    a
    ry Li
    f
    e
    S
    c i
    e
    n c
    e
    i
    s
    a
    b
    o
    u
    t
    d
    a
    t
    a


    r
    ec
    e
    nt a
    d
    v
    a
    n
    c
    e s i
    n s e
    q
    u e
    n
    ci
    n
    g
    t
    ec
    h
    s
    a
    nd i
    n
    s
    t
    r
    u
    m
    e
    n
    t
    s
    (e.g. “b
    io
    -i
    m a
    g
    e
    s
    ”)


    h
    u g
    e
    d
    a
    ta
    s
    e t
    s
    g
    e
    ne
    r
    a
    t
    e
    d
    a
    t
    i
    n
    c
    r
    e
    d
    i
    bl
    e
    p
    a
    ce

    f
    ro
    m
    h
    u
    m a
    n
    o
    b
    se
    r
    v
    a
    t
    i
    o
    n
    t
    o
    d
    a
    ta a
    n
    a
    l
    y
    s
    is

    c
    h
    e
    m
    o
    -i
    nf
    o
    rm
    a
    ti
    c
    s
    (d
    r u
    g
    d
    i
    sc
    o
    v e
    r
    y
    )


    R
    e
    s
    e
    a
    r
    c
    h
    I
    m
    p
    a
    ct
    —>
    S
    o
    c
    ia
    l
    a
    n
    d
    H
    u
    m
    a
    n
    I
    mp
    a
    c
    t

    View full-size slide

  8. Why Deep Learning, btw?
    S
    u b
    s
    et o
    f M
    L w
    /
    v
    e
    r
    y
    s
    pe
    c
    if
    i
    c
    m
    o
    d
    e
    l
    :
    (D
    e
    e
    p?)
    N
    e
    u
    ra
    l
    N
    e
    tw
    o
    r
    k
    s


    S
    ta
    t
    e
    o
    f
    t
    h
    e a
    r
    t


    T
    h
    e
    o
    r y
    ’50 / ’80


    h
    w
    a
    cc
    e
    l
    e
    r
    a
    t
    i
    o
    n t
    o tr
    a
    i
    n

    (~n
    ew
    )
    l
    e
    a
    rn
    i
    ng st
    r
    u
    c
    t
    ur
    e +
    c
    o
    m
    po
    s
    a
    b
    i
    l
    i
    t
    y
    (2018/23)

    View full-size slide

  9. Ml / Dl basics in a NutShell

    View full-size slide

  10. f
    e a
    t
    ur
    e
    s
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    L
    /D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    P r
    e
    di
    c
    ti
    o
    n
    s
    Supervised


    learning
    s
    up
    e
    rv
    i
    si
    o
    n

    View full-size slide

  11. f
    e a
    t
    ur
    e
    s
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    L
    /D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    S
    i
    m
    i
    l
    a
    r
    i
    t
    i
    e s
    /l
    i
    k
    e
    l
    i
    h
    o
    o
    d
    UnSupervised


    learning

    View full-size slide

  12. l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    D
    L 

    M
    o
    d
    e
    l
    T
    r
    a
    i
    n
    i
    n
    g
    T
    ra
    i
    ne
    d

    M
    o
    d
    e
    l
    (u
    n
    s
    e
    e n
    )
    d
    a
    t
    a
    T
    e
    s
    t
    P r
    e
    di
    c
    ti
    o
    n
    s
    s
    up
    e
    rv
    i
    si
    o
    n
    f
    e
    a
    t
    ur
    e
    s
    Deep

    Supervised


    learning

    View full-size slide

  13. What Deep Learning is about…

    View full-size slide

  14. Neural Networks
    A
    m
    u
    l
    t
    i-l
    a
    y
    e
    r
    f
    e e
    d-f
    o
    r
    w
    a
    r
    d
    n
    e
    ur
    a
    l
    n
    et
    w
    o
    r
    k
    c
    o
    mp
    o
    se
    d
    b
    y
    m
    u
    l
    t
    i
    pl
    e

    d
    en
    s
    e
    (a.k.a.
    f
    u
    l
    l
    y
    -c
    o
    n
    n
    ec
    t
    ed
    )
    h
    i
    d
    d
    e
    n
    l
    a
    y
    e
    r
    s
    a
    nd no
    n-l
    in
    e
    a
    r
    t
    r a
    n
    s f
    o
    rm
    a
    t
    i
    o
    ns

    View full-size slide

  15. More details…

    View full-size slide

  16. More details…
    ReLu |


    sigmoid |


    tanh

    View full-size slide

  17. More details…
    Repeat for each layer…

    View full-size slide

  18. More details…
    Image Classification Task

    View full-size slide

  19. More details…
    S
    u m
    m
    a
    r
    y:


    A (b
    a
    s
    i
    c
    )
    D e
    e
    p
    N
    e
    u
    r
    a
    l N
    e
    t
    w o
    r
    k
    i
    s
    :


    • C
    o
    m
    p
    o
    s
    e
    d
    o
    f
    m
    u
    l
    t
    i
    p
    l
    e
    l
    a
    y
    e
    rs
    ;


    • E
    a
    c
    h
    L
    a
    y
    e
    r
    i
    s
    f
    (W
    x
    +b):


    • L
    in
    e
    a
    r m
    o
    d
    e
    l
    ->
    W
    x
    +b


    • f()
    is a
    n
    o
    n
    -l
    i
    n
    e
    a
    r f
    u
    n
    c t
    i
    o
    n
    G
    o
    a
    l
    o
    f
    t
    h
    e
    l
    ea
    r
    ni
    n
    g
    t
    a
    s
    k:
    fi
    n
    d
    i
    n
    g
    t
    h
    e va
    l
    ue
    s
    o
    f
    a
    l
    l 

    W
    s
    a
    n d bs
    (i.e.
    m o
    d
    e l p
    a
    r
    a
    m
    e
    te
    r
    s
    )
    s o t h
    a
    t
    t
    h
    e
    c
    l
    a
    s
    s i
    f
    ic
    a
    ti
    o
    n
    e
    rr
    o
    r
    w
    o
    u
    l
    d
    b
    e
    m
    in
    i
    m
    u
    m.

    View full-size slide

  20. Supervised Learning Loop
    l
    a
    b
    e
    l
    s
    (r
    a
    w
    )
    d a
    t
    a
    M
    o
    d
    e
    l
    P
    a
    r
    a
    m e
    t
    e r
    s
    L
    o
    s
    s
    l
    o
    s
    s
    p
    r e
    d
    ic
    t
    i
    o
    n
    s

    View full-size slide

  21. Supervised


    Training Loop


    breakdown & terminology
    (r
    a
    w
    )
    D
    a
    t
    a
    -
    a
    .k.a.
    O
    b
    s
    e r
    v
    a
    t
    i
    o
    n
    s /
    I
    n
    p
    u t

    I
    te
    m
    s
    a
    b o
    u
    t
    w
    h
    i
    c
    h
    w
    e
    w
    a
    n
    t
    t
    o
    p
    r
    e d
    i
    c t s
    o
    m
    e
    t
    h
    i n
    g.
    W
    e u s
    u
    a
    l
    l
    y
    w
    i l
    l
    d
    en
    o
    te o
    b
    s
    e
    r
    v
    a
    t
    i
    o
    n w i
    t
    h x
    .


    L
    a
    b
    e
    l
    s
    -
    a
    .k.a.
    T
    a
    r
    g
    e
    t
    s
    (i.e.
    G
    r
    o
    u
    n
    d
    T
    r
    u
    t
    h)

    L a
    b
    el
    s
    c
    o
    rr
    e
    s p
    o
    n d
    i
    n
    g t
    o o
    b
    s
    e
    r
    v
    a
    t
    i
    o
    n
    s.
    T
    h
    e
    s
    e a
    r
    e
    u
    s
    u
    a
    l
    l
    y t
    h
    e
    t
    h
    i n
    g
    s
    b
    e i
    n
    g
    p
    re
    d
    ic
    t
    e
    d
    .
    F
    o
    l
    l
    o
    w
    i n
    g
    s
    t
    a
    n
    d
    a
    r
    d
    n
    o
    t a
    t
    i o
    n
    s
    i
    n
    M
    L
    /D
    L
    ,
    w
    e w
    i
    l
    l
    u
    s
    e y
    t
    o
    r
    e f
    e
    r
    t
    o
    t
    h
    e
    s
    e
    .


    M
    o
    d
    e
    l
    f(x) = ˆy

    A
    m
    a
    th
    e
    m
    a
    t
    i
    c
    a
    e
    x
    p
    r
    e
    s
    s
    i
    o
    n o
    r a
    f
    u
    n
    c
    t i
    o
    n
    t
    h
    a
    t
    t
    a
    k e
    s
    a
    n
    o
    b
    s
    e
    r
    v a
    t
    i
    o
    n
    x a
    n
    d
    p
    re
    d
    ic
    t
    s
    t
    h
    e v a
    l
    u
    e o
    f i
    t
    s
    t
    a
    r g
    e
    t
    l
    a
    b
    e
    l
    .


    P
    r e
    d
    i
    c
    t
    i o
    n
    s
    -
    a
    .k.a.
    Es
    t
    i m
    a
    t
    e
    s:
    V
    a
    l
    u e
    s
    o
    f
    t
    h
    e
    T
    a
    r
    g
    e
    t
    s
    g
    e
    n
    e
    r
    a
    t
    e
    d
    b
    y
    t
    h
    e
    m
    o
    d
    e
    l
    -
    u
    s
    u
    a
    l
    l
    y
    r
    e
    f
    e
    r
    r
    e
    d
    t
    o
    a
    s
    ˆy


    P
    a
    r
    a
    m
    e
    t
    e r
    s -
    a
    .k.a.
    W
    e
    i
    g
    h
    t
    s
    (i
    n
    D
    L
    t
    e r
    m
    i
    n
    o
    l
    o
    g
    y
    )

    P
    a
    r
    a
    m
    e
    t
    e r
    s
    o
    f
    t
    h
    e
    M
    o
    d
    e
    l
    .
    W
    e w i
    l
    l
    r
    e
    f
    e
    r
    t
    o
    t
    h
    e
    m
    u
    s
    i
    n
    g
    t
    h
    e
    w.


    L
    o
    s
    s
    F
    u
    n
    c
    t
    i
    o
    n
    L
    (y, ˆy):

    F
    u
    n
    c
    t i
    o
    n
    t
    h
    a
    t
    c
    o
    m
    p
    a
    r
    e
    s
    h
    o
    w
    f
    a
    r o
    f
    f
    a p r
    e
    d i
    c
    t i
    o
    n
    i
    s
    f
    ro
    m
    i
    t
    s
    t
    a
    r
    g
    e t f
    o
    r
    o
    bs
    e
    r v
    a
    t i
    o
    n
    s i
    n t
    h
    e
    t
    r
    a
    i
    n
    i
    n
    g
    d
    a
    t a
    .
    T
    h
    e
    l
    o
    s
    s f u
    n
    c
    t
    i
    o
    n a
    s
    s
    i
    g
    n
    s
    a
    s
    c
    a
    l
    a
    r
    r
    e a
    l
    v
    a
    l
    u
    e
    c
    a
    l
    l
    e d t h
    e
    l
    o
    s
    s
    .
    T
    h
    e
    l
    o
    w
    e
    r
    t
    h
    e
    v
    a
    l
    u
    e
    o
    f
    t
    h
    e l
    o
    s
    s
    ,
    t h
    e
    b
    e
    t t
    e
    r
    t
    h
    e
    m
    o
    d
    e
    l
    i
    s
    p
    r e
    d
    i
    c
    t
    i
    n
    g.
    T
    h
    e
    L
    o
    s
    s i
    s u
    s
    u
    a
    l
    l
    y
    r
    e f
    e
    r r
    e
    d
    t
    o
    a
    s
    L
    S
    o
    u
    r
    c
    e
    :D.
    R
    a
    o
    e
    t
    a
    l. -
    N
    a
    t
    ur
    a
    l
    L
    a
    n
    g
    u a
    g
    e
    P
    r
    o
    c
    e s
    s
    i n
    g
    w
    i
    th Py
    T
    o
    r
    c
    h
    ,
    O
    ’R
    e i
    l
    l
    y
    2019

    View full-size slide

  22. Deep Learning in

    Bioinformatics and BioMedicine
    A
    f
    e
    w
    e
    xa
    m
    p
    l
    e
    s

    View full-size slide

  23. Diabetic Retinopathy from
    fundus retinal images
    DeepDR

    View full-size slide

  24. DL for Digital Pathology
    Digital Pathology (Pipeline)
    Detect


    Tissue Region
    Transfer Learning

    View full-size slide

  25. DL for Digital Pathology
    Transfer Learning

    View full-size slide

  26. BioMedical Image Segmentation
    U-Net
    Note: AutoEncoder Model
    (Unsupervised Learning)

    View full-size slide

  27. Toxicity Prediction
    using Deep Learning


    ToX21 Dataset
    DeepTox Pipeline

    View full-size slide

  28. Now it’s time to switch to
    notebooks…

    View full-size slide

  29. Materials and Practical
    R
    e
    p
    o
    si
    t
    o
    r
    y:

    h
    t
    tp
    s://g
    i t
    h
    ub
    .c
    o
    m
    /l
    er
    i
    o
    m
    a
    gg
    i
    o
    /d
    ee
    p-l
    ea
    r
    ni
    n
    g
    -c
    l
    a
    s
    s

    m
    y
    b
    i
    nd
    e
    r

    View full-size slide