Cultural transformation towards using AI

Lecture given at UKIO (June 2023) and also adapted and given to Oxford University Hospitals annual regional meeting (12th July 2023).

Transcript

1. Cultural transformation
   

   towards using AI
   Consultant Musculoskeletal Radiologist
   
   
   Regional Clinical Director of Yorkshire Imaging Collaborative
   
   
   Chair of the RCR Informatics Committee
   The imaging network perspective
   
   
   Dr Daniel Fascia
   

   View Slide

2. AI is very new… in medicine
   

   View Slide

3. New things… need nurture
   

   View Slide

4. Picture this…
   

   View Slide

5. 21:55
   FY1 in medicine undertaking first shift in new
   hospital, started on nights.
   
   Nurse asks them to check results on an NG tube
   placement from the prior shift to see if they
   can administer medicines down it.
   FY1 with no training,
   - logs on and accesses imaging results
   

   View Slide

6. REPEAT
   Singh, V., Danda, V., Gorniak, R. et al. Assessment of Critical Feeding Tube Malpositions on Radiographs Using Deep Learning. J Digit Imaging 32, 651–655 (2019).
   

   View Slide

7. 22:45
   Asks StR for help because they are tricky x-rays
   and “I’ve had no training”
   Don’t worry… the hospital has AI to assist
   Just look at that.
   

   View Slide

8. REPEAT
   INDETERMINATE INDETERMINATE
   

   View Slide

9. REPEAT
   INDETERMINATE INDETERMINATE
   Not useful. No training. Unsafe
   

   View Slide

10. Meanwhile down in ED…
    

    View Slide

11. 01:28
    Middle grade in Emergency Department attends
    patient with wrist pain after trauma
    Examination and X-ray carried out
    - Department has AI fracture detection
    - Radiology reports take 5-days
    

    View Slide

12. 01:28
    Middle grade in Emergency Department attends
    patient with wrist pain after trauma
    Examination and X-ray carried out
    - Department has AI fracture detection.
    - Radiology reports take 5-days
    

    View Slide

13. 01:28
    Middle grade in Emergency Department attends
    patient with wrist pain after trauma
    Examination and X-ray carried out
    - Department has AI fracture detection.
    - Radiology reports take 5-days
    Crikey! what do I make of this?
    FRACTURE DETECTED
    72% CI
    50% CI
    50% CI
    64% CI
    41% CI
    

    View Slide

14. 01:28
    Middle grade in Emergency Department attends
    patient with wrist pain after trauma
    Examination and X-ray carried out
    - Department has AI fracture detection.
    - Radiology reports take 5-days
    Crikey! what do I make of this?
    Safety net action
    - Tell patient they have multiple hand fractures
    - Advice (Work, activity etc)
    - Put in splint
    - Fracture clinic
    FRACTURE DETECTED
    72% CI
    50% CI
    50% CI
    64% CI
    41% CI
    

    View Slide

15. 5-days later
    MSK Radiology Report
    No bone injury identified.
    If clinical presentation supports scaphoid
    fracture, secondary imaging with MR recommended.
    

    View Slide

16. MRI Scan shows scaphoid fracture
    But no other “multiple hand fractures”
    

    View Slide

17. MRI Scan shows scaphoid fracture
    But no other “multiple hand fractures”
    Roller Coaster Medicine
    

    View Slide

18. MRI Scan shows scaphoid fracture
    But no other “multiple hand fractures”
    Roller Coaster Medicine
    Poor satisfaction
    
    
    High litigation
    

    View Slide

19. Failed cultural design
    Common feature in both situations
    

    View Slide

20. How can we avoid this?
    

    View Slide

21. Vendor
    responsibilities
    Adopter
    

    responsibilities
    

    View Slide

22. Vendor
    responsibilities
    Adopter
    

    responsibilities
    Perfect AI partnership
    

    View Slide

23. For vendors
    

    View Slide

24. Who is your product for?
    
    
    UI design
    
    
    Tried, tested, tweaked for user
    

    View Slide

25. Most AI tools are (currently)
    not really for radiologists
    
    
    - Keep the radiology simple
    
    
    - Clear
    
    
    - Fast
    
    
    - Actionable
    
    
    

    View Slide

26. It’s good to annotate but don’t
    overwhelm
    
    
    - Slower to read output
    
    
    - Indecision
    
    
    - More training needed
    
    
    - Scales poorly to small
    screen and tiled interface
    

    View Slide

27. Detail and terminology are OK
    in speci
    f
    ic specialist settings
    
    
    - Orthopaedic measurements
    
    
    - Angles
    
    
    - Common abbreviations from
    clinical practice (OA, RA)
    

    View Slide

28. What good looks like
    AI data display for good clinical adoption
    

    View Slide

29. FRACTURE DETECTED
    👍 Dan.AI
    Very con
    f
    ident
    Area for review
    sID: 987523123
    
    
    Time of analysis: 23-04-2023 17:45
    
    
    Images analysed: 2
    Report a
    f
    inding
    • Company / product name
    
    
    • Clear signalling of normal/
    abnormal
    
    
    • Clear labelling of detected
    features
    
    
    • Well distilled, minimal statistical
    supporting evidence
    
    
    • Vendor study ID
    
    
    • Timestamp for the analysis
    
    
    • Indication of what was analysed
    
    
    • Functionality to report
    discrepancy
    

    View Slide

30. When you deploy AI
    Your vendor should
    • Provide speci
    f
    ic on site training for radiology and clinical power users
    
    
    • Identify and overtrain ‘key trainers’
    
    
    • Provide supporting materials to support the deployment
    
    
    • Paper (yes!)
    
    
    • Electronic
    

    • Be receptive and action discrepancy reports
    
    
    • Don’t disenfranchise those who ‘cannot attend’
    

    View Slide

31. For adopters
    

    View Slide

32. 31% of my fracture clinic appointments have no fracture
    Identify a problem to address
    6% of Brain CT overnight has a signi
    f
    icant reporting error
    1
    

    View Slide

33. Identify those affected by the AI solution
    2
    Radiology
    
    
    Clinical Specialisms
    
    
    IT / RIS / PACS
    
    
    Information Governance & Risk
    
    
    Hospital Board
    
    
    Patients
    

    View Slide

34. Assemble a mini-board of stakeholders
    3
    Radiology
    
    
    Clinical Specialisms
    
    
    IT / RIS / PACS
    
    
    Information Governance & Risk
    
    
    Hospital Board
    
    
    Patients
    Radiology Lead
    
    
    Clinical Lead
    
    
    IT / RIS / PACS Representation
    
    
    Project Management (liaison to IG etc)
    
    
    Executive Sponsor
    
    
    Patient Expert Group
    

    View Slide

35. Identify an off the shelf AI solution
    

    to solve problem
    4
    It is unrealistic for most active clinical care pipeline services
    

    to develop and deploy their own AI solutions
    

    View Slide

36. Work together with your vendor
    

    to deploy to clinical sandbox
    5
    Test. Tweak. Train users.
    

    View Slide

37. Launch together
    6
    Vendors physically present supporting each stakeholder group.
    

    View Slide

38. Launch AI together
    Radiology
    
    
    Clinical Specialisms
    
    
    IT / RIS / PACS
    
    
    Information Governance & Risk
    
    
    Hospital Board
    
    
    Patients
    Empower radiologists and
    radiographers to power use AI
    
    
    Support sta
    ff
    in interpreting AI results
    
    
    Clear line of post launch support
    
    
    

    Reassurance
    
    
    

    Congratulate for innovation.
    

    Media opportunities.
    
    
    Clear messaging: AI role in their care
    

    View Slide

39. Re-measure your metric
    7
    Did AI help to solve your problem?
    

    View Slide

40. Innovating to get things into clinical use
    Part 2: Kickstarting AI funding
    

    View Slide

41. Yorkshire Imaging AI Seed Fund Accelerator
    • April 2023 - funds derived from NHSE (via Imaging Network)
    
    
    • Awarded 6 x seed fund awards of £40,000
    
    
    • Member Trusts of our Regional Imaging Network
    
    
    • To be used for a
    f
    ixed term ‘one off’ trial of an AI product in clinical use
    
    
    • No refunding or further resourcing
    
    
    • Not for research use
    
    
    • Must carry out in process audit and reporting
    
    
    • Demonstrate baseline hypothesis
    
    
    • Measure whether AI helped this metric
    

    View Slide

42. Examples
    

    View Slide

43. Can AI fracture detection at “the front door” reduce the burden of
    

    non-fracture cases
    f
    illing up the fracture clinic?
    31% of fracture clinic attendances
    

    are false negative
    🚧
    

    View Slide

44. Can AI supported reading of CT head in the overnight setting reduce the rate of
    misinterpreted
    f
    indings as measured by the REALM process
    CT heads reported by trainees overnight
    carry an 6% rate of
    

    ‘signi
    f
    icant misinterpretation’
    🚧
    

    View Slide

45. By AI 1st reading CXRs and prioritising those with positive
    f
    indings,
    

    we can reduce the time to MDTM for lung cancer diagnoses
    Backlogs in CXR reporting cause a delay in
    the lung cancer diagnostic pathway
    🚧
    

    View Slide

46. What we are trialling
    

    View Slide

47. Harrogate Leeds Leeds Leeds Mid Yorkshire Bradford
    Radiobotics Qure.ai Radiobotics AIdoc Annalise Annalise
    RBfracture QXR RBfracture CT suite CT Head CXR
    Fracture
    detection
    Chest x-rays
    Fracture
    detection
    CT safety 2nd
    reading
    Triage of normal
    for delayed
    reporting
    Chest x-rays
    Deployed Early Deploying Early Early Deploying
    

    View Slide

48. Challenges so far
    

    View Slide

49. Challenges you may encounter
    • Small amount of funding
    
    
    • Needs to cover Trust internal costs
    
    
    • IT, RIS, PACS, overtime etc
    
    
    • Integration with vendors if required
    

    (advise DICOM 2nd capture approach for pilot projects)
    
    
    • Over-governance of a
    f
    ixed trial
    
    
    • Over-contracting of a
    f
    ixed trial
    
    
    • Over-processing of a low risk fully funded endeavour
    
    
    • Lack of zest for innovation
    
    
    • NHS timelines causing failure to demonstrate progress
    

    View Slide

50. Our ambition
    

    View Slide

51. Ambitions for AI in YIC
    • To be a leader in clinical translation of AI products to frontline care
    
    
    • To test a wide battery of products and choose the best
    
    
    • Deploy the best at a regional scale
    
    
    • Use our cloud infrastructure built over many years
    
    
    • Bene
    f
    it our 4-Million patient users by careful selection of AI
    
    
    • Deliver at population scale for better population health
    

    View Slide

52. Thank you
    

    View Slide