Cultural transformation towards using AI

Lecture given at UKIO (June 2023) and also adapted and given to Oxford University Hospitals annual regional meeting (12th July 2023).

Transcript

1. Cultural transformation 
 towards using AI Consultant Musculoskeletal Radiologist Regional Clinical Director of Yorkshire Imaging Collaborative Chair of the RCR Informatics Committee The imaging network perspective Dr Daniel Fascia
2. AI is very new… in medicine
3. New things… need nurture
4. Picture this…
5. 21:55 FY1 in medicine undertaking first shift in new hospital, started on nights. <bleep…> Nurse asks them to check results on an NG tube placement from the prior shift to see if they can administer medicines down it. FY1 with no training, - logs on and accesses imaging results
6. REPEAT Singh, V., Danda, V., Gorniak, R. et al. Assessment of Critical Feeding Tube Malpositions on Radiographs Using Deep Learning. J Digit Imaging 32, 651–655 (2019).
7. 22:45 Asks StR for help because they are tricky x-rays and “I’ve had no training” Don’t worry… the hospital has AI to assist Just look at that.
8. REPEAT INDETERMINATE INDETERMINATE
9. REPEAT INDETERMINATE INDETERMINATE Not useful. No training. Unsafe
10. Meanwhile down in ED…
11. 01:28 Middle grade in Emergency Department attends patient with wrist pain after trauma Examination and X-ray carried out - Department has AI fracture detection - Radiology reports take 5-days
12. 01:28 Middle grade in Emergency Department attends patient with wrist pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days
13. 01:28 Middle grade in Emergency Department attends patient with wrist pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days Crikey! what do I make of this? FRACTURE DETECTED 72% CI 50% CI 50% CI 64% CI 41% CI
14. 01:28 Middle grade in Emergency Department attends patient with wrist pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days Crikey! what do I make of this? Safety net action - Tell patient they have multiple hand fractures - Advice (Work, activity etc) - Put in splint - Fracture clinic FRACTURE DETECTED 72% CI 50% CI 50% CI 64% CI 41% CI
15. 5-days later MSK Radiology Report No bone injury identified. If clinical presentation supports scaphoid fracture, secondary imaging with MR recommended.
16. MRI Scan shows scaphoid fracture But no other “multiple hand fractures”
17. MRI Scan shows scaphoid fracture But no other “multiple hand fractures” Roller Coaster Medicine
18. MRI Scan shows scaphoid fracture But no other “multiple hand fractures” Roller Coaster Medicine Poor satisfaction High litigation
19. Failed cultural design Common feature in both situations
20. How can we avoid this?
21. Vendor responsibilities Adopter 
 responsibilities
22. Vendor responsibilities Adopter 
 responsibilities Perfect AI partnership
23. For vendors
24. Who is your product for? UI design Tried, tested, tweaked for user
25. Most AI tools are (currently) not really for radiologists - Keep the radiology simple - Clear - Fast - Actionable
26. It’s good to annotate but don’t overwhelm - Slower to read output - Indecision - More training needed - Scales poorly to small screen and tiled interface
27. Detail and terminology are OK in speci f ic specialist settings - Orthopaedic measurements - Angles - Common abbreviations from clinical practice (OA, RA)
28. What good looks like AI data display for good clinical adoption
29. FRACTURE DETECTED 👍 Dan.AI Very con f ident Area for review sID: 987523123 Time of analysis: 23-04-2023 17:45 Images analysed: 2 Report a f inding • Company / product name • Clear signalling of normal/ abnormal • Clear labelling of detected features • Well distilled, minimal statistical supporting evidence • Vendor study ID • Timestamp for the analysis • Indication of what was analysed • Functionality to report discrepancy
30. When you deploy AI Your vendor should • Provide speci f ic on site training for radiology and clinical power users • Identify and overtrain ‘key trainers’ • Provide supporting materials to support the deployment • Paper (yes!) • Electronic 
 • Be receptive and action discrepancy reports • Don’t disenfranchise those who ‘cannot attend’
31. For adopters
32. 31% of my fracture clinic appointments have no fracture Identify a problem to address 6% of Brain CT overnight has a signi f icant reporting error 1
33. Identify those affected by the AI solution 2 Radiology Clinical Specialisms IT / RIS / PACS Information Governance & Risk Hospital Board Patients
34. Assemble a mini-board of stakeholders 3 Radiology Clinical Specialisms IT / RIS / PACS Information Governance & Risk Hospital Board Patients Radiology Lead Clinical Lead IT / RIS / PACS Representation Project Management (liaison to IG etc) Executive Sponsor Patient Expert Group
35. Identify an off the shelf AI solution 
 to solve problem 4 It is unrealistic for most active clinical care pipeline services 
 to develop and deploy their own AI solutions
36. Work together with your vendor 
 to deploy to clinical sandbox 5 Test. Tweak. Train users.
37. Launch together 6 Vendors physically present supporting each stakeholder group.
38. Launch AI together Radiology Clinical Specialisms IT / RIS / PACS Information Governance & Risk Hospital Board Patients Empower radiologists and radiographers to power use AI Support sta ff in interpreting AI results Clear line of post launch support 
 Reassurance 
 Congratulate for innovation. 
 Media opportunities. Clear messaging: AI role in their care
39. Re-measure your metric 7 Did AI help to solve your problem?
40. Innovating to get things into clinical use Part 2: Kickstarting AI funding
41. Yorkshire Imaging AI Seed Fund Accelerator • April 2023 - funds derived from NHSE (via Imaging Network) • Awarded 6 x seed fund awards of £40,000 • Member Trusts of our Regional Imaging Network • To be used for a f ixed term ‘one off’ trial of an AI product in clinical use • No refunding or further resourcing • Not for research use • Must carry out in process audit and reporting • Demonstrate baseline hypothesis • Measure whether AI helped this metric
42. Examples
43. Can AI fracture detection at “the front door” reduce the burden of 
 non-fracture cases f illing up the fracture clinic? 31% of fracture clinic attendances 
 are false negative 🚧
44. Can AI supported reading of CT head in the overnight setting reduce the rate of misinterpreted f indings as measured by the REALM process CT heads reported by trainees overnight carry an 6% rate of 
 ‘signi f icant misinterpretation’ 🚧
45. By AI 1st reading CXRs and prioritising those with positive f indings, 
 we can reduce the time to MDTM for lung cancer diagnoses Backlogs in CXR reporting cause a delay in the lung cancer diagnostic pathway 🚧
46. What we are trialling
47. Harrogate Leeds Leeds Leeds Mid Yorkshire Bradford Radiobotics Qure.ai Radiobotics AIdoc Annalise Annalise RBfracture QXR RBfracture CT suite CT Head CXR Fracture detection Chest x-rays Fracture detection CT safety 2nd reading Triage of normal for delayed reporting Chest x-rays Deployed Early Deploying Early Early Deploying
48. Challenges so far
49. Challenges you may encounter • Small amount of funding • Needs to cover Trust internal costs • IT, RIS, PACS, overtime etc • Integration with vendors if required 
 (advise DICOM 2nd capture approach for pilot projects) • Over-governance of a f ixed trial • Over-contracting of a f ixed trial • Over-processing of a low risk fully funded endeavour • Lack of zest for innovation • NHS timelines causing failure to demonstrate progress
50. Our ambition
51. Ambitions for AI in YIC • To be a leader in clinical translation of AI products to frontline care • To test a wide battery of products and choose the best • Deploy the best at a regional scale • Use our cloud infrastructure built over many years • Bene f it our 4-Million patient users by careful selection of AI • Deliver at population scale for better population health
52. Thank you