Share
Embed
Cultural transformation towards using AI
Lecture given at UKIO (June 2023) and also adapted and given to Oxford University Hospitals annual regional meeting (12th July 2023).
Dr Daniel Fascia
July 12, 2023
Transcript
52 slides
-
Cultural transformation towards using AI Consultant Musculoskeletal Radiologist Regional
Clinical Director of Yorkshire Imaging Collaborative Chair of the RCR Informatics Committee The imaging network perspective Dr Daniel Fascia -
21:55 FY1 in medicine undertaking first shift in new hospital,
started on nights. <bleep…> Nurse asks them to check results on an NG tube placement from the prior shift to see if they can administer medicines down it. FY1 with no training, - logs on and accesses imaging results -
REPEAT Singh, V., Danda, V., Gorniak, R. et al. Assessment
of Critical Feeding Tube Malpositions on Radiographs Using Deep Learning. J Digit Imaging 32, 651–655 (2019). -
22:45 Asks StR for help because they are tricky x-rays
and “I’ve had no training” Don’t worry… the hospital has AI to assist Just look at that. -
01:28 Middle grade in Emergency Department attends patient with wrist
pain after trauma Examination and X-ray carried out - Department has AI fracture detection - Radiology reports take 5-days -
01:28 Middle grade in Emergency Department attends patient with wrist
pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days -
01:28 Middle grade in Emergency Department attends patient with wrist
pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days Crikey! what do I make of this? FRACTURE DETECTED 72% CI 50% CI 50% CI 64% CI 41% CI -
01:28 Middle grade in Emergency Department attends patient with wrist
pain after trauma Examination and X-ray carried out - Department has AI fracture detection. - Radiology reports take 5-days Crikey! what do I make of this? Safety net action - Tell patient they have multiple hand fractures - Advice (Work, activity etc) - Put in splint - Fracture clinic FRACTURE DETECTED 72% CI 50% CI 50% CI 64% CI 41% CI -
5-days later MSK Radiology Report No bone injury identified. If
clinical presentation supports scaphoid fracture, secondary imaging with MR recommended. -
MRI Scan shows scaphoid fracture But no other “multiple hand
fractures” Roller Coaster Medicine -
MRI Scan shows scaphoid fracture But no other “multiple hand
fractures” Roller Coaster Medicine Poor satisfaction High litigation -
Most AI tools are (currently) not really for radiologists -
Keep the radiology simple - Clear - Fast - Actionable -
It’s good to annotate but don’t overwhelm - Slower to
read output - Indecision - More training needed - Scales poorly to small screen and tiled interface -
Detail and terminology are OK in speci f ic specialist
settings - Orthopaedic measurements - Angles - Common abbreviations from clinical practice (OA, RA) -
FRACTURE DETECTED 👍 Dan.AI Very con f ident Area for
review sID: 987523123 Time of analysis: 23-04-2023 17:45 Images analysed: 2 Report a f inding • Company / product name • Clear signalling of normal/ abnormal • Clear labelling of detected features • Well distilled, minimal statistical supporting evidence • Vendor study ID • Timestamp for the analysis • Indication of what was analysed • Functionality to report discrepancy -
When you deploy AI Your vendor should • Provide speci
f ic on site training for radiology and clinical power users • Identify and overtrain ‘key trainers’ • Provide supporting materials to support the deployment • Paper (yes!) • Electronic • Be receptive and action discrepancy reports • Don’t disenfranchise those who ‘cannot attend’ -
31% of my fracture clinic appointments have no fracture Identify
a problem to address 6% of Brain CT overnight has a signi f icant reporting error 1 -
Identify those affected by the AI solution 2 Radiology Clinical
Specialisms IT / RIS / PACS Information Governance & Risk Hospital Board Patients -
Assemble a mini-board of stakeholders 3 Radiology Clinical Specialisms IT
/ RIS / PACS Information Governance & Risk Hospital Board Patients Radiology Lead Clinical Lead IT / RIS / PACS Representation Project Management (liaison to IG etc) Executive Sponsor Patient Expert Group -
Identify an off the shelf AI solution to solve
problem 4 It is unrealistic for most active clinical care pipeline services to develop and deploy their own AI solutions -
Work together with your vendor to deploy to clinical
sandbox 5 Test. Tweak. Train users. -
Launch AI together Radiology Clinical Specialisms IT / RIS /
PACS Information Governance & Risk Hospital Board Patients Empower radiologists and radiographers to power use AI Support sta ff in interpreting AI results Clear line of post launch support Reassurance Congratulate for innovation. Media opportunities. Clear messaging: AI role in their care -
Yorkshire Imaging AI Seed Fund Accelerator • April 2023 -
funds derived from NHSE (via Imaging Network) • Awarded 6 x seed fund awards of £40,000 • Member Trusts of our Regional Imaging Network • To be used for a f ixed term ‘one off’ trial of an AI product in clinical use • No refunding or further resourcing • Not for research use • Must carry out in process audit and reporting • Demonstrate baseline hypothesis • Measure whether AI helped this metric -
Can AI fracture detection at “the front door” reduce the
burden of non-fracture cases f illing up the fracture clinic? 31% of fracture clinic attendances are false negative 🚧 -
Can AI supported reading of CT head in the overnight
setting reduce the rate of misinterpreted f indings as measured by the REALM process CT heads reported by trainees overnight carry an 6% rate of ‘signi f icant misinterpretation’ 🚧 -
By AI 1st reading CXRs and prioritising those with positive
f indings, we can reduce the time to MDTM for lung cancer diagnoses Backlogs in CXR reporting cause a delay in the lung cancer diagnostic pathway 🚧 -
Harrogate Leeds Leeds Leeds Mid Yorkshire Bradford Radiobotics Qure.ai Radiobotics
AIdoc Annalise Annalise RBfracture QXR RBfracture CT suite CT Head CXR Fracture detection Chest x-rays Fracture detection CT safety 2nd reading Triage of normal for delayed reporting Chest x-rays Deployed Early Deploying Early Early Deploying -
Challenges you may encounter • Small amount of funding •
Needs to cover Trust internal costs • IT, RIS, PACS, overtime etc • Integration with vendors if required (advise DICOM 2nd capture approach for pilot projects) • Over-governance of a f ixed trial • Over-contracting of a f ixed trial • Over-processing of a low risk fully funded endeavour • Lack of zest for innovation • NHS timelines causing failure to demonstrate progress -
Ambitions for AI in YIC • To be a leader
in clinical translation of AI products to frontline care • To test a wide battery of products and choose the best • Deploy the best at a regional scale • Use our cloud infrastructure built over many years • Bene f it our 4-Million patient users by careful selection of AI • Deliver at population scale for better population health