Share
Embed
Is In-hospital Meta-information Useful for Abstractive Discharge Summary Generation?
Slides of my presentation at TAAI 2022.
ando
December 16, 2022
Transcript
23 slides
-
2 What is discharge summary? summarize Inpatient reports Radiology reports
Referral documents Nursing records ……………… ……………… ……………… ……………… ……………… ……………… In-hospital Discharge -
3 What is discharge summary? - Written day by day
during the patient's stay. - Difficult for other physicians to read - A short summary is created at the patient's discharge. - Important document, sharing information with other institutions - Creation is a hard work -
4 Discharge summary steals physiciansʼ time Seeing patients : Paperwork
36~40 h/week vs. 16~18 h/week https://www.medscape.com/slideshow/2019-compensation-overview-6011286 -
- Support for discharge summary creation by AI - We
applied an abstractive summarization method to a portion of the largest multi-institutional healthcare data in Japan held by the National Hospital Organization. 6 Our motivation 144 hospitals in Japan Security room Server National Hospital Organization (NHO) -
- Healthcare data has more bias than other domains <7BSPRVBVY
> 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT 7 Research question -
- Healthcare data has more bias than other domains <7BSPRVBVY
> 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT Meta-information may be useful for discharge summary generation models? 8 Research question -
- Healthcare data has more bias than other domains <7BSPRVBVY
> 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT Meta-information may be useful for discharge summary generation models? 5PWFSJGZ • *UJTJNQPSUBOUUPDPMMFDUBWBSJFUZPGNFUBJOGPSNBUJPO EVSJOHUIFEBUBJOGSBTUSVDUVSFEFWFMPQNFOUQIBTF • *UJTJNQPSUBODFUPUFTUUIBUJODMVEFTNVMUJJOTUJUVUJPOBM NVMUJEJTFBTF FUD UPSFEVDFEBUBCJBT 9 Research question -
- /PTUVEJFTBOBMZ[JOHUIFJNQBDUPGNFUBJOGPSNBUJPOPO EJTDIBSHFTVNNBSZHFOFSBUJPO 1. We found that a model encoding
meta-information generates higher quality summaries. 2. We found that a model encoding disease information can produce proper disease and symptom words following the source. In addition, we found that the model using physician and hospital information can generate proper symbols. 10 Contributions -
- /PTUVEJFTBOBMZ[JOHUIFJNQBDUPGNFUBJOGPSNBUJPOPO EJTDIBSHFTVNNBSZHFOFSBUJPO 1. We found that a model encoding
meta-information generates higher quality summaries. 2. We found that a model encoding disease information can produce proper disease and symptom words following the source. In addition, we found that the model using physician and hospital information can generate proper symbols. - /PTUVEJFTBCTUSBDUEJTDIBSHFTVNNBSZJO+BQBOFTF 3. We are the first to apply the abstractive summarization method to generate Japanese discharge summaries. 11 Contributions -
- [Towards Automatic Generation of Context-Based Abstractive Discharge Summaries for
Supporting Transition of Care. Diana Diaz et al. 2020.] They tried to introduce meta-information into a model, but its impact has not been investigated. 12 Prior work Encode patient health records with tf-idf grouping Encode the writer's information -
- Our method is based on the encoder-decoder transformer model.
- We employ Longformer because both input and output are long texts. - To introduce meta-information, a new feature embedding layer is added to the model's input following previous work (BERT and XLM). 13 Methods: Our model Disease Hospital Physician Stay length 4 types -
- Extracted from the header of XML or free text
fields. 1. Disease • 1SJNBSZEJTFBTFJTPCUBJOFEGSPNUIFGSFFUFYUGJFME • &ODPEJOHUIFGJSTUBMQIBCFUBOEUIFGPMMPXJOHEJHJUTPG*$% DPEFT YY LFZTJOEJDUJPOBSZ • *GUIFSFJTOP*$%DPEF QBSTFUIFEJTFBTFOBNFXJUI.F$BC + .F%JD UPPCUBJOB*$%DPEF .F$BC JTB+BQBOFTFNPSQIPMPHJDBMBOBMZ[FSBOEDBOFYUSBDUWBSJPVT NFEJDBMJOGPSNBUJPOCZVTJOH+.F%JD BFYUFSOBMEJDUJPOBSZ 14 Methods: Extracting meta-information -
2. Length of stay • 0CUBJOFECZDBMDVMBUJOHGSPNUIFEBUFPGBENJTTJPOBOEEJTDIBSHF PG9.- • &ODPEFJOVQUP EBZT
LFZT 3. Physician • 0CUBJOFEQIZTJDJBO*%TGSPN9.- • 8FIBTIFEUIFQIZTJDJBO*%TJOUPHSPVQTDPOUBJOJOHQFPQMF FBDI 4. Hospital • 0CUBJOFEIPTQJUBM*%TGSPN9.- • 'JWFIPTQJUBMT 15 Methods: Extracting meta-information -
- A part of NHO data. - For use in
the experiment, free-text fields were extracted from both the inpatient record and the discharge summary. - The data are randomly split into 22,630, 1,000, and 1,000 for train, validation, and test, respectively. 16 Dataset -
- ROUGE-1 F1, ROUGE-2 F1, ROUGE-L F1 • 5IFTFBSFHSBN HSBN
BOEMPOHFTUDPNNPOTVCTFRVFODF CBTFE FWBMVBUJPONFUSJDT SFTQFDUJWFMZ 306(&' 17 Evaluation metrics .PEFMl"NFSJDBOTBSFFYQPTFEUPUIF$07*%z 3FGl"NFSJDBOTXFSFFYQPTFEUPUIF&CPMBWJSVTz ˣ .PEFM< "NFSJDBOT "NFSJDBOTBSF BSFFYQPTFE FYQPTFEUP UPUIF UIF$07*% > 3FG< "NFSJDBOT "NFSJDBOTXFSF XFSFFYQPTFE FYQPTFEUP UPUIF UIF&CPMB &CPMB WJSVT > ˣ 3FDBMM 1SFDJTJPO ˣ ' -
- ROUGE-1 F1, ROUGE-2 F1, ROUGE-L F1 • 5IFTFBSFHSBN HSBN
BOEMPOHFTUDPNNPOTVCTFRVFODF CBTFE FWBMVBUJPONFUSJDT SFTQFDUJWFMZ 306(&' - BertScore, BLEURT • 7FDUPSCBTFEFWBMVBUJPONFUSJDTGPSOBUVSBMMBOHVBHFHFOFSBUJPO • 5IFZDBONFBTVSFUIFTJNJMBSJUZPGNFBOJOHBOEOBUVSBMOFTTUP IVNBOT • #-&635XBTUSBJOFEPOIVNBOFWBMVBUJPOPGNBDIJOFUSBOTMBUJPO CVUJUJTBMTPVTFGVMGPSUIFTVNNBSJ[BUJPOUBTL<(BTLFMM > 18 Evaluation metrics .PEFMl"NFSJDBOTBSFFYQPTFEUPUIF$07*%z 3FGl"NFSJDBOTXFSFFYQPTFEUPUIF&CPMBWJSVTz ˣ .PEFM< "NFSJDBOT "NFSJDBOTBSF BSFFYQPTFE FYQPTFEUP UPUIF UIF$07*% > 3FG< "NFSJDBOT "NFSJDBOTXFSF XFSFFYQPTFE FYQPTFEUP UPUIF UIF&CPMB &CPMB WJSVT > ˣ 3FDBMM 1SFDJTJPO ˣ ' -
- Disease was best. • *$%POUPMPHZFGGJDJFOUMZDMVTUFSFEHSPVQTXJUITJNJMBSFYQSFTTJPOT - All models with
meta-information outperform vanilla in BS and BR. • 0VUQVUTPGNPEFMXJUINFUBJOGPSNBUJPOBSFNPSFOBUVSBMGPSIVNBOT 19 Results #FSU4DPSF #-&635 All features is all four types features are added to the embedding layer (4*Feature embeddings) -
- Disease was best. • *$%POUPMPHZFGGJDJFOUMZDMVTUFSFEHSPVQTXJUITJNJMBSFYQSFTTJPOT - All models with
meta-information outperform vanilla in BS and BR. • 0VUQVUTPGNPEFMXJUINFUBJOGPSNBUJPOBSFNPSFOBUVSBMGPSIVNBOT - All features is the lowest in BS. • 5IJTNBZCFEVFUPMBDLPGUSBJOJOHEBUBCFDBVTFUIFBMMGFBUVSFTIBT RVBESVQMFQBSBNFUFS 20 Results #FSU4DPSF #-&635 All features is all four types features are added to the embedding layer (4*Feature embeddings) -
- Are the generated words included in the Gold Summaries?
• -BCFMJOHCZ.F$BC +.FEJD - A model encoded disease information generates more suitable disease and symptom words. - Models encoded with physician and hospital information improved the precision of symbol generation. • %JGGFSFOUIPTQJUBMTBOEEPDUPSTIBWFEJGGFSFOUXSJUJOHIBCJUT CVMMFU QPJOUTTVDIBT<・c * c - > QVODUVBUJPOTVDIBT<、c,c.c。> 21 Analysis: precisions of words -
- We conducted a discharge summary generation experiment by adding
four types of information to Longformer and verified the impact of the meta-information. - All four types of information exceeded the performance of the vanilla Longformer model, with the highest performance achieved by encoding disease information. 22 Conclusion