Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Is In-hospital Meta-information Useful for Abstractive Discharge Summary Generation?

ando
December 16, 2022

Is In-hospital Meta-information Useful for Abstractive Discharge Summary Generation?

Slides of my presentation at TAAI 2022.

ando

December 16, 2022
Tweet

More Decks by ando

Other Decks in Research

Transcript

  1. 2 What is discharge summary? summarize Inpatient reports Radiology reports

    Referral documents Nursing records ……………… ……………… ……………… ……………… ……………… ……………… In-hospital Discharge
  2. 3 What is discharge summary? - Written day by day

    during the patient's stay. - Difficult for other physicians to read - A short summary is created at the patient's discharge. - Important document, sharing information with other institutions - Creation is a hard work
  3. 4 Discharge summary steals physiciansʼ time Seeing patients : Paperwork

    36~40 h/week vs. 16~18 h/week https://www.medscape.com/slideshow/2019-compensation-overview-6011286
  4. - Support for discharge summary creation by AI - We

    applied an abstractive summarization method to a portion of the largest multi-institutional healthcare data in Japan held by the National Hospital Organization. 6 Our motivation 144 hospitals in Japan Security room Server National Hospital Organization (NHO)
  5. - Healthcare data has more bias than other domains <7BSPRVBVY

    > 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT 7 Research question
  6. - Healthcare data has more bias than other domains <7BSPRVBVY

    > 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT Meta-information may be useful for discharge summary generation models? 8 Research question
  7. - Healthcare data has more bias than other domains <7BSPRVBVY

    > 0VSPCTFSWBUJPOT • %JGGFSFOUIPTQJUBMTBOEQIZTJDJBOTIBWFEJGGFSFOUQBUUFSOT PG DMJOJDBMSFDPSET • 4FWFSFEJTFBTFTIBWFMBSHFSTVNNBSJFT Meta-information may be useful for discharge summary generation models? 5PWFSJGZ • *UJTJNQPSUBOUUPDPMMFDUBWBSJFUZPGNFUBJOGPSNBUJPO EVSJOHUIFEBUBJOGSBTUSVDUVSFEFWFMPQNFOUQIBTF • *UJTJNQPSUBODFUPUFTUUIBUJODMVEFTNVMUJJOTUJUVUJPOBM  NVMUJEJTFBTF FUD UPSFEVDFEBUBCJBT 9 Research question
  8. - /PTUVEJFTBOBMZ[JOHUIFJNQBDUPGNFUBJOGPSNBUJPOPO EJTDIBSHFTVNNBSZHFOFSBUJPO 1. We found that a model encoding

    meta-information generates higher quality summaries. 2. We found that a model encoding disease information can produce proper disease and symptom words following the source. In addition, we found that the model using physician and hospital information can generate proper symbols. 10 Contributions
  9. - /PTUVEJFTBOBMZ[JOHUIFJNQBDUPGNFUBJOGPSNBUJPOPO EJTDIBSHFTVNNBSZHFOFSBUJPO 1. We found that a model encoding

    meta-information generates higher quality summaries. 2. We found that a model encoding disease information can produce proper disease and symptom words following the source. In addition, we found that the model using physician and hospital information can generate proper symbols. - /PTUVEJFTBCTUSBDUEJTDIBSHFTVNNBSZJO+BQBOFTF 3. We are the first to apply the abstractive summarization method to generate Japanese discharge summaries. 11 Contributions
  10. - [Towards Automatic Generation of Context-Based Abstractive Discharge Summaries for

    Supporting Transition of Care. Diana Diaz et al. 2020.] They tried to introduce meta-information into a model, but its impact has not been investigated. 12 Prior work Encode patient health records with tf-idf grouping Encode the writer's information
  11. - Our method is based on the encoder-decoder transformer model.

    - We employ Longformer because both input and output are long texts. - To introduce meta-information, a new feature embedding layer is added to the model's input following previous work (BERT and XLM). 13 Methods: Our model Disease Hospital Physician Stay length 4 types
  12. - Extracted from the header of XML or free text

    fields. 1. Disease • 1SJNBSZEJTFBTFJTPCUBJOFEGSPNUIFGSFFUFYUGJFME • &ODPEJOHUIFGJSTUBMQIBCFUBOEUIFGPMMPXJOHEJHJUTPG*$% DPEFT YY LFZTJOEJDUJPOBSZ • *GUIFSFJTOP*$%DPEF QBSTFUIFEJTFBTFOBNFXJUI.F$BC + .F%JD UPPCUBJOB*$%DPEF .F$BC JTB+BQBOFTFNPSQIPMPHJDBMBOBMZ[FSBOEDBOFYUSBDUWBSJPVT NFEJDBMJOGPSNBUJPOCZVTJOH+.F%JD BFYUFSOBMEJDUJPOBSZ 14 Methods: Extracting meta-information
  13. 2. Length of stay • 0CUBJOFECZDBMDVMBUJOHGSPNUIFEBUFPGBENJTTJPOBOEEJTDIBSHF PG9.- • &ODPEFJOVQUP EBZT

     LFZT  3. Physician • 0CUBJOFEQIZTJDJBO*%TGSPN9.- • 8FIBTIFEUIFQIZTJDJBO*%TJOUPHSPVQTDPOUBJOJOHQFPQMF FBDI 4. Hospital • 0CUBJOFEIPTQJUBM*%TGSPN9.- • 'JWFIPTQJUBMT 15 Methods: Extracting meta-information
  14. - A part of NHO data. - For use in

    the experiment, free-text fields were extracted from both the inpatient record and the discharge summary. - The data are randomly split into 22,630, 1,000, and 1,000 for train, validation, and test, respectively. 16 Dataset
  15. - ROUGE-1 F1, ROUGE-2 F1, ROUGE-L F1 • 5IFTFBSFHSBN HSBN

    BOEMPOHFTUDPNNPOTVCTFRVFODF CBTFE FWBMVBUJPONFUSJDT SFTQFDUJWFMZ 306(&' 17 Evaluation metrics .PEFMl"NFSJDBOTBSFFYQPTFEUPUIF$07*%z 3FGl"NFSJDBOTXFSFFYQPTFEUPUIF&CPMBWJSVTz ˣ .PEFM<  "NFSJDBOT  "NFSJDBOTBSF  BSFFYQPTFE  FYQPTFEUP  UPUIF UIF$07*% > 3FG<  "NFSJDBOT  "NFSJDBOTXFSF  XFSFFYQPTFE  FYQPTFEUP  UPUIF UIF&CPMB  &CPMB WJSVT > ˣ 3FDBMM 1SFDJTJPO ˣ '
  16. - ROUGE-1 F1, ROUGE-2 F1, ROUGE-L F1 • 5IFTFBSFHSBN HSBN

    BOEMPOHFTUDPNNPOTVCTFRVFODF CBTFE FWBMVBUJPONFUSJDT SFTQFDUJWFMZ 306(&' - BertScore, BLEURT • 7FDUPSCBTFEFWBMVBUJPONFUSJDTGPSOBUVSBMMBOHVBHFHFOFSBUJPO • 5IFZDBONFBTVSFUIFTJNJMBSJUZPGNFBOJOHBOEOBUVSBMOFTTUP IVNBOT • #-&635XBTUSBJOFEPOIVNBOFWBMVBUJPOPGNBDIJOFUSBOTMBUJPO  CVUJUJTBMTPVTFGVMGPSUIFTVNNBSJ[BUJPOUBTL<(BTLFMM > 18 Evaluation metrics .PEFMl"NFSJDBOTBSFFYQPTFEUPUIF$07*%z 3FGl"NFSJDBOTXFSFFYQPTFEUPUIF&CPMBWJSVTz ˣ .PEFM<  "NFSJDBOT  "NFSJDBOTBSF  BSFFYQPTFE  FYQPTFEUP  UPUIF UIF$07*% > 3FG<  "NFSJDBOT  "NFSJDBOTXFSF  XFSFFYQPTFE  FYQPTFEUP  UPUIF UIF&CPMB  &CPMB WJSVT > ˣ 3FDBMM 1SFDJTJPO ˣ '
  17. - Disease was best. • *$%POUPMPHZFGGJDJFOUMZDMVTUFSFEHSPVQTXJUITJNJMBSFYQSFTTJPOT - All models with

    meta-information outperform vanilla in BS and BR. • 0VUQVUTPGNPEFMXJUINFUBJOGPSNBUJPOBSFNPSFOBUVSBMGPSIVNBOT 19 Results #FSU4DPSF #-&635 All features is all four types features are added to the embedding layer (4*Feature embeddings)
  18. - Disease was best. • *$%POUPMPHZFGGJDJFOUMZDMVTUFSFEHSPVQTXJUITJNJMBSFYQSFTTJPOT - All models with

    meta-information outperform vanilla in BS and BR. • 0VUQVUTPGNPEFMXJUINFUBJOGPSNBUJPOBSFNPSFOBUVSBMGPSIVNBOT - All features is the lowest in BS. • 5IJTNBZCFEVFUPMBDLPGUSBJOJOHEBUBCFDBVTFUIFBMMGFBUVSFTIBT RVBESVQMFQBSBNFUFS 20 Results #FSU4DPSF #-&635 All features is all four types features are added to the embedding layer (4*Feature embeddings)
  19. - Are the generated words included in the Gold Summaries?

    • -BCFMJOHCZ.F$BC +.FEJD - A model encoded disease information generates more suitable disease and symptom words. - Models encoded with physician and hospital information improved the precision of symbol generation. • %JGGFSFOUIPTQJUBMTBOEEPDUPSTIBWFEJGGFSFOUXSJUJOHIBCJUT CVMMFU QPJOUTTVDIBT<・c * c - > QVODUVBUJPOTVDIBT<、c,c.c。>  21 Analysis: precisions of words
  20. - We conducted a discharge summary generation experiment by adding

    four types of information to Longformer and verified the impact of the meta-information. - All four types of information exceeded the performance of the vanilla Longformer model, with the highest performance achieved by encoding disease information. 22 Conclusion