Upgrade to Pro — share decks privately, control downloads, hide ads and more …

全文検索システムFESSを使った Web+DB PRESS 総集編の全文検索

全文検索システムFESSを使った Web+DB PRESS 総集編の全文検索

全文検索システムFESSを使って WEB+DB PRESS 総集編を全文検索しました。

169abc470372d3972631198c35737350?s=128

Tetsuya Tatsumi

June 03, 2018
Tweet

More Decks by Tetsuya Tatsumi

Other Decks in Technology

Transcript

  1. શจݕࡧγεςϜ'&44Λ࢖ͬͨ 8FC %#13&44૯ूฤͷશจݕࡧ Mishima.pm #3 @dokechin

  2. 8FC %#13&44

  3. ֓ཁ • 2000೥݄̍̎૑ץɻ • ٕज़ධ࿦ࣾൃߦɻִ݄ൃചʢۮ਺݄ʣ • Webٕज़ܥࡶࢽɻ • ݱࡏɹWPM

  4. ૯ूฤ • vol.1-24 2005೥ൃߦɹCD-ROM • vol.1-36 2007೥ൃߦɹCD-ROM • vol.1-60 2011೥ൃߦɹDVD-ROM

    • vol.1-72 2013೥ൃߦɹDVD-ROM • vol.1-84 2015೥ൃߦɹDVD-ROM • vol.1-102 2018೥ൃߦɹDVD-ROM
  5. WPM • PDF༰ྔ4.4G • αΠζॱʹฒ΂ΔͱWPM (240M) ɺWPM (76.8M)ɺvol.66(69.5M)

  6. '&44ͱ͸

  7. ֓ཁ • ΢Σϒ΍ϑΝΠϧγεςϜΛର৅ͱͨ͠Φʔϓϯιʔ εͷશจݕࡧγεςϜ • ApacheϥΠηϯε

  8. ΞʔΩςΫνϟ • Javaϕʔε • ݕࡧΤϯδϯ͸&MBTUJDTFBSDI • ݕࡧը໘ɺ؅ཧը໘ɺΫϩʔϥ΋ಉࠝ

  9. ؀ڥߏங

  10. %PDLFSͷ෮श IUUQTEPDTEPDLFSDPNFOHJOFEPDLFSPWFSWJFXEPDLFSBSDIJUFDUVSF

  11. ؀ڥߏஙखॱ • Docker for MacOS ͷઃఆ • ϝϞϦαΠζͷมߋ 2G->3.5G •

    ΠϝʔδDPEFMJCTGFTT͔Βίϯςφىಈ • FESSͷνϡʔχϯά • ࠷େϑΝΠϧαΠζ • ώʔϓαΠζ • Ϋϩʔϧઃఆ • Ϋϩʔϧ
  12. %PDLFSGPS.BD04 • https://store.docker.com/editions/ community/docker-ce-desktop-mac

  13. .FNPSZ4FUUJOH

  14. ίϯςφىಈ docker run -d -p 8080:8080 --name fess -v /Users/dokechin/Downloads/

    webdb_sp_001-102/:/webdb_sp_001-102 codelibs/fess:latest
  15. ύϥϝʔλνϡʔχϯά docker exec -it fess bash perl -i -pe 's/10485760/248000000/'

    /usr/ share/fess/app/WEB-INF/classes/crawler/ contentlength.xml perl -i -pe 's/-Xms1g/-Xms2g/' /etc/ elasticsearch/jvm.options perl -i -pe 's/-Xmx1g/-Xmx2g/' /etc/ elasticsearch/jvm.options exit
  16. QFSMJQF

  17. QFSMEPDQFSMSVO -i[extension] specifies that files processed by the "<>" construct

    are to be edited in-place.
  18. .0%FQBSTF perl -MO=Deparse -i -pe 's/-Xms1g/- Xms2g/' /etc/elasticsearch/jvm.options BEGIN {

    $^I = ""; } LINE: while (defined($_ = <ARGV>)) { s/-Xms1g/-Xms2g/; } continue { die "-p destination: $!\n" unless print $_; } -e syntax OK
  19. Ϋϩʔϧઃఆ

  20. ݕࡧ

  21. '&44ͷόʔδϣϯ্͕͕ Δͨͼʹύϥϝʔλͷνϡʔ χϯά͢Δͷʁ

  22. %PDLFSpMFʹࠩ෼Λهड़ FROM codelibs/fess:latest LABEL maintainer "dokechin" RUN perl -i -pe

    's/10485760/248000000/' /usr/share/ fess/app/WEB-INF/classes/crawler/contentlength.xml RUN perl -i -pe 's/-Xms1g/-Xms2g/' /etc/elasticsearch/ jvm.options RUN perl -i -pe 's/-Xmx1g/-Xmx2g/' /etc/elasticsearch/ jvm.options
  23. ΠϝʔδϏϧυɹˍɹ36/ # FESSͷόʔδϣϯΞοϓͷλΠϛϯάͰ docker build -t dokechin/fess . docker run

    -d -p 8080:8080 --name fess -v /Users/dokechin/Downloads/ webdb_sp_001-102/:/webdb_sp_001-102 dokechin/fess
  24. ࣮͸ѱखɻ ઃఆมߋΛ%PDLFSΠϝʔ δʹ͠ͳ͍ʂ

  25. ઃఆϑΝΠϧΛϩʔΧϧʹอଘ docker cp fess:/usr/share/ fess/app/WEB-INF/classes/ crawler/contentlength.xml .

  26. ىಈ࣌ʹ౉͢ docker run -d -p 8080:8080 --name fess -e 'ES_JAVA_OPTS="-Xms2g

    - Xmx2g"' -v $PWD/contentlength.xml:/ usr/share/fess/app/WEB-INF/classes/ crawler/contentlength.xml -v $PWD/ webdb_sp/:/webdb_sp codelibs/fess