$30 off During Our Annual Pro Sale. View Details »

今日から始める全文検索

 今日から始める全文検索

2019年1月22日に開催された【とらのあな主催】オタクが最新技術を追うライトニングトークイベント2回目のライトニングトーク資料です。

虎の穴ラボ株式会社

January 22, 2019
Tweet

More Decks by 虎の穴ラボ株式会社

Other Decks in Technology

Transcript



  1. Copyright (C) 2019 Toranoana Inc. All Right Reserved.


    ΦλΫ͕࠷৽ٕज़Λ௥͏ϥΠτχϯάτʔΫΠϕϯτ̎ճ໨



    View Slide



  2. Copyright (C) 2019 Toranoana Inc. All Right Reserved.

    • 201810
    • Fantia

    Ruby, JavaScript
    Java
    2


    View Slide



  3. Copyright (C) 2019 Toranoana Inc. All Right Reserved.








    3

    View Slide



  4. Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    <,+%'>
    $)-2<&10: #0:
    69

    <(;>
    ● $)-2753
    8
    0*
    ● $)-2"/
    .4
    ! *
    4

    View Slide

  5. EB EB EB EB E
    B EB
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    0<>C
    0<>CH###
    D: Full text searchI
    &,(*-'F;<=H)$%
    +I?8<72">C )$%+4
    >C3.)$%+1<72>CA
    F;<= <=5! 0<"9G
    >C:6/@ !
    HWikipediaI
    5
    F;<=()$%+)5!
    0<7"9G>C

    View Slide



  6. Copyright (C) 2019 Toranoana Inc. All Right Reserved.




    6

    View Slide

  7. $" $" $" $" $
    " $"
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    7
    !#
    !#
    !#
    Apache Lucene)Java*
    Whoosh)Python*

    Apache Solr
    Elasticsearch
    Groonga)C&'*

    Google!#
    Fess


    %

    !#(

    View Slide

  8. $
    " $
    " $
    " $
    " $

    " $
    "
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    #

    #% #&
    ≒ # !
    8

    View Slide



  9. Copyright (C) 2019 Toranoana Inc. All Right Reserved.



    • grep

    9

    View Slide



  10. Copyright (C) 2019 Toranoana Inc. All Right Reserved.

    grep

    10

    View Slide

  11. 41 41 41 41 4
    1 41
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    7-,2.)
    <.)>
    • "' 6
    9 %8*( 7/&
    5+:
    <$>
    • Linuxgrep!
    • #30
    Ctrl+F,2
    11

    View Slide



  12. Copyright (C) 2019 Toranoana Inc. All Right Reserved.





    12

    View Slide

  13. )$ )$ )$ )$ )
    $ )$
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    '-
    ,!*!%!&#


    13
    Ruby(,Ruby +2"

    View Slide

  14. %! %! %! %! %
    ! %!
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    ##
    < >
    #&#(*(
    #)
    <>
    • Google"#

    • RDB (B-Tree)
    • #
    ('$ )
    14

    View Slide

  15. $! $! $! $! $
    ! $!
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    %#
    "
    "
    '&"
    (
    15

    View Slide

  16. 3/ 3/ 3/ 3/ 3
    / 3/
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    72-%
    <-%>
    56(1#$)&)
    .2*04,9
    +18"<8=&)!>
    :&) =2,3>
    1#(';"!
    16

    View Slide

  17. ($ ($ ($ ($ (
    $ ($
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    &
    !
    "
    "ID
    1
    2 -
    3
    17
    )+.&+/ "ID
    2, 3
    1, 2
    3
    1, 2, 3
    1, 3
    - 2
    2
    & (,')
    N-Gram
    %*#
    ."/

    View Slide

  18. .
    * .
    * .
    * .
    * .

    * .
    *
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    &)+
    ,
    <> )+
    → '!
    &(%
    / 03+$04 &(ID
    2, 3
    1, 2
    3
    1, 2, 3
    " 1, 3
    2 2
    2
    &(ID #
    1 "
    2 2
    3 "
    18
    +$ (1-)
    &(

    View Slide

  19. &# &# &# &# &
    # &#
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    ') "
    )+
    ()
    .%))) ,!$
    *-
    19
    ⇒)

    View Slide

  20. ., ., ., ., .
    , .,
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    /2%$*
    25
    12
    9&(2 26 2
    74!38
    20

    ...
    $*+
    • "#-0)7'8
    • N-Gram

    View Slide

  21. (& (& (& (& (
    & (&
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    ')$

    +"%
    !#*,
    '-
    ,"-
    21

    View Slide



  22. Copyright (C) 2019 Toranoana Inc. All Right Reserved.

    <>


    ...

    !6
    22

    View Slide

  23. 50 50 50 50 5
    0 50
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    $(16*/%
    <.>
    +2#

    83:N-Gram"
    +2
    '+2!4
    <,.>
    9)&4 +2
    7
    ;+2-<
    23

    View Slide



  24. Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    N-Gram
    !"

    N

    24

    View Slide



  25. Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    N-Gram
    <> 2-Gram
    "

    ↓ 2 ...
    "
    "


    !#10
    25

    View Slide

  26. :4 :4 :4 :4 :
    4 :4
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    N-Gram3(
    < 2>
    /61A?9
    %9 /67.
    < B
    <02>
    /6"
    >8@')5;-#
    N(,&+$*)!= /6
    26

    View Slide

  27. 63 63 63 63 6
    3 63
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    04#!
    -*04 "$@-*81
    04:(,
    047 @%<'-*
    04 -*A
    >).=-*+29;&?
    27

    045/

    View Slide

  28. 82 82 82 82 8
    2 82
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    15!
    . >
    • )+4:/A9$ <→. >
    . >&,315 >&
    ,-15 ?(#7";@
    • 2-GramA9$ <→. >
    >&,-1560$

    =%*'
    ⇨15
    28

    View Slide



  29. Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    N-
    Gram
    29



    N-Gram

    View Slide

  30. 94 94 94 94 9
    4 94
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    &"
    • &,06BA6(#

    C?.@(D
    • &,066( >8!
    • >8! :'=#-)+5;
    /N-GramE
    – 0627< ⇒ N-Gram%
    – * ⇒ )+5;/%
    – 0621*06
    ⇒ N-Gram + )+5;/ $3 30

    View Slide

  31. %! %! %! %! %
    ! %!
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    %!
    Fantia
    # $
    # '
    • "& ( N-Gram

    31

    View Slide









  32. Copyright (C) 2019 Toranoana Inc. All Right Reserved.


    32

    View Slide