今日から始める全文検索 その２

2019年2月19日に開催されたオタクが最新技術を追うライトニングトークイベン 3回目のライトニングトーク資料です。

Transcript

1. 
   
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   
   
   
   
   ~ ~
   ΦλΫ͕࠷৽ٕज़Λ௥͏ϥΠτχϯάτʔΫΠϕϯτճ໨
   
   
   

   View Slide

2. .* .* .* .* .
   * .*
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   "%
   • 201810# .*)
   • Fantia2'31&
   • /0
   - Ruby, JavaScript, Java
   •
   - , SAO
   • $!
   - (,
   
   
   2
   -+
   

   View Slide

3. 
   
   
   
   
   
   
   
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   ● *%-(
   ● )..
   ○
   ○ /101'*
   ● /101)
   "
   
   !#
   
   &)2$3
   ● /101) +
   ● ,'-
   3
   
   

   View Slide

4. 
   
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   2747+'
   *%
   &!-"
   
   ● 0/ 1!#83965
   1$&.#'
   
   ● 27471$&.#(01
   ,/)'
   
   4
   
   

   View Slide

5. %" %" %" %" %
   " %"
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   (
   • !#+*
   #
   ,' )
   -
   • !##&$
   5
   

   View Slide

6. %! %! %! %! %
   ! %!
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   #
   
   ID
   1
   
   2
   
   *
   3
   
   7
   &(+#(, ID
   
   2, 3
   
   1, 2
   3
   1, 2, 3
   1, 3
   * 2
   2
   # ()$ )
   "'
   or
   N-Gram
   

   View Slide

7. 3. 3. 3. 3. 3
   . 3.
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   !&9
   • 8247'#%/5
   (
   N-Gram>
   – )0+16 ⇒ N-Gram
   – $ ⇒ #%/5(
   – )0+*$)0
   ⇒ N-Gram + #%/5( ,
   <>-; :"=
   8
   

   View Slide

8. 
   
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   
   
   
   
   9
   

   View Slide

9. $" $" $" $" $
   " $"
   Copyright (C) 2019 Toranoana Inc. All Right Reserved.
   1
   0
   !#
   !#
   !#
   Apache Lucene)Java*
   Whoosh)Python*
   
   Apache Solr
   Elasticsearch
   Groonga)C&'*
   
   Google!#
   Fess
   
   
   %
   
   !#(
   

   View Slide

10. $
    " $
    " $
    " $
    " $
    
    " $
    "
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    #
    
    #% #&
    ≒ # !
    11
    

    View Slide

11. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    
    
    1. Groonga
    
    2. Apache Lucene
    
    3. MySQL
    12
    

    View Slide

12. $ $ $ $ $
    $
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    1. Groonga!
    • #"
    
    Groonga &%
    
    13
    

    View Slide

13. F> F> F> F> F
    > F>
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    GroongaMN
    • L=C67' 67J;*:DM#% /
    9A?N
    • 9A% CIJ!!
    ")<,D
    • N-Gram23@H8(Mecab)&4)<,D
    • 50G.B- )
    !$K
    1MMroonga, PGroonga?N
    14
    +E(COGroonga#
    

    View Slide

14. <
    6 <
    6 <
    6 <
    6 <
    
    6 <
    6
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Mroonga@A
    • Groonga '
    MySQL$'&&
    • SQL!&Groonga)@SQL01
    =
    +/28A
    • MySQL
    *,3:@$#'"&7A)
    
    • Groonga5.)4
    • MySQL
    FULLTEXT& ?>
    15
    -;(9BGroonga%
    

    View Slide

15. 3 - 3 - 3 - 3 - 3
    - 3 -
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    PGroonga4
    5
    • PostgreSQL %$*1
    • Groonga, 1
    • LIKE+."4'#SQL!(
    &)/ 15
    16
    206Groonga
    

    View Slide

16. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Groonga
    • Mroonga PGroongaCloudSQL
    GCP RDSAWS
    
    
    17
    

    View Slide

17. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    2. Apache Lucene
    • Apache Lucene
    
    
    18
    

    View Slide

18. .+ .+ .+ .+ .
    + .+
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Apache Lucene
    2
    3
    • Apache1)Java/!%&,
    23
    • Java(jar)$
    • "(-
    19
    Apache LuceneApache Software Foundation*0#'#'
    

    View Slide

19. @; @; @; @; @
    ; @;
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Apache Solr
    H
    ''I
    • ApacheD9Lucene#'5='!
    • '"&'
    • C/.>(2006/~)*F1G
    • N-Gram02• Elasticsearch7$ '%&
    E+
    20
    Apache SolrApache Software Foundation:B-6 -6
    

    View Slide

20. 62 62 62 62 6
    2 62
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Elasticsearch
    : !
    ;
    • $"0(,4
    • Elastic19/Lucene!,4!
    • ! !
    • RESTful API CRUD*#&5:-8
    ) ;
    • %+.57':Kibana, Logstash3;
    21
    

    View Slide

21. )% )% )% )% )
    % )%
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Kibana-.
    • Elastic$+
    • Elasticsearch !&
    "'
    *#
    22
    ,( https://www.elastic.co/jp/products/kibana
    

    View Slide

22. OI OI OI OI O
    I OI
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Elasticsearch
    V )W
    • &"#" )'(J1V&"#0=SR
    W
    • N-Gram:.D2M
    • Apache Solr;GVv1.020149~W/$%
    NH7Q T9U-
    68
    • 4D3
    @L(!(AEC5*F,B
    23
    

    View Slide

23. @< @< @< @< @
    < @<
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Elasticsearch
    E)
    F
    • $!) 0)"15-
    – Amazon Elasticsearch Service
    • AWS968 ,:/? 4BA
    • AWS()'* =.&+3
    • #&(Kibana, Logstash
    • %))2>C7;D
    24
    

    View Slide

24. KF KF KF KF K
    F KF
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Elasticsearch
    M .N
    • '".!7.$8;2
    – Elastic CloudMAWS, GCPN
    • @:ElasticE;2
    %,'".!.$M.#J
    0AWS, GCP4DN
    • 3&*-1D6I
    • (..9HL>1
    • ?<#.)-++.=1
    • #.)- &G5
    •
    A/+.)-CBAWS
    25
    

    View Slide

25. E? E? E? E? E
    ? E?
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Elasticsearch
    H
    . I
    • (%.$6.& 7:1
    – ElasticsearchHGoogle Click to DeployI
    • A3* ! Elasticsearch"',4C
    • - !- D/D2@<
    9F
    • 5>'*-)..8BG;0=4C
    HD2- #.+
    9FI
    26
    

    View Slide

26. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    
    1. Groonga
    – Groonga"!
    – Mroonga"MySQLGroonga
    – Gproonga"PostgreSQLGroonga
    2. Apache Lucene
    – Apache Solr"
    – Elasticsearch"
    
    27
    

    View Slide

27. '
    # '
    # '
    # '
    # '
    
    # '
    #
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    3.
    
    …
    • MySQLHyper EstraierNamazu$
    (
    )"!
    MySQL
    &%
    29
    

    View Slide

28. '# '# '# '# '
    # '#
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    MySQLFULLTEXT
    
    • MySQL! &+v3.23.23*,
    • v5.7 )
    • N-Gram$((Mecab)"
    &
    • SQL%&
    • CloudSQLRDS"&
    30
    

    View Slide

29. ! ! ! ! !
    !
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    #
    1. %$
    &
    #
    2. "#
    31
    

    View Slide

30. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    
    
    32
    MySQL PostgreSQL
    
    Oracle
    

    View Slide

31. 
    
    
    
    
    
    
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    MySQL
    33
    MySQL
    
    
    FULLTEXT
    
    
    

    View Slide

32. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    PostgreSQL
    34
    PostgreSQL
    
    
    
    

    View Slide

33. 
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    Oracle
    
    35
    
    
    Oracle
    
    
    

    View Slide

34. , + , + , + , + ,
    + , +
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    -(
    
    MySQLMroonga
    /
    .*& %$FULLTEXT
    0
    PostgreSQLPGroonga
    "Elasticsearch
    #) Elasticsearch!'
    Elasticsearch
    36
    

    View Slide

35. #
    #
    #
    #
    #
    
    #
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    #
    
    Fantia
    !"
    !
    &
    <%> DB)CloudSQL'MySQL(
    →Mroonga
    
    Elasticsearch$
    37
    

    View Slide

36. 
    
    
    
    
    
    
    
    Copyright (C) 2019 Toranoana Inc. All Right Reserved.
    
    
    38
    

    View Slide