Percolatorを用いたカテゴリ分類

2018-08-02 Elasticsearch勉強会
https://www.meetup.com/ja-JP/Tokyo-Elastic-Fantastics/events/252971593/

Transcript

1. Percolator
   id:tarao @oarat
   2018-08-02
   Elasticsearch
   

   View Slide

2. (id:tarao @oarat)
   2008-10 ( )
   2010-04 (DC1)
   2013-04
   2013-12
   2017-02
   ◮
   ◮
   ,
   ◮
   ◮
   Scala
   

   View Slide

3. View Slide

4. ◮
   (2015 )
   ◮
   

   View Slide

5. View Slide

6. :
   

   View Slide

7. ◮ ◮
   ◮
   

   View Slide

8. pull
   ◮
   (Complement Naive Bayes)
   ◮
   ◮
   ◮
   N
   ◮
   ◮
   

   View Slide

9. pull
   

   View Slide

10. pull
    ◮
    ◮
    ◮
    Elasticsearch
    ◮
    Memcached
    

    View Slide

11. pull
    

    View Slide

12. pull
    ◮
    ◮
    ◮
    ◮
    ( )
    ◮
    N
    ( N )
    ◮
    

    View Slide

13. push
    ◮
    ( )
    ◮
    ( )
    

    View Slide

14. push
    

    View Slide

15. ◮
    ◮
    

    View Slide

16. ◮
    ◮
    ⇒ Percolator
    

    View Slide

17. Percolator
    https://en.wikipedia.org/wiki/Coffee_percolator
    

    View Slide

18. Percolator (Scala)
    createIndex(MainIndex).mappings(
    mapping("issue_item_query") fields percolatorField("query"))
    val query =
    issueItem.query.toCategorizingQuery.toFilteredQuery
    register(query) into(MainIndex / "issue_item_query", "query")
    withId issueItemId.toPercolatorId
    search(MainIndex / "issue_item_query") query {
    percolateQuery("entry", "query") usingSource
    entry.toEsEntry.toJsonString
    }
    

    View Slide

19. Percolator (API)
    PUT /MainIndex/issue_item_query
    { "mappings": { "_doc": { "properties": {
    "query": { "type": "percolator" } } } } }
    PUT /MainIndex/issue_item_query/issueItemId
    { "query": query }
    GET /MainIndex/issue_item_query/_search
    { "query": {
    "percolate": {
    "document_type": "entry",
    "field": "query",
    "document": entry } } }
    

    View Slide

20. ◮
    ◮
    OR
    

    View Slide

21. ◮
    ◮
    ◮
    ◮
    

    View Slide

22. ◮ ×8
    ◮ ×250+
    ◮ ×800+
    

    View Slide

23. View Slide

24. ◮
    ◮
    1
    ◮
    ◮
    ( )
    ◮
    ◮
    

    View Slide

25. ◮
    ◮
    ◮
    ?
    ( kuromoji )
    ◮
    ◮ →
    ◮
    ◮
    ◮
    Percolator
    ◮
    Percolator ?
    ◮
    

    View Slide

26. ◮
    push
    ◮
    Percolator
    ◮
    push
    ◮
    

    View Slide