Percolatorを用いたカテゴリ分類

Transcript

1. 1.

   Percolator id:tarao @oarat 2018-08-02 Elasticsearch

2. 2.

   (id:tarao @oarat) 2008-10 ( ) 2010-04 (DC1) 2013-04 2013-12 2017-02

   ◮ ◮ , ◮ ◮ Scala
3. 3.

   None
4. 4.

   ◮ (2015 ) ◮

5. 5.

   None
6. 6.

   :

7. 7.

   ◮ ◮ ◮

8. 8.

   pull ◮ (Complement Naive Bayes) ◮ ◮ ◮ N ◮

   ◮
9. 9.

   pull

10. 10.

    pull ◮ ◮ ◮ Elasticsearch ◮ Memcached

11. 11.

    pull

12. 12.

    pull ◮ ◮ ◮ ◮ ( ) ◮ N (

    N ) ◮
13. 13.

    push ◮ ( ) ◮ ( )

14. 14.

    push

15. 15.

    ◮ ◮

16. 16.

    ◮ ◮ ⇒ Percolator

17. 17.

    Percolator https://en.wikipedia.org/wiki/Coffee_percolator

18. 18.

    Percolator (Scala) createIndex(MainIndex).mappings( mapping("issue_item_query") fields percolatorField("query")) val query = issueItem.query.toCategorizingQuery.toFilteredQuery

    register(query) into(MainIndex / "issue_item_query", "query") withId issueItemId.toPercolatorId search(MainIndex / "issue_item_query") query { percolateQuery("entry", "query") usingSource entry.toEsEntry.toJsonString }
19. 19.

    Percolator (API) PUT /MainIndex/issue_item_query { "mappings": { "_doc": { "properties":

    { "query": { "type": "percolator" } } } } } PUT /MainIndex/issue_item_query/issueItemId { "query": query } GET /MainIndex/issue_item_query/_search { "query": { "percolate": { "document_type": "entry", "field": "query", "document": entry } } }
20. 20.

    ◮ ◮ OR

21. 21.

    ◮ ◮ ◮ ◮

22. 22.

    ◮ ×8 ◮ ×250+ ◮ ×800+

23. 23.

    None
24. 24.

    ◮ ◮ 1 ◮ ◮ ( ) ◮ ◮

25. 25.

    ◮ ◮ ◮ ? ( kuromoji ) ◮ ◮ →

    ◮ ◮ ◮ Percolator ◮ Percolator ? ◮
26. 26.

    ◮ push ◮ Percolator ◮ push ◮