$30 off During Our Annual Pro Sale. View Details »

Percolatorを用いたカテゴリ分類

 Percolatorを用いたカテゴリ分類

INA Lintaro

August 02, 2018
Tweet

More Decks by INA Lintaro

Other Decks in Programming

Transcript

  1. Percolator
    id:tarao @oarat
    2018-08-02
    Elasticsearch

    View Slide

  2. (id:tarao @oarat)
    2008-10 ( )
    2010-04 (DC1)
    2013-04
    2013-12
    2017-02


    ,


    Scala

    View Slide

  3. View Slide


  4. (2015 )

    View Slide

  5. View Slide

  6. :

    View Slide

  7. ◮ ◮

    View Slide

  8. pull

    (Complement Naive Bayes)



    N


    View Slide

  9. pull

    View Slide

  10. pull



    Elasticsearch

    Memcached

    View Slide

  11. pull

    View Slide

  12. pull




    ( )

    N
    ( N )

    View Slide

  13. push

    ( )

    ( )

    View Slide

  14. push

    View Slide



  15. View Slide



  16. ⇒ Percolator

    View Slide

  17. Percolator
    https://en.wikipedia.org/wiki/Coffee_percolator

    View Slide

  18. Percolator (Scala)
    createIndex(MainIndex).mappings(
    mapping("issue_item_query") fields percolatorField("query"))
    val query =
    issueItem.query.toCategorizingQuery.toFilteredQuery
    register(query) into(MainIndex / "issue_item_query", "query")
    withId issueItemId.toPercolatorId
    search(MainIndex / "issue_item_query") query {
    percolateQuery("entry", "query") usingSource
    entry.toEsEntry.toJsonString
    }

    View Slide

  19. Percolator (API)
    PUT /MainIndex/issue_item_query
    { "mappings": { "_doc": { "properties": {
    "query": { "type": "percolator" } } } } }
    PUT /MainIndex/issue_item_query/issueItemId
    { "query": query }
    GET /MainIndex/issue_item_query/_search
    { "query": {
    "percolate": {
    "document_type": "entry",
    "field": "query",
    "document": entry } } }

    View Slide



  20. OR

    View Slide





  21. View Slide

  22. ◮ ×8
    ◮ ×250+
    ◮ ×800+

    View Slide

  23. View Slide



  24. 1


    ( )


    View Slide




  25. ?
    ( kuromoji )

    ◮ →



    Percolator

    Percolator ?

    View Slide


  26. push

    Percolator

    push

    View Slide