$30 off During Our Annual Pro Sale. View Details »

About searchnos

About searchnos

2023-03-10 Nostr Meetup #1

This is an English translation of the material I presented at "Nostr勉強会 #1" meetup.

Yoji Shidara

March 10, 2023
Tweet

More Decks by Yoji Shidara

Other Decks in Technology

Transcript

  1. About searchnos
    npub1q7qyk7rvdga5qzmmyrvmlj29qd0n45snmfuhkrzsj4rk0sm4c4psvqwt9c
    2023-03-10 Nostr Meetup #1

    View Slide

  2. @darashi
    Author of nostrbuzzs

    View Slide

  3. What is searchnos?
    https://github.com/darashi/searchnos
    I want search capability in Nostrbuzzs, but...
    There are not many relays that support NIP-50 (Search Capability), so I made one
    myself.
    It is a relay-like server, but not a full-featured relay.

    View Slide

  4. View Slide

  5. Thoughts
    Seems difficult to build relays on our own
    It seems to be already difficult even just following the current spec
    It will be difficult to continue to support NIP in the future
    It will be difficult to operate relays on our own
    It needs to be added to the user's relay list and have them throw an events to
    the relay.
    It will be hard to operate at that quality.
    It would be good to retrieve the events from an existing relay.

    View Slide

  6. How it works
    searchnos
    indexer searcher
    relay-1
    relay-2
    Elasticsearch
    web-browser

    View Slide

  7. Indexer
    Connects to the specified relay and subscribes to kind 1 and kind 5. Update
    Elasticsearch state according to events.
    Searcher
    When the REQ comes in:
    Throw a query to Elasticsearch, return results to the client, and send EOSE at
    the end.
    Add the connection to the (internal) query subscriber list.
    Polling at regular intervals:
    Throw queries with subscribers to Elasticsearch sequentially, and broadcast
    the new matches to the subscribers.

    View Slide

  8. client
    searcher Elasticsearch indexer relay
    loop
    loop
    REQ w/"search"
    1
    query
    2
    response
    3
    EVENT (if matched)
    4
    EVENT (if matched)
    5
    EOSE
    6
    kind:1
    7
    index request
    8
    wait
    9
    query
    10
    response
    11
    EVENT (if matched & unread)
    12
    CLOSE
    13
    client
    searcher Elasticsearch indexer relay

    View Slide

  9. Interesting points
    Queries from clients subscribing to the same query can be combined into one and
    thrown to Elasticsearch.
    As for buzz phrases, it is often the case that several people will subscribe the
    same query.
    Can perform searches with complex conditions, as it uses "Simple query string
    query"
    https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-
    string-query.html

    View Slide

  10. Conclusion
    A relay-like server that collects notes from existing relays and provides a search
    function for the NIP-50 was created.

    View Slide

  11. Future issues
    Support regular filter conditions (NIP-01).
    Currently, only limit is supported.
    support kind 0 (NIP-01)...?
    A bit more intelligent query scheduling, such as polling queries with more
    subscribers more often.
    Support language detection and filtering by language...?
    Support spam filtering...? How to...?
    Since we have NIP-56, it might be possible to define pubkey of trusted users
    and learn spam filter from their kind 1984.
    It would be hard to operate a search engine...

    View Slide