About searchnos

2023-03-10 Nostr Meetup #1

This is an English translation of the material I presented at "Nostr勉強会 #1" meetup.

Transcript

1. About searchnos
   npub1q7qyk7rvdga5qzmmyrvmlj29qd0n45snmfuhkrzsj4rk0sm4c4psvqwt9c
   2023-03-10 Nostr Meetup #1
   

   View Slide

2. @darashi
   Author of nostrbuzzs
   

   View Slide

3. What is searchnos?
   https://github.com/darashi/searchnos
   I want search capability in Nostrbuzzs, but...
   There are not many relays that support NIP-50 (Search Capability), so I made one
   myself.
   It is a relay-like server, but not a full-featured relay.
   

   View Slide

4. View Slide

5. Thoughts
   Seems difficult to build relays on our own
   It seems to be already difficult even just following the current spec
   It will be difficult to continue to support NIP in the future
   It will be difficult to operate relays on our own
   It needs to be added to the user's relay list and have them throw an events to
   the relay.
   It will be hard to operate at that quality.
   It would be good to retrieve the events from an existing relay.
   

   View Slide

6. How it works
   searchnos
   indexer searcher
   relay-1
   relay-2
   Elasticsearch
   web-browser
   

   View Slide

7. Indexer
   Connects to the specified relay and subscribes to kind 1 and kind 5. Update
   Elasticsearch state according to events.
   Searcher
   When the REQ comes in:
   Throw a query to Elasticsearch, return results to the client, and send EOSE at
   the end.
   Add the connection to the (internal) query subscriber list.
   Polling at regular intervals:
   Throw queries with subscribers to Elasticsearch sequentially, and broadcast
   the new matches to the subscribers.
   

   View Slide

8. client
   searcher Elasticsearch indexer relay
   loop
   loop
   REQ w/"search"
   1
   query
   2
   response
   3
   EVENT (if matched)
   4
   EVENT (if matched)
   5
   EOSE
   6
   kind:1
   7
   index request
   8
   wait
   9
   query
   10
   response
   11
   EVENT (if matched & unread)
   12
   CLOSE
   13
   client
   searcher Elasticsearch indexer relay
   

   View Slide

9. Interesting points
   Queries from clients subscribing to the same query can be combined into one and
   thrown to Elasticsearch.
   As for buzz phrases, it is often the case that several people will subscribe the
   same query.
   Can perform searches with complex conditions, as it uses "Simple query string
   query"
   https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-
   string-query.html
   

   View Slide

10. Conclusion
    A relay-like server that collects notes from existing relays and provides a search
    function for the NIP-50 was created.
    

    View Slide

11. Future issues
    Support regular filter conditions (NIP-01).
    Currently, only limit is supported.
    support kind 0 (NIP-01)...?
    A bit more intelligent query scheduling, such as polling queries with more
    subscribers more often.
    Support language detection and filtering by language...?
    Support spam filtering...? How to...?
    Since we have NIP-56, it might be possible to define pubkey of trusted users
    and learn spam filter from their kind 1984.
    It would be hard to operate a search engine...
    

    View Slide