Upgrade to Pro — share decks privately, control downloads, hide ads and more …

About searchnos

About searchnos

2023-03-10 Nostr Meetup #1

This is an English translation of the material I presented at "Nostr勉強会 #1" meetup.

Yoji Shidara

March 10, 2023

More Decks by Yoji Shidara

Other Decks in Technology


  1. What is searchnos? https://github.com/darashi/searchnos I want search capability in Nostrbuzzs,

    but... There are not many relays that support NIP-50 (Search Capability), so I made one myself. It is a relay-like server, but not a full-featured relay.
  2. Thoughts Seems difficult to build relays on our own It

    seems to be already difficult even just following the current spec It will be difficult to continue to support NIP in the future It will be difficult to operate relays on our own It needs to be added to the user's relay list and have them throw an events to the relay. It will be hard to operate at that quality. It would be good to retrieve the events from an existing relay.
  3. Indexer Connects to the specified relay and subscribes to kind

    1 and kind 5. Update Elasticsearch state according to events. Searcher When the REQ comes in: Throw a query to Elasticsearch, return results to the client, and send EOSE at the end. Add the connection to the (internal) query subscriber list. Polling at regular intervals: Throw queries with subscribers to Elasticsearch sequentially, and broadcast the new matches to the subscribers.
  4. client searcher Elasticsearch indexer relay loop loop REQ w/"search" 1

    query 2 response 3 EVENT (if matched) 4 EVENT (if matched) 5 EOSE 6 kind:1 7 index request 8 wait 9 query 10 response 11 EVENT (if matched & unread) 12 CLOSE 13 client searcher Elasticsearch indexer relay
  5. Interesting points Queries from clients subscribing to the same query

    can be combined into one and thrown to Elasticsearch. As for buzz phrases, it is often the case that several people will subscribe the same query. Can perform searches with complex conditions, as it uses "Simple query string query" https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query- string-query.html
  6. Conclusion A relay-like server that collects notes from existing relays

    and provides a search function for the NIP-50 was created.
  7. Future issues Support regular filter conditions (NIP-01). Currently, only limit

    is supported. support kind 0 (NIP-01)...? A bit more intelligent query scheduling, such as polling queries with more subscribers more often. Support language detection and filtering by language...? Support spam filtering...? How to...? Since we have NIP-56, it might be possible to define pubkey of trusted users and learn spam filter from their kind 1984. It would be hard to operate a search engine...