Design and learning of the Algolia API client

Transcript

1. Design and learnings of Algolia’s Scala API Rémy-Christophe Schermesser @El_Picador
2. A hosted search API that focuses on Developer and User Experiences
4. Replies in milliseconds A hosted search API
5. From anywhere A hosted search API Replies in milliseconds
6. With intuitive relevance A hosted search API From anywhere Replies in milliseconds
7. HOW DOES ALGOLIA WORK?
8. HOW DOES ALGOLIA WORK?
9. HOW DOES ALGOLIA WORK?
10. Algolia Today 15 regions 36 data centers 1500+ customers in 100+ countries 30B+ Write operations per month 12B+ User-generated queries per month
11. =
12. Cluster = 3 identical machines
13. to DSL or not to DSL?
14. Some will like it
15. Some will complain
16. DSL in Scala One on One
17. Remove all the parenthesis myObject.myMethod(myArg) myObject myMethod myArg myObject.myMethod(myArg).anotherMethod(anotherArg) myObject myMethod myArg anotherMethod anotherArg
18. clear.synonyms(of).index("toto").and(forwardToSlave) clear synonyms of index “toto” and forwardToSlave
19. sealed trait Of case object of extends Of case object clear { def synonyms(of: Of): ClearSynonymsDefinition = ClearSynonymsDefinition() } sealed trait ForwardToSlave case object forwardToSlaves extends ForwardToSlave case class ClearSynonymsDefinition { def and(option: ForwardToSlave) = copy(option = Some(option)) }
20. “object” is what you upload to Algolia, but a reserved keyword in scala def `object`() = ??? index into "toto" `object` MyObject("algolia", 1)
22. sealed trait Ranking object Ranking { case class asc(attribute: String) extends Ranking case class desc(attribute: String) extends Ranking case object typo extends Ranking }
24. • Futures are trendy • Future → synchronous is easy, not the other way around • Scala futures • for { f <- c.myMethod(); g <- f.doSomething() } yield g Why Futures?
25. Limitations • implicit executor: ExecutionContext • Browse with cursors • each call depends on the previous one
26. GET /1/indexes/index/browse { "cursor": "XXX", "hits":[ ... ] } GET /1/indexes/index/browse?cursor=XXX { "cursor": "YYY", "hits":[ ... ] } GET /1/indexes/index/browse?cursor=YYY etc.
27. GET /1/indexes/index/browse GET /1/indexes/index/browse?cursor=XXX GET /1/indexes/index/browse?cursor=YYY Future[BrowseResult] Future[BrowseResult] Future[BrowseResult] => => =>
28. Future[BrowseResult]] How to chain N ? Iteratee[BrowseResult]] cf. scalaz Await.result(future, duration) X
29. Eat your own dog food
30. Eat your own dog food • use your api, if possible before releasing it • tests are a good way to “test” your API
31. it("should index multiple objects") { batch( index into "test" `object` BasicObject("name1", 1), index into "tutu" objects Seq(BasicObject("name2", 2)), index into "test" objectId "oid1" `object` BasicObject("name3", 3), index into "tutu" objects Map("oid2" -> BasicObject("name4", 4)) ) }
32. Provide good defaults • Timeouts • HTTP features • keep-alive • gzip
33. RTFM
34. A good documentation x 14
36. A good documentation • README, API documentation • Use common vocabulary (object, batch, browse, etc.) • But adapt the naming/idioms/syntax for each language
37. Ethernet IP TCP HTTP DNS Bonus: Network #fail
38. Ethernet IP TCP HTTP DNS
39. Ethernet IP TCP HTTP DNS
40. Ethernet IP TCP HTTP DNS
41. TCP
43. val serverSocket = new ServerSocket(8080) val socket = serverSocket.accept() socket.setSoLinger(true, 0) socket.close() java.net.SocketException: Connection reset
44. & DNS
45. DNS
46. DNS Future<InetAddress> getByName(final String host) { FutureTask<InetAddress> future = new FutureTask(new Callable<InetAddress>() { public InetAddress call() throws UnknownHostException { return InetAddress.getByName(host); } }); executor.execute(future); return future; }
47. networkaddress.cache.ttl Specified in java.security to indicate the caching policy for successful name lookups from the name service.. The value is specified as integer to indicate the number of seconds to cache the successful lookup. A value of -1 indicates "cache forever". The default behaviour is to cache forever when a security manager is installed, and to cache for an implementation specific period of time, when a security manager is not installed. No workaround possible in the code
48. (possible) Improvements
49. batch( index into "indexToBrowse" `object` Test("algolia1", 10, alien = false), index into "indexToBrowse" `object` Test("algolia2", 10, alien = false), index into "indexToBrowse" `object` Test("algolia3", 10, alien = false), index into "indexToBrowse" `object` Test("anything", 10, alien = false) )
50. client.inIndex("indexToBrowse") { implicit index => index `object` Test("algolia1", 10, alien = false) index `object` Test("algolia2", 10, alien = false) index `object` Test("algolia3", 10, alien = false) index `object` Test("anything", 10, alien = false) }
51. to DSL or not to DSL? still a good question
52. Thanks for listening! Questions? rcs@algolia.com @El_Picador