Mixing in Elixir to Build Search

Code BEAM SF. March 16, 2018
This talk will point out some of the discoveries they've made while porting the search page from their old stack (PHP). He will also exemplify Elixir, as their go-to language for building out tools on the Search team and how these tools help them move faster with choosing the right decisions when it comes to search algorithm tweaks.

Transcript

1. Mixing in Elixir to Build Search Friday, 16 March Code BEAM SF
2. Adrian Software Engineer Search & Personalization Building all things search at Teachers Pay Teachers www.teacherspayteachers.com • github/twitter: @drincruz • drincruz.com
3. We're the World's Most Popular Online Marketplace for Original Educational Resources
5. Mixing in Elixir What does that mean?
6. Mixing in Elixir What does that mean? Breaking the monolith...
7. Mixing in Elixir What does that mean? Breaking the monolith...by porting pages piecemeal
8. Let’s talk about search
9. Let’s talk about search
10. Let’s start searching...
13. Search suggestions
14. Seller suggestions
15. Resource suggestions
16. JSON returned from autosuggest
17. Process to power autosuggest
18. Process to power autosuggest Built on Phoenix
19. autosuggest A standalone service built as a proof-of-concept for Elixir adoption at TPT
20. autosuggest A standalone service built as a proof-of-concept for Elixir adoption at TPT Thanks to Samira for owning this project
21. Now the information retrieval The search part
22. previous way to serve search (simplified)
23. previous way to serve search (simplified) search
24. previous way to serve search (simplified) search product
25. previous way to serve search (simplified) search product home
26. previous way to serve search (simplified) search product home Et cetera. ...everything else :)
27. Implementing more elixir A GraphQL API
28. Mixing in GraphQL
29. Mixing in GraphQL And then this gave us more tools to debug and analyze search
30. Mixing in GraphQL And then this gave us more tools to debug and analyze search graphiql
31. Mixing in GraphQL And then this gave us more tools to debug and analyze search graphiql Search replayer Built with Elixir running Cowboy
32. Mixing in GraphQL And then this gave us more tools to debug and analyze search graphiql Search replayer Built with Elixir running Cowboy Recommendations Side-by-side
33. Mixing in GraphQL And then this gave us more tools to debug and analyze search graphiql Search replayer Built with Elixir running Cowboy Recommendations Side-by-side And lots of personalization tooling
34. graphiql
35. graphiql graphql query
36. graphiql results
37. graphiql Just with graphiql, search became easier to debug
38. graphiql Just with graphiql, search became easier to debug And also easier to test
39. test "facet on grade group should return grades in the resultFacets" do query = """ query { searchProducts(query: "", limit: 5, facets: ["Grade-Level/1-2"]) { resultFacets { grades { id } } } } """
40. assert {:ok, result} = Poison.Parser.parse(response.body) assert Map.has_key?(result, "data") assert not Map.has_key?(result, "errors") search_results = result["data"]["searchProducts"] obs_grades = search_results["resultFacets"]["grades"] assert length(obs_grades) == 2 assert Enum.member?(obs_grades, %{"id" => "Grades_First"}) assert Enum.member?(obs_grades, %{"id" => "Grades_Second"}) end
41. Testing our Search terms also became easier!
42. Testing our Search terms also became easier! We convert our search string into an AST (abstract syntax tree)
43. TptApi.SearchText.from_string("this and that") {:ok, {:ast_and, [ast_word: 'this', ast_word: 'and', ast_word: 'that']}} Typical unquoted text string
44. TptApi.SearchText.from_string(~s("this and that")) {:ok, {:ast_quoted, 'this and that'}} Quoted string
45. TptApi.SearchText.from_string("this -that") {:ok, {:ast_and, [ast_word: 'this', ast_not: {:ast_word, 'that'}]}} Advanced negative searching
46. test "successfully parse nice inputs", %{nice_inputs: inputs} do for {input, expected} <- inputs do assert {:ok, expected} == TptApi.SearchText.from_string(input) end end We test a list of different search terms we support
47. [... {"fractions \"cheese bananas\"", {:ast_and, [ast_word: 'fractions', ast_quoted: 'cheese bananas']}}, {"fractions -\"cheese bananas\"", {:ast_and, [ast_word: 'fractions', ast_not: {:ast_quoted, 'cheese bananas'}]}}, {"fractions \"cheese or bananas\"", {:ast_and, [{:ast_word, 'fractions'}, {:ast_quoted, 'cheese or bananas'}]}}, ...] That list looks similar to the following
48. Search analytics Search is not easy
49. Search analytics Search is not easy But that’s why we A/B test and look at a lot of data
50. Internal eval is important
51. Internal eval is important replays Search or Browse graphql queries and gets aggregate data
52. Internal eval is important Side by Side search comparisons for A/B tests
53. facets |> Enum.map(&(Task.async(Replayer.Browse, :do_browse, [&1, cfg.a_url_params]))) |> Enum.map(&Task.await/1) |> update_with_rank() Example on Replayer of Top 100 Browses
54. # Snippet of GraphQL query in Replayer.Browse.do_browse/2 totalCount, results{ type, url, name, description(maxLength: 300), images{large}, downloads, views, votes, abbreviatedGrades, ratings{scoreAverage, count}, sold, price, isFree, author{id, name, icon{location}} # … more code
55. *WAD: weighted average discount Full disclosure: the data here does not represent what is in production :)
56. Mixing Elixir for tooling Replaced PHP tooling
57. Mixing Elixir for tooling Replaced PHP tooling Also a great learning project for onboarding
58. Mixing Elixir for tooling Replaced PHP tooling Also a great learning project for onboarding Thanks to Louis for bootstrapping this project
59. Related searches An SEO initiative
60. Related searches An SEO initiative Built easily in the new elixir platform
61. (refresher) Autosuggest endpoint
62. (refresher) Autosuggest endpoint GraphQL API
63. defp run_query(path) do uri = Application.get_env(:tpt_api, :autosuggest_api_root) <> "/#{path}" |> URI.parse() HTTPUtil.get_json!(uri, [], [timeout: @timeout, recv_timeout: @timeout]) end
64. Now on our Search pages in React
65. Elixir Mixed In
66. Elixir Mixed In • Autosuggest helped push adoption forward
67. Elixir Mixed In • Autosuggest helped push adoption forward • GraphQL API provided great decoupling benefits
68. Elixir Mixed In • Autosuggest helped push adoption forward • GraphQL API provided great decoupling benefits • Having engineers bootstrap Elixir projects provide good learning
69. Elixir Mixed In • Autosuggest helped push adoption forward • GraphQL API provided great decoupling benefits • Having engineers bootstrap Elixir projects provide good learning • engineering.teacherspayteachers.com
70. Thank You Code BEAM SF! @drincruz https://www.teacherspayteachers.com/Careers