Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ETS versus Elasticsearch for Queryable Caching

ETS versus Elasticsearch for Queryable Caching

If your data sets aren't sufficiently large enough while building your OTP applications, consider reaching for built-in functionality of ETS.

David Schainker

August 30, 2019
Tweet

Other Decks in Programming

Transcript

  1. ETS FEATURES Built into OTP (:ets) K/V table containing tuples

    No garbage collection Concurrently accessible data (“serialized”)
  2. USING ETS REQUIREMENTS Where does our data come from? How

    much data do we have? What’s our R/W frequency? How often does this data change?
  3. USING ETS REQUIREMENTS Transactional DB How much data do we

    have? What’s our R/W frequency? How often does this data change?
  4. USING ETS REQUIREMENTS Transactional DB Fits in RAM What’s our

    R/W frequency? How often does this data change?
  5. %Fonts.Font{ name: “Futura”, classification: “sans-serif”, description: “Futura is a geometric

    sans-serif typeface designed by Paul Renner and released in 1927.” }
  6. INDEX AND LOOKUP > table = :ets.new(:fonts_lookup, [:set, :public]) >

    :ets.insert(table, {f.name, f.classification, f.description}) > :ets_insert(table, {"acumin", “sans-serif", "Balanced and rational quality."}) > :ets.lookup(table, "futura") [{"futura", "sans-serif"}]
  7. fun2ms/1 > fun = :ets.fun2ms( fn {n, c, d} when

    c == "sans-serif" -> n end) [{ {:”$1", :"$2"}, [{:==, :"$2", "sans-serif"}], [:”$1”] }] > :ets.select(table, fun) ["futura"] ETS MATCH SPECIFICATION
  8. fun2ms/1 > fun = :ets.fun2ms( fn {n, c, d} when

    c == "sans-serif" -> n end) [{ {:”$1", :"$2"}, [{:==, :"$2", "sans-serif"}], [:”$1”] }] > :ets.select(table, fun) ["futura"] ETS MATCH SPECIFICATION
  9. fun2ms/1 > fun = :ets.fun2ms( fn {n, c, d} when

    c == "sans-serif" -> n end) [{ {:”$1", :"$2"}, [{:==, :"$2", "sans-serif"}], [:”$1”] }] > :ets.select(table, fun) ["futura"] ETS MATCH SPECIFICATION
  10. fun2ms/1 > fun = :ets.fun2ms( fn {n, c, d} when

    c == "sans-serif" -> n end) [{ {:”$1", :"$2"}, [{:==, :"$2", "sans-serif"}], [:”$1”] }] > :ets.select(table, fun) ["futura"] ETS MATCH SPECIFICATION
  11. FILTERING ETSO: AN ETS ADAPTER FOR ECTO Use Ecto schemas

    with ETS tables Handles ETS table supervision Understands match specifications
  12. fun2ms/1 > fun = :ets.fun2ms( fn {n, c, d} when

    c == "sans-serif" -> n end) [{ {:”$1", :"$2"}, [{:==, :"$2", "sans-serif"}], [:”$1”] }] > :ets.select(table, fun) ["futura"] ETS MATCH SPECIFICATION
  13. AUTOCOMPLETE iex(19)> :ets.fun2ms(fn({key, "serif"} = el) -> key end) [{{:"$1",

    "serif"}, [], [:"$1"]}] iex(20)> :ets.select(table, fun) ["futura", "adobe garamond"]
  14. persistent_term Much Faster than ETS for fetching whole terms Garbage

    collect on update / delete No filtering Good for NIF references
  15. REFERENCES 1. Armstrong, Joe. Elib1 - https://github.com/joearms/elib1 2. Bekker, Ward.

    A Basic Full Text Search Server in Erlang - http://blog.equanimity.nl/blog/2011/10/07/a-basic- full-text-search-server-in-erlang/ 3. Herbert, Fred. Learn You Some Erlang For Great Good. ETS - https://learnyousomeerlang.com/ets 4. Nelson, Jay. Avoiding Single Process Bottlenecks By Using Ets Concurrency. Erlang Factory, 6 Mar. 2014. http:// www.erlang-factory.com/static/upload/media/1394716488140115jaynelson.pdf, https://www.youtube.com/ watch?v=XrkY9WRY8p0 5. Pena, Maria. Beam Extreme: Don't Do This At Home - https://www.youtube.com/watch?v=_ePnlegZqgI 6. Implementing Autocomplete in Elixir Using ETS? https://elixirforum.com/t/implementing-autocomplete-in-elixir- using-ets/8651/2