Comredis

How to use elixir metaprogramming to generate syntatically correct Redis commands.

Transcript

1. Comredis Iuri Fernandes Redis, Elixir, Ruby, Metaprogramming, Testing and other Buzzwords
2. Ruby require "redis" redis = Redis.new redis.set("hello ", "world") # => "OK" redis.get("hello") # => "world"
3. Connection management Authentication Communication (Request-response) Client RESP Server
4. “Requests are sent from the client to the Redis server as arrays of strings representing the arguments of the command to execute.” –Redis documentation RESP (REdis Serialization Protocol)
5. Ruby # Get the value of a key. # # @param [String] key # @return [String] def get(key) synchronize do |client| client.call([:get, key]) end end https:/ /github.com/redis/redis-rb/blob/master/lib/ redis.rb
7. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, ~w(SET hello world)) # => {:ok, "OK"} Redix.command(conn, ~w(GET hello)) # => {:ok, "world"}
8. Elixir Redix.pipeline(conn, [ ~w(INCR foo), ~w(INCR foo), ~w(INCR foo 2) ]) #=> {:ok, [1, 2, 4]}
9. Elixir Redix.pipeline(conn, [ ~w(INCR foo), ~w(INCR foo), ~w(INCRBY foo 2) ]) #=> {:ok, [1, 2, 4]}
11. Ruby redis = Redis.new redis.set("hello ", "world") # => "OK" redis.get("hello") # => "world"
12. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, ~w(SET hello world)) # => {:ok, "OK"} Redix.command(conn, ~w(GET hello)) # => {:ok, "world"}
13. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, set("hello", "world"))) # => {:ok, "OK"} Redix.command(conn, get("hello")) # => {:ok, "world"}
14. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} conn |> Redix.command(set("hello", "world"))) # => {:ok, "OK"} conn |> Redix.command(get("hello")) # => {:ok, "world"}
15. Elixir defmodule Commands do def get(key) do ~w(GET #{key}) end def set(key, value) do ~w(SET #{key} #{value}) end # ... end
16. Bad idea!
17. lib/redis.rb => 2730 LOC* * Includes a lot of documentation * Handles responses * It does not support all the commands Not for the redis-rb
18. Imagine if I could generate all these functions automatically! You can! Metaprogramming FTW! Better, at compile time!
19. { ... "GET": { "summary": "Get the value of a key", "complexity": "O(1)", "arguments": [ { "name": "key", "type": "key" } ], "since": "1.0.0", "group": "string" }, ... } https:/ /github.com/antirez/redis-doc/blob/master/commands.json
20. Parser (Poison) Commands JSON specification Command structured data Elixir macros Functions
21. Macro rules Rule 1: Don’t Write Macros Rule 2: Use Macros Gratuitously
22. Elixir defmodule Comredis do use Comredis.Command.Generator ... end
24. Abstract Syntax Tree Expression quote unquote 1 + 1
25. quote unquote 1 + 1 {:+, [context: Elixir, import: Kernel], [1, 1]}
26. Elixir defmodule Comredis.Command.Generator do defmacro __using__(_options) do for command <- FileReader.load do generate(command) end end defp generate(command = %Command{}) do quote do @doc unquote doc(command) unquote bodies(command, Argument.split_options(command.arguments)) end end ... end
27. Elixir defp bodies(command, {required_args, []}) do args = argument_names(required_args) quote do def unquote(command.canonical_name)(unquote_splicing(args)) do List.flatten [unquote(command.name), unquote_splicing(args)] end end end
28. Elixir defp bodies(%Command{canonical_name: :get, name: "GET"}, {[%Argument{canonical_name: :key}], []}) do args = [{:key, [], Elixir}] quote do def unquote(command.canonical_name)(unquote_splicing(args)) do List.flatten [unquote(command.name), unquote_splicing(args)] end end end defp bodies(%Command{canonical_name: :get, name: "GET"}, {%Argument{canonical_name: :key}, []}) do quote do def get(key) do List.flatten ["GET", key] end end end
30. How to test it? Don’t test code generation, but the generated code Write tests for each function? No, automatically test all of them
31. Property-based testing Generator: random arguments compliant to command arguments Property: commands have a valid syntax Use a Redis server as a test oracle github.com/parroty/excheck (triq)
33. What is missing? Type checking to use Dialyzer More documentation Something else?
34. Who uses similar ideas? Elixir - unicode representation ex2ms - ETS match expressions
35. Questions? @fqiuri github.com/iurifq/comredis