A variant of recursive descent parsing

Showing how C++17 std::variant<>, std::visit<> and overloads of lambdas can help make a simple parser very easy to read and understand.

Presentation given at the distributed SwedenCPP Stockholm + London C++ UG meetup.

Transcript

1. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller A variant of recursive descent parsing Björn Fahller E --> E "+" E | E "-" E | "-" E | E "*" E | E "/" E | "(" E ")" | v
2. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Parser generators suck %token NUM %left '-' '+' %left '*' '/' %precedence NEG /* negation--unary minus */ %right '^' /* exponentiation */ %% /* The grammar follows. */ exp: NUM { $$ = $1; } | exp '+' exp { $$ = $1 + $3; } | exp '-' exp { $$ = $1 - $3; } | exp '*' exp { $$ = $1 * $3; }
3. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Lexers suck using token = enum { number, identifier, lparen, …}; union yystype { double num; const char* id; }; yystype* yylval; token yylex(); … if (yylex() == number) { … blah(yylval->num); … }
4. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::variant<T...> using v = std::variant<int, std::string, bool, std::list<int>>;
5. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::variant<T...> A very brief intro. using v = std::variant<int, std::string, bool, std::list<int>>; v var{std::string(”foo”)}; Construct holding a std::string
6. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::variant<T...> A very brief intro. using v = std::variant<int, std::string, bool, std::list<int>>; v var{std::string(”foo”)}; std::get<std::string>(v) = ”bar”; Access the std::string and assign it a new value
7. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::visit(func_obj, v...) visitor visitor_obj; int i = std::visit(visitor_obj, variant_var);
8. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::visit(func_obj, v...) Matt Kline ”std::visit is everything wrong with modern C++” https://bitbashing.io/std-visit.html visitor visitor_obj; int i = std::visit(visitor_obj, variant_var);
9. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::visit(func_obj, v...) class visitor { public: int operator()(std::string) { /* str things */ } int operator()(int) { /* int things */ } int operator()(auto) { /* other things */ } }; visitor visitor_obj; int i = std::visit(visitor_obj, variant_var);
10. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller class visitor { public: int operator()(std::string) { /* str things */ } int operator()(int) { /* int things */ } int operator()(auto) { /* other things */ } }; visitor visitor_obj; int i = std::visit(visitor_obj, variant_var); std::visit(func_obj, v...) • Uncool! Implementation far away from use.
11. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller class visitor { public: int operator()(std::string) { /* str things */ } int operator()(int) { /* int things */ } int operator()(auto) { /* other things */ } }; visitor visitor_obj; int i = std::visit(visitor_obj, variant_var); std::visit(func_obj, v...) • Uncool! Implementation far away from use. • A lambda is an object of class type with operator()
12. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller class visitor { public: int operator()(std::string) { /* str things */ } int operator()(int) { /* int things */ } int operator()(auto) { /* other things */ } }; visitor visitor_obj; int i = std::visit(visitor_obj, variant_var); std::visit(func_obj, v...) • Uncool! Implementation far away from use. • A lambda is an object of class type with operator() • A lambda can be a base class
13. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller class visitor { public: int operator()(std::string) { /* str things */ } int operator()(int) { /* int things */ } int operator()(auto) { /* other things */ } }; visitor visitor_obj; int i = std::visit(visitor_obj, variant_var); std::visit(func_obj, v...) • Uncool! Implementation far away from use. • A lambda is an object of class type with operator() • A lambda can be a base class
14. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller overload<T...> template <typename ... T> class overload : T... { public: overload(T... t) : T(t)... {} using T::operator()...; }; auto x = overload( [](int) {}, [](std::string){} );
15. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller overload<T...> template <typename ... T> class overload : T... { public: overload(T... t) : T(t)... {} using T::operator()...; }; auto x = overload( [](int) {}, [](std::string){} ); C++17 variadic using, to bring operator() from each T into view
16. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller overload<T...> template <typename ... T> class overload : T... { public: overload(T... t) : T(t)... {} using T::operator()...; }; auto x = overload( [](int) {}, [](std::string){} ); C++17 automatic template type deduction from constructor.
17. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::variant<T...> using v = std::variant<int, std::string, bool, std::list<int>>; void func(v obj) { std::visit(overload( [](int) { /* int stuff */ }, [](std::string) { /* string stuff */ }, [](const auto&) { /* other stuff */ }), obj); }
18. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller std::variant<T...> using v = std::variant<int, std::string, bool, std::list<int>>; void func(v obj) { std::visit(overload( [](int) { /* int stuff */ }, [](std::string) { /* string stuff */ }, [](const auto&) { /* other stuff */ }), obj); } Now the logic is where we use it. Cool!
19. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {};
20. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; For single character tokens like + - * / (
21. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; Let numeric literals hold the value
22. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; And identifiers the name.
23. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; Keywords for storing and dropping variables
24. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; and for lexer errors an exception is thrown
25. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; using token = std::variant< number,ident, C<'-'>, C<'+'>, C<'/'>, C<'*'>, C<'('>, C<')'>, C<'='>, eof, remember, forget >;
26. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Representing tokens Tokens for a very simple calculator. template <char> struct C {}; struct number { double value; }; struct ident { std::string_view value; }; struct eof {}; struct remember {}; struct forget {}; struct other {}; using token = std::variant< number,ident, C<'-'>, C<'+'>, C<'/'>, C<'*'>, C<'('>, C<')'>, C<'='>, eof, remember, forget >; class lex { public: lex(std::string s); token next_token(); // consume token peek(); // look ahead void drop(); // consume private: ... };
27. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; }
28. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; }
29. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); }
30. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); }
31. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; }
32. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; }
33. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; }
34. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller double calculator::parse(std::string s) { lexer = std::make_unique<lex>(move(s)); auto t = lexer->peek(); auto rv = std::visit( overload{ [=](remember) { lexer->drop();return parse_remember_var();}, [=](forget) { lexer->drop();return parse_forget_var();}, [=](auto) { return parse_expr();} }, t); if (!std::holds_alternative<eof>(lexer next_token())) → throw "garbage after expr"; return rv; } double calculator::parse_expr() { auto v = parse_term(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'+'>) { lexer->drop(); return v+parse_term();}, [=](C<'-'>) { lexer->drop(); return v-parse_term();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_term() { auto v = parse_factor(); for (bool done = false; !done;) { auto t = lexer->peek(); v = std::visit( overload{ [=](C<'/'>) { lexer->drop(); return v/parse_factor();}, [=](C<'*'>) { lexer->drop(); return v*parse_factor();}, [&](auto) { done = true; return v; } }, t); } return v; } double calculator::parse_factor() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident var) { return lookup(var.value); }, [=](number n) { return n.value; }, [=](C<'+'>) { return parse_term(); }, [=](C<'-'>) { return -parse_term(); }, [=](C<'('>) { return parse_paren(); }, [=](auto) -> double { throw "unexpected"; } }, t); } double calculator::parse_paren() { auto v = parse_expr(); auto t = lexer->next_token(); if (!std::holds_alternative<C<')'>>(t)) throw "expected ')'"; return v; } double calculator::parse_forget_var() { auto t = lexer->next_token(); return std::visit( overload{ [=](ident i) { return drop_variable(i.value);}, [=](auto) -> double { throw "expected variable name";} }, t); } double calculator::parse_remember_var() { auto var = lexer->next_token(); if (!std::holds_alternative<ident>(var)) throw "expected variable name"; if (!std::holds_alternative<C<'='>>(lexer->next_token())) throw "expected ="; auto v = parse_expr(); memory[std::get<ident>(var).value] = v; return v; } std::variant<T...> std::visit(visitor, v…) and overload of lambdas makes for simple type safe lexers and easy to read parser code
35. Variant of Parsing, C++ Stockholm 0x07, Björn Fahller @bjorn_fahller Björn Fahller https://github.com/rollbear/variant_parse bjorn@fahller.se @bjorn_fahller @rollbear cpplang, swedencpp A variant of recursive descent parsing