Upgrade to Pro — share decks privately, control downloads, hide ads and more …

memefish the Spanner SQL parser and analyzer

memefish the Spanner SQL parser and analyzer



September 13, 2019


  1. memefish the Spanner SQL parser and analyzer make.now.just @Go Expert

    Engineer Team
  2. memefish Cloud Spanner SQL parser and analyzer 2 https://github.com/MakeNowJust/memefish

  3. Analyze? 3 > SELECT * FROM customers “SELECT *” contains

    private information: customer_super_private_column
  4. memefish: features • Parse Spanner SQL ◦ AST with source

    code location for diagnostics ◦ Also, generate Spanner SQL from AST • Check expression type and semantics in SQL ◦ SELECT X + 1 FROM (SELECT “foo” AS X) type error: operator + requires two INT64/FLOAT64 ◦ SELECT X, Y FROM (SELECT 1 AS X, 2 AS Y) AS T GROUP BY T.X semantic error: cannot use non-aggregate key: Y ◦ SELECT X FROM (SELECT 1 AS X), (SELECT 2 AS X) semantic error: ambiguous name: X 4 Establish foundation to analyze Spanner SQL
  5. memefish: future work memefish is the just foundation, and it

    can use for general purpose. e.g. • migration tool like ridgepole • SQL linter in Go source code • local Spanner emulator 5
  6. Other Contributions I published utilities and contributed some OSS in

    this internship. 6
  7. enumcase Check switch statement in Go handles all enum values.

    7 https://github.com/MakeNowJust/enumcase switch op { case Add: // ... case Sub: // ... case Mul: // ... case Div: // ... } missing handling enum value: Mod
  8. bluesoul Detect duplicate articles (e.g. the the). Note that many

    famous OSS codes have such grammar errors... 8 https://github.com/MakeNowJust/bluesoul
  9. #44490: spanner/spansql: add parser for all forms of string and

    bytes literal Contribute to Spanner SQL parser in official GCP Go library. diff size: +332, -32 9
  10. Conclusion • Develop the Go library “memefish” to analyze Spanner

    SQL. • Create some Go static analysis tools. • Contribute Go OSS library. 10 Thank you for listening!
  11. Appendix: what’s Spanner SQL? SQL accepted by Spanner. • poor

    syntax ◦ no WITH clause (of course no WITH RECURSIVE) ◦ no analytic (window) function • Spanner specific features ◦ TABLESAMPLE clause ◦ hints (@{...}) and query parameters (@foo) But, parsing SQL is hard task... 11