Upgrade to Pro — share decks privately, control downloads, hide ads and more …

memefish the Spanner SQL parser and analyzer

memefish the Spanner SQL parser and analyzer


September 13, 2019

More Decks by TSUYUSATO Kitsune

Other Decks in Programming


  1. memefish the Spanner SQL parser and analyzer make.now.just @Go Expert

    Engineer Team
  2. memefish Cloud Spanner SQL parser and analyzer 2 https://github.com/MakeNowJust/memefish

  3. Analyze? 3 > SELECT * FROM customers “SELECT *” contains

    private information: customer_super_private_column
  4. memefish: features • Parse Spanner SQL ◦ AST with source

    code location for diagnostics ◦ Also, generate Spanner SQL from AST • Check expression type and semantics in SQL ◦ SELECT X + 1 FROM (SELECT “foo” AS X) type error: operator + requires two INT64/FLOAT64 ◦ SELECT X, Y FROM (SELECT 1 AS X, 2 AS Y) AS T GROUP BY T.X semantic error: cannot use non-aggregate key: Y ◦ SELECT X FROM (SELECT 1 AS X), (SELECT 2 AS X) semantic error: ambiguous name: X 4 Establish foundation to analyze Spanner SQL
  5. memefish: future work memefish is the just foundation, and it

    can use for general purpose. e.g. • migration tool like ridgepole • SQL linter in Go source code • local Spanner emulator 5
  6. Other Contributions I published utilities and contributed some OSS in

    this internship. 6
  7. enumcase Check switch statement in Go handles all enum values.

    7 https://github.com/MakeNowJust/enumcase switch op { case Add: // ... case Sub: // ... case Mul: // ... case Div: // ... } missing handling enum value: Mod
  8. bluesoul Detect duplicate articles (e.g. the the). Note that many

    famous OSS codes have such grammar errors... 8 https://github.com/MakeNowJust/bluesoul
  9. #44490: spanner/spansql: add parser for all forms of string and

    bytes literal Contribute to Spanner SQL parser in official GCP Go library. diff size: +332, -32 9
  10. Conclusion • Develop the Go library “memefish” to analyze Spanner

    SQL. • Create some Go static analysis tools. • Contribute Go OSS library. 10 Thank you for listening!
  11. Appendix: what’s Spanner SQL? SQL accepted by Spanner. • poor

    syntax ◦ no WITH clause (of course no WITH RECURSIVE) ◦ no analytic (window) function • Spanner specific features ◦ TABLESAMPLE clause ◦ hints (@{...}) and query parameters (@foo) But, parsing SQL is hard task... 11