memefish the Spanner SQL parser and analyzer

Transcript

1. memefish the Spanner SQL parser and analyzer make.now.just @Go Expert

   Engineer Team

2. memefish Cloud Spanner SQL parser and analyzer 2 https://github.com/MakeNowJust/memefish

3. Analyze? 3 > SELECT * FROM customers “SELECT *” contains

   private information: customer_super_private_column

4. memefish: features • Parse Spanner SQL ◦ AST with source

   code location for diagnostics ◦ Also, generate Spanner SQL from AST • Check expression type and semantics in SQL ◦ SELECT X + 1 FROM (SELECT “foo” AS X) type error: operator + requires two INT64/FLOAT64 ◦ SELECT X, Y FROM (SELECT 1 AS X, 2 AS Y) AS T GROUP BY T.X semantic error: cannot use non-aggregate key: Y ◦ SELECT X FROM (SELECT 1 AS X), (SELECT 2 AS X) semantic error: ambiguous name: X 4 Establish foundation to analyze Spanner SQL

5. memefish: future work memefish is the just foundation, and it

   can use for general purpose. e.g. • migration tool like ridgepole • SQL linter in Go source code • local Spanner emulator 5

6. Other Contributions I published utilities and contributed some OSS in

   this internship. 6

7. enumcase Check switch statement in Go handles all enum values.

   7 https://github.com/MakeNowJust/enumcase switch op { case Add: // ... case Sub: // ... case Mul: // ... case Div: // ... } missing handling enum value: Mod

8. bluesoul Detect duplicate articles (e.g. the the). Note that many

   famous OSS codes have such grammar errors... 8 https://github.com/MakeNowJust/bluesoul

9. #44490: spanner/spansql: add parser for all forms of string and

   bytes literal Contribute to Spanner SQL parser in official GCP Go library. diff size: +332, -32 9

10. Conclusion • Develop the Go library “memefish” to analyze Spanner

    SQL. • Create some Go static analysis tools. • Contribute Go OSS library. 10 Thank you for listening!

11. Appendix: what’s Spanner SQL? SQL accepted by Spanner. • poor

    syntax ◦ no WITH clause (of course no WITH RECURSIVE) ◦ no analytic (window) function • Spanner specific features ◦ TABLESAMPLE clause ◦ hints (@{...}) and query parameters (@foo) But, parsing SQL is hard task... 11