memefish the Spanner SQL parser and analyzer

Transcript

1. memefish the Spanner SQL parser and analyzer make.now.just @Go Expert Engineer Team
2. memefish Cloud Spanner SQL parser and analyzer 2 https://github.com/MakeNowJust/memefish
3. Analyze? 3 > SELECT * FROM customers “SELECT *” contains private information: customer_super_private_column
4. memefish: features • Parse Spanner SQL ◦ AST with source code location for diagnostics ◦ Also, generate Spanner SQL from AST • Check expression type and semantics in SQL ◦ SELECT X + 1 FROM (SELECT “foo” AS X) type error: operator + requires two INT64/FLOAT64 ◦ SELECT X, Y FROM (SELECT 1 AS X, 2 AS Y) AS T GROUP BY T.X semantic error: cannot use non-aggregate key: Y ◦ SELECT X FROM (SELECT 1 AS X), (SELECT 2 AS X) semantic error: ambiguous name: X 4 Establish foundation to analyze Spanner SQL
5. memefish: future work memefish is the just foundation, and it can use for general purpose. e.g. • migration tool like ridgepole • SQL linter in Go source code • local Spanner emulator 5
6. Other Contributions I published utilities and contributed some OSS in this internship. 6
7. enumcase Check switch statement in Go handles all enum values. 7 https://github.com/MakeNowJust/enumcase switch op { case Add: // ... case Sub: // ... case Mul: // ... case Div: // ... } missing handling enum value: Mod
8. bluesoul Detect duplicate articles (e.g. the the). Note that many famous OSS codes have such grammar errors... 8 https://github.com/MakeNowJust/bluesoul
9. #44490: spanner/spansql: add parser for all forms of string and bytes literal Contribute to Spanner SQL parser in official GCP Go library. diff size: +332, -32 9
10. Conclusion • Develop the Go library “memefish” to analyze Spanner SQL. • Create some Go static analysis tools. • Contribute Go OSS library. 10 Thank you for listening!
11. Appendix: what’s Spanner SQL? SQL accepted by Spanner. • poor syntax ◦ no WITH clause (of course no WITH RECURSIVE) ◦ no analytic (window) function • Spanner specific features ◦ TABLESAMPLE clause ◦ hints (@{...}) and query parameters (@foo) But, parsing SQL is hard task... 11