僕とRedashとAthena

Transcript

1. ๻ͱRedashͱAthena Redash Meetup #1 @dyuju_you
2. @dyuju_you • Yuki Maeno • Ϗοάσʔλ෦ ॴଐ • Data Engineer (not Scientist) • Hadoop(৅͞Μ)͕େ޷͖
3. Architecture
4. 5SBDLJOH XFCϖʔδ
   OHJOY
   
   HPMBOH qVFOUE 4
5. 4 3FEBTI 1SFTUP "UIFOB
6. ৅͞Μͷେ͖͞
7. 100TB
8. < RedashͰΫΤϦॻ͚ΔͷͰ ࣗ༝ʹͲ͏ͧʙ
9. select * from activity
10. ʊਓਓਓਓਓਓʊ ʼɹಥવͷࢮɹʻ ʉY^Y^Y^Y^Yʉ
11. ಥવͷࢮ • Redashͷࢮ • ϒϥ΢βͷࢮ • ͓ۚͷࢮ
12. Ͳ͏͢Δ͔ • ӡ༻ͰΧόʔ • ϢʔβͷSQLྗΛ্͛Δ • γεςϜతʹͳΜͱ͔͢Δ
13. Redash͸OSS
14. ࣗ෼ͰΧελϚΠζͰ͖Δʂ
15. ΍ͬͨ͜ͱ • Limit੍໿ • Partition੍໿
17. Limit੍໿ • sqlparseΛར༻ • Limit۟ͷΈ͚ͩΛݟΔ • ࢦఆͷ਺஋ΑΓଟ͍৔߹͸raise Exception
18. Partition੍໿ • sqlparseΛར༻ • FROMͱJOIN۟ͷςʔϒϧΛநग़ • ͦͷςʔϒϧʹPartition͕͋Δ͔֬ೝ • Partition͕Where۟Ͱࢦఆ͞Ε͍ͯΔ͔֬ೝ
19. Partition੍໿ • sqlparseΛར༻ • FROMͱJOIN۟ͷςʔϒϧΛநग़ • ͦͷςʔϒϧʹPartition͕͋Δ͔֬ೝ ← ͜͜ • Partition͕Where۟Ͱࢦఆ͞Ε͍ͯΔ͔֬ೝ
20. SQL͚ͩ͡Ό Partition͕͋Δ͔Θ͔Βͳ͍
21. redash#2045
22. GlueͷDataCatalog͔Β Partition৘ใ͕औΕΔʂ
25. Enjoy SQL Lifeʙ