Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Growing Pains - Automating an ETL with Limited ...

Avatar for Texas Natural Resources Information System Texas Natural Resources Information System
October 24, 2018
Technology
0
160

Growing Pains - Automating an ETL with Limited Resources

Avatar for Texas Natural Resources Information System

Texas Natural Resources Information System

October 24, 2018
Tweet

More Decks by Texas Natural Resources Information System

See All by Texas Natural Resources Information System
The 3rd Quarter GIS Community Meeting of 2020
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
160
The 2nd Quarter GIS Community Meeting of 2020
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
570
2020 1st Quarter GIS Community Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
710
PLANET OVERVIEW for TNRIS first Quarter Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
620
Hurricane Harvey H-E-B Disaster Response
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
400
Improving Community Services by Helping Citizens Find Their Lost Pets
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
290
National Address Database (NAD) and NG9-1-1
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
190
StratMap Celebrating Successful Public and Private Partnerships
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
1
290
2019 State of the State Town Hall Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
1
280

Other Decks in Technology

See All in Technology
4月15日の AZ 障害をテクサポの中の人目線で振り返ってみる
Avatar for kazzpapa3 kazzpapa3
3
160
人間性を捧げる生成AI時代の技術選定
Avatar for yo4raw yo4raw
1
860
250510 StepFunctionのテスト自動化始めました vol.1
Avatar for Takumi Abe east_takumi
1
260
VitePress & MCPでアプリ仕様のオープン化に挑戦する
Avatar for Hal hal_spidernight
0
130
正式リリースされた Semantic Kernel の Agent Framework 全部紹介！
Avatar for Kazuki okazuki
1
1.4k
Google Cloud Next 2025 Recap 生成AIモデルとマーケティングでのコンテンツ生成 / Generative AI models and content creation in marketing
Avatar for Kyohei Saito kyou3
0
320
DynamoDB のデータを QuickSight で可視化する際につまづいたこと/stumbling-blocks-when-visualising-dynamodb-with-quicksight
Avatar for emi emiki
0
170
Developer 以外にこそ使って欲しい Amazon Q Developer
Avatar for mita mita
0
170
LangfuseではじめるAIアプリのLLMトレーシング
Avatar for Tadashi Shigeoka codenote
0
200
計装を見直してアプリケーションパフォーマンスを改善させた話
Avatar for Kazuki Obata donkomura
2
170
既存の開発資産を活かしながら、 《新規開発コスト抑制》と《開発体験向上》 を両立する拡張アーキテクチャ事例
Avatar for kubell kubell_hr
0
260
LLM アプリケーションのためのクラウドセキュリティ - CSPM の実装ポイント-
Avatar for KINTOテクノロジーズ Osaka Tech Lab osakatechlab
0
460

Featured

See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
129
19k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
31
1.2k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
32
2.3k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
331
21k
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
205
24k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
280
13k
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
41
2.3k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
349
20k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
233
140k
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
462
33k
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
329
24k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1031
460k

Transcript

  1. Automating an ETL with Limited Resources

  2. None

  3. 1953

  4. 1968

  5. 1999

  6. 2007

  7. 2014

  8. 2018

  9. Scarcity

  10. Soft Skills

  11. Broader Context

  12. None
  13. None

  14. Database ETL Environment

  15. Environment

  16. None

  17. The Environment

  18. The Environment

  19. The Environment

  20. We could buy more. Clouds scale

  21. The Environment

  22. The Environment

  23. The Environment 1

  24. Total Security VS. Risk Mitigation Clouds cost money Use all

    the resources available

  25. Database

  26. The Database

  27. The Database

  28. None
  29. None
  30. None
  31. None

  32. Tables.

  33. The Database 2

  34. None

  35. The Database

  36. None

  37. Development Environments are isolated for a reason Maps are Helpful

    Nightly Data is usually OK

  38. ETL

  39. The ETL

  40. The ETL

  41. 3

  42. We thought that this runtime was great! Well, kind of.

  43. The ETL

  44. 4

  45. 5

  46. The ETL

  47. None
  48. None

  49. But it broke? Well, kind of.

  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None

  57. I learned 6 ways to perform an ETL 10.5 |

    register view Rules can be broken

  58. Resources can constrain our options. Or spur creativity.

  59. Thanks! Contact: Jordan Carmona GIS Specialist City of McKinney linkedin.com/in/jordancarmona/

    [email protected]

  60. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  61. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  62. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  63. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]