Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Growing Pains - Automating an ETL with Limited ...

Avatar for Texas Natural Resources Information System Texas Natural Resources Information System
October 24, 2018
Technology
200
0

Growing Pains - Automating an ETL with Limited Resources

Avatar for Texas Natural Resources Information System

Texas Natural Resources Information System

October 24, 2018

More Decks by Texas Natural Resources Information System

See All by Texas Natural Resources Information System
The 3rd Quarter GIS Community Meeting of 2020
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
200
The 2nd Quarter GIS Community Meeting of 2020
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
630
2020 1st Quarter GIS Community Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
760
PLANET OVERVIEW for TNRIS first Quarter Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
650
Hurricane Harvey H-E-B Disaster Response
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
520
Improving Community Services by Helping Citizens Find Their Lost Pets
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
350
National Address Database (NAD) and NG9-1-1
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
0
270
StratMap Celebrating Successful Public and Private Partnerships
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
1
360
2019 State of the State Town Hall Meeting
Avatar for Texas Natural Resources Information System texasnaturalresourcesinformationsytem
1
340

Other Decks in Technology

See All in Technology
AI Testing Talks: Challenges of Applying AI in Software Testing: From Hype to Practical Use
Avatar for Exactpro exactpro
PRO
1
140
「速く作る」から「正しく作る」へ ─ 生成AI時代の開発フロー改革の ロードマップと実行 ─
Avatar for starfish719 starfish719
0
9.3k
noUncheckedIndexedAccess、3時間、1万円。 / noUncheckedIndexedAccess, 3 Hours, 10,000 JPY.
Avatar for 株式会社カオナビ kaonavi
1
340
AI Engineering Summit Tokyo 2026 AIの前に、やることがある 〜医療データ企業の4フェーズ〜
Avatar for Daisuke Taniwaki dtaniwaki
0
2.3k
サプライチェーンセキュリティの空白地帯 - 信頼できる”依存性”の未来を考える
Avatar for Hiroki Suezawa (@rung) rung
PRO
2
800
価格.comをAI駆動で全面刷新する ー 30年分の技術的負債を返し、次の30年の土台をつくる ー / AI Engineering Summit Tokyo 2026
Avatar for tkyowa tkyowa
53
58k
マーケットプレイス版Oracle WebCenter Content For OCI
Avatar for oracle4engineer oracle4engineer
PRO
5
1.8k
エンジニアリング戦略の作り方 / Crafting Engineering Strategy
Avatar for iwashi iwashi86
16
5.3k
Oracle Cloud Infrastructure IaaS 新機能アップデート 2026/3 - 2026/5
Avatar for oracle4engineer oracle4engineer
PRO
1
240
非定型業務をAI slackbotで自動化する ~ 社内要望を自動壁打ちするbotを作った ~/automating-ad-hoc-work-with-ai-slackbot
Avatar for shibayu36 shibayu36
0
540
個人最適 から 全体最適 へ　AI情報共有会・AIギルド・AI-DLC で進める カンリーの組織展開
Avatar for ふくだ（fukuda ryu） rfdnxbro
0
2k
「嘘をつくテスト」の失敗例から学ぶ 良いテストコード #frontend_phpcon_do
Avatar for asumikam asumikam
0
590

Featured

See All Featured
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
141
35k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
201
75k
How to Grow Your eCommerce with AI & Automation
Avatar for Katarina Dahlin katarinadahlin
PRO
1
200
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8.2k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
698
190k
What Being in a Rock Band Can Teach Us About Real World SEO
Avatar for Ade Holder 427marketing
0
250
GraphQLの誤解/rethinking-graphql
Avatar for sonatard sonatard
75
12k
Why Mistakes Are the Best Teachers: Turning Failure into a Pathway for Growth
Avatar for Umar Saidu Auna auna
0
150
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
16k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
356
21k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
672
58k
Discover your Explorer Soul
Avatar for Emna emna__ayadi
2
1.1k

Transcript

  1. Automating an ETL with Limited Resources

  2. None

  3. 1953

  4. 1968

  5. 1999

  6. 2007

  7. 2014

  8. 2018

  9. Scarcity

  10. Soft Skills

  11. Broader Context

  12. None
  13. None

  14. Database ETL Environment

  15. Environment

  16. None

  17. The Environment

  18. The Environment

  19. The Environment

  20. We could buy more. Clouds scale

  21. The Environment

  22. The Environment

  23. The Environment 1

  24. Total Security VS. Risk Mitigation Clouds cost money Use all

    the resources available

  25. Database

  26. The Database

  27. The Database

  28. None
  29. None
  30. None
  31. None

  32. Tables.

  33. The Database 2

  34. None

  35. The Database

  36. None

  37. Development Environments are isolated for a reason Maps are Helpful

    Nightly Data is usually OK

  38. ETL

  39. The ETL

  40. The ETL

  41. 3

  42. We thought that this runtime was great! Well, kind of.

  43. The ETL

  44. 4

  45. 5

  46. The ETL

  47. None
  48. None

  49. But it broke? Well, kind of.

  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None

  57. I learned 6 ways to perform an ETL 10.5 |

    register view Rules can be broken

  58. Resources can constrain our options. Or spur creativity.

  59. Thanks! Contact: Jordan Carmona GIS Specialist City of McKinney linkedin.com/in/jordancarmona/

    [email protected]

  60. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  61. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  62. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  63. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]