Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Growing Pains - Automating an ETL with Limited Resources

Texas Natural Resources Information System
October 24, 2018
Technology
0
110

Growing Pains - Automating an ETL with Limited Resources

Texas Natural Resources Information System

October 24, 2018
Tweet

More Decks by Texas Natural Resources Information System

See All by Texas Natural Resources Information System
The 3rd Quarter GIS Community Meeting of 2020
texasnaturalresourcesinformationsytem
0
120
The 2nd Quarter GIS Community Meeting of 2020
texasnaturalresourcesinformationsytem
0
460
2020 1st Quarter GIS Community Meeting
texasnaturalresourcesinformationsytem
0
650
PLANET OVERVIEW for TNRIS first Quarter Meeting
texasnaturalresourcesinformationsytem
0
600
Hurricane Harvey H-E-B Disaster Response
texasnaturalresourcesinformationsytem
0
250
Improving Community Services by Helping Citizens Find Their Lost Pets
texasnaturalresourcesinformationsytem
0
190
National Address Database (NAD) and NG9-1-1
texasnaturalresourcesinformationsytem
0
67
StratMap Celebrating Successful Public and Private Partnerships
texasnaturalresourcesinformationsytem
1
200
2019 State of the State Town Hall Meeting
texasnaturalresourcesinformationsytem
1
190

Other Decks in Technology

See All in Technology
Reducing Cross-Zone Egress at Spotify with Custom gRPC Load Balancing Recap
koh_naga
0
200
Além do else! Categorizando Pokemóns com Pattern Matching no JavaScript
wmsbill
0
610
私が trocco を推す理由
__allllllllez__
1
220
JSON攻略法.pdf
miyakemito
8
5k
Google Cloud の AI を支える裏側のインフラを垣間見る！
maroon1st
0
340
リテール金融（キャッシュレス・ネット銀行・ネット証券）の競争環境と経済圏
8maki
0
1k
プロンプトエンジニアリングでがんばらない-Agentic Workflow へ-近藤憲児
kenjikondobai
2
450
Python と Snowflake はズッ友だょ！~ Snowflake の Python 関連機能をふりかえる ~
__allllllllez__
1
120
Google Cloud Next '24 Recap(Cloud Run/k8s)
mokocm
0
170
EMとして2023年度に頑張ったこと / What we did well in FY2023 as a EM
pauli
1
170
ワールドカフェI ／チューターを改良する / World Café I and Improving the Tutors
ks91
PRO
0
120
UIコンポーネント Vue3マイグレーション体験記
lycorptech_jp
PRO
0
110

Featured

See All Featured
From Idea to $5000 a Month in 5 Months
shpigford
377
45k
GraphQLとの向き合い方2022年版
quramy
32
12k
Web Components: a chance to create the future
zenorocha
305
41k
Design by the Numbers
sachag
274
18k
Infographics Made Easy
chrislema
238
18k
Visualization
eitanlees
136
14k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
227
16k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
6
1.5k
The Brand Is Dead. Long Live the Brand.
mthomps
49
28k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
Docker and Python
trallard
34
2.7k

Transcript

  1. Automating an ETL with Limited Resources

  2. None

  3. 1953

  4. 1968

  5. 1999

  6. 2007

  7. 2014

  8. 2018

  9. Scarcity

  10. Soft Skills

  11. Broader Context

  12. None
  13. None

  14. Database ETL Environment

  15. Environment

  16. None

  17. The Environment

  18. The Environment

  19. The Environment

  20. We could buy more. Clouds scale

  21. The Environment

  22. The Environment

  23. The Environment 1

  24. Total Security VS. Risk Mitigation Clouds cost money Use all

    the resources available

  25. Database

  26. The Database

  27. The Database

  28. None
  29. None
  30. None
  31. None

  32. Tables.

  33. The Database 2

  34. None

  35. The Database

  36. None

  37. Development Environments are isolated for a reason Maps are Helpful

    Nightly Data is usually OK

  38. ETL

  39. The ETL

  40. The ETL

  41. 3

  42. We thought that this runtime was great! Well, kind of.

  43. The ETL

  44. 4

  45. 5

  46. The ETL

  47. None
  48. None

  49. But it broke? Well, kind of.

  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None

  57. I learned 6 ways to perform an ETL 10.5 |

    register view Rules can be broken

  58. Resources can constrain our options. Or spur creativity.

  59. Thanks! Contact: Jordan Carmona GIS Specialist City of McKinney linkedin.com/in/jordancarmona/

    [email protected]

  60. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  61. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  62. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]

  63. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ [email protected]