Growing Pains - Automating an ETL with Limited Resources

0fb5a75afd70eab94e2fb2697721e536?s=47 Texas Natural Resources Information System
October 24, 2018
Technology
0
36

Growing Pains - Automating an ETL with Limited Resources

0fb5a75afd70eab94e2fb2697721e536?s=128

Texas Natural Resources Information System

October 24, 2018
Tweet

More Decks by Texas Natural Resources Information System

See All by Texas Natural Resources Information System
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
77
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
260
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
470
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
440
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
27
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
22
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
0
4
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
1
30
0fb5a75afd70eab94e2fb2697721e536?s=48 texasnaturalresourcesinformationsytem
1
25

Other Decks in Technology

See All in Technology
953fe5f1a04d8944ead78db3991cfa6a?s=48 anoriqq
0
230
Bf7fe621f4fe1615c228ef8a79b87282?s=48 shirayanagiryuji
2
690
D98ee74ffe0fafbdc83b23907dda3665?s=48 doublemarket
2
220
91e7aee9769836f448a9f1ce91255749?s=48 mathatelle
0
1.5k
1817c3d5995710d04506a17cc9b5c9ba?s=48 runrunsan
1
210
F0ec1a6cb9e7be1911ed97b6561ed5ac?s=48 k1nakayama
0
140
D5a8875907107294786ee8cba72c7f4c?s=48 torufuru
2
580
098ad475722e3697ec2fba28c8654f9f?s=48 yuhta28
0
390
0d29d155ce5c5a93ed46363626da3ea7?s=48 ocise
1
2k
9b600b6d2f592d0944507e989770b9b3?s=48 issei126
1
750
8a84268593355816432ceaf78777d585?s=48 dena_tech
0
290
2ec42cd8400b1e8e5ba125936c1d27e9?s=48 smiyaza
0
200

Featured

See All Featured
0fe2973fa8d27d96547e43322341183a?s=48 swwweet
205
6.7k
C157b16234f1e75e8eac3698c1d4414a?s=48 ddemaree
273
31k
C44e1f7e22c3f23cff7bc130871047ef?s=48 eileencodes
112
24k
6bb82b13cda86d3b821fa44100335ddf?s=48 thoeni
3
370
E6c6e133e74c3b83f04d2861deaa1c20?s=48 searls
202
35k
B6a8f005f39d23ffc930508ac9da68b9?s=48 vanstee
115
4.7k
D09fad3e36ae6841f54820be0e907f7a?s=48 rocio
155
11k
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
218
14k
168ccec72eee0530b818d44f3fedaacf?s=48 shlominoach
175
7.3k
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
19
1k
6804f1775cb4babfcc3851298566fbce?s=48 tanoku
257
24k
11c84dff30266b907714802088852677?s=48 jasonvnalue
80
7.9k

Transcript

  1. Automating an ETL with Limited Resources

  2. None

  3. 1953

  4. 1968

  5. 1999

  6. 2007

  7. 2014

  8. 2018

  9. Scarcity

  10. Soft Skills

  11. Broader Context

  12. None
  13. None

  14. Database ETL Environment

  15. Environment

  16. None

  17. The Environment

  18. The Environment

  19. The Environment

  20. We could buy more. Clouds scale

  21. The Environment

  22. The Environment

  23. The Environment 1

  24. Total Security VS. Risk Mitigation Clouds cost money Use all

    the resources available

  25. Database

  26. The Database

  27. The Database

  28. None
  29. None
  30. None
  31. None

  32. Tables.

  33. The Database 2

  34. None

  35. The Database

  36. None

  37. Development Environments are isolated for a reason Maps are Helpful

    Nightly Data is usually OK

  38. ETL

  39. The ETL

  40. The ETL

  41. 3

  42. We thought that this runtime was great! Well, kind of.

  43. The ETL

  44. 4

  45. 5

  46. The ETL

  47. None
  48. None

  49. But it broke? Well, kind of.

  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None

  57. I learned 6 ways to perform an ETL 10.5 |

    register view Rules can be broken

  58. Resources can constrain our options. Or spur creativity.

  59. Thanks! Contact: Jordan Carmona GIS Specialist City of McKinney linkedin.com/in/jordancarmona/

    jcarmona@mckinneytexas.org

  60. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org

  61. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org

  62. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org

  63. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org