Growing Pains - Automating an ETL with Limited Resources

Growing Pains - Automating an ETL with Limited Resources

Transcript

  1. Automating an ETL with Limited Resources

  2. None
  3. 1953

  4. 1968

  5. 1999

  6. 2007

  7. 2014

  8. 2018

  9. Scarcity

  10. Soft Skills

  11. Broader Context

  12. None
  13. None
  14. Database ETL Environment

  15. Environment

  16. None
  17. The Environment

  18. The Environment

  19. The Environment

  20. We could buy more. Clouds scale

  21. The Environment

  22. The Environment

  23. The Environment 1

  24. Total Security VS. Risk Mitigation Clouds cost money Use all

    the resources available
  25. Database

  26. The Database

  27. The Database

  28. None
  29. None
  30. None
  31. None
  32. Tables.

  33. The Database 2

  34. None
  35. The Database

  36. None
  37. Development Environments are isolated for a reason Maps are Helpful

    Nightly Data is usually OK
  38. ETL

  39. The ETL

  40. The ETL

  41. 3

  42. We thought that this runtime was great! Well, kind of.

  43. The ETL

  44. 4

  45. 5

  46. The ETL

  47. None
  48. None
  49. But it broke? Well, kind of.

  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None
  57. I learned 6 ways to perform an ETL 10.5 |

    register view Rules can be broken
  58. Resources can constrain our options. Or spur creativity.

  59. Thanks! Contact: Jordan Carmona GIS Specialist City of McKinney linkedin.com/in/jordancarmona/

    jcarmona@mckinneytexas.org
  60. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org
  61. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org
  62. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org
  63. Q & A Contact: Jordan Carmona GIS Specialist City of

    McKinney linkedin.com/in/jordancarmona/ jcarmona@mckinneytexas.org