Amazon SageMaker Autopilot を試す / pyml-niigata-20200620-automl

Amazon SageMaker Autopilot を試す / pyml-niigata-20200620-automl

2020/06/20 Python機械学習勉強会 in 新潟 で発表した資料です。

82d6167c4d14393c2e20b37a74b363c5?s=128

kasacchiful

June 20, 2020
Tweet

Transcript

  1. Amazon SageMaker Autopilot Λࢼ͢ ʙ AutoGluonΛఴ͑ͯ ʙ 2020-06-20 Pythonػցֶशษڧձ in

    ৽ׁ ּݪ ޺ (@kasacchiful)
  2. Hiroshi Kasahara @kasacchiful @kasacchiful Software Developer Favorite: Community: • JAWS-UG

    Niigata • JaSST Niigata • ASTER • SWANII • etc. 
  3. ຊ೔ͷ಺༰ • Amazon SageMaker AutopilotΛ࢖ͬͯ AutoMLΛମݧ͢Δ • AutoGluonͷ঺հ 

  4. Amazon SageMaker ͷ AutoML ػೳ IUUQTBXTBNB[PODPNKQTBHFNBLFSBVUPQJMPU 

  5. Amazon SageMaker • AWSͷػցֶशϓϥοτϑΥʔϜαʔϏε  ग़య"NB[PO4BHF.BLFSμογϡϘʔυ

  6. Amazon SageMaker Autopilot • Amazon SageMaker ͷ AutoML ػೳ •

    τϨʔχϯά࣌ͷΞϧΰϦζϜɺύϥϝʔλ ௐ੔ΛࣗಈԽ͠ɺ࠷దͳϞσϧΛ࡞੒Ͱ͖Δ IUUQTBXTBNB[PODPNKQTBHFNBLFSBVUPQJMPU 
  7. ͬͦ͘͞΍ͬͯΈΔ 

  8. ΍Δ͜ͱ • Amazon SageMaker AutopilotͷαϯϓϧΛಈ͔͢ • αϯϓϧίʔυ͸ҎԼͷϊʔτϒοΫ • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/ autopilot/sagemaker_autopilot_direct_marketing.ipynb

    • ࢖༻σʔληοτ: UCI Machine Learning Repository: Bank Marketing Data Set • https://archive.ics.uci.edu/ml/datasets/bank+marketing • ϙϧτΨϧͷۚ༥ػؔͷμΠϨΫτϚʔέςΟϯάΩϟ ϯϖʔϯʹؔ࿈͠ɺΫϥΠΞϯτ͕ఆظ༬ۚΛαϒεΫ ϥΠϒ͢Δ͔Ͳ͏͔Λ༧ଌ͢Δɻ 
  9. ΍Δ͜ͱ௥Ճ • Kaggleͷॅ୐Ձ֨༧ଌίϯϖͷσʔλͰࢼ͢ • https://www.kaggle.com/c/house-prices-advanced-regression-techniques/overview • ࣮ߦ಺༰͸ҎԼͷϊʔτϒοΫʹهࡌ • https://github.com/kasacchiful/autopilot-sample-kaggle-house-prices •

    λΠλχοΫίϯϖ͸σʔλ݅਺͕଍Γͳͯ͘ AutopilotͰΤϥʔʹͳͬͯ͠·ͬͨɻ • trainσʔλ݅਺: 891݅ (Autopilot͸1,000݅Ҏ্ඞཁ) 
  10. σϞ 

  11. Kaggleͷ݁Ռ 

  12. ྑ͍఺ • JupyterNotebook্ͷPythonιʔε͔Βɺ͍ Ζ͍Ζ͍͡ΕΔ • νϡʔχϯά·ͰશࣗಈͰ΍ͬͯ͘Εͯɺͱͯ ΋ϥΫ 

  13. վળ΄͍͠఺ • ςʔϒϧܗࣜҎ֎ͷσʔλ΋ѻ͑ΔΑ͏ʹͯ͠΄͍͠ • ͲΕ͘Β͍͓͔͔ۚΔͷ͔ɺ͍ͭAutopilotδϣϒ͕ ऴΘΔͷ͔Α͘Θ͔ͬͯͳ͍ͷͰɺϋϥϋϥυΩυΩ ͠ͳ͕Β࣮ߦ͠ͳ͍ͱ͍͚ͳ͍ɻ • ֶश࣌͸ΠϯελϯεՔಇ࣌ؒͰ՝ۚ͞ΕΔ͕ɺνϡʔχϯάͷτ Ϩʔχϯάδϣϒ͕ΨϯΨϯ૿͍͑ͯ͘ͷ͕ɺͳΜ͔ා͍...

    → ݁ہ2 ࣌ؒ͘Β͍͔͔ͬͨ • ࠓճ͸ΦϨΰϯϦʔδϣϯͰ"ml.m5.4xlarge"ΠϯελϯεΛ࢖͍ͬͯ ΔͷͰɺ1࣌ؒ͋ͨΓ1.075USD͔͔Δɻ 
  14. ςʔϒϧҎ֎ͷσʔλ΋ ѻ͍͍ͨ… 

  15. ը૾ͷAutoMLࢼ͍ͨ͠Μ͕ͩ... • Amazon SageMaker Autopilotͷػೳ֦ு·Ͱ ଴ͭ • ଴ͯͳ͍ํ͸ɺGCPͷAutoMLͳͲΛࢼͯ͠Έ Δ͔ɺʮAutoGluonʯͰ΋ࢼͯ͠ΈΔ? ςʔϒϧσʔλҎ֎ͷ΋ͷΛAutoMLͰ͖ͳ͍͔

    ୳͍ͯͨ͠Βۮવݟ͚ͭͨʮAutoGluonʯ 
  16. AutoGluon • AutoML Toolkit for Deep Learning • https://github.com/awslabs/autogluon “AutoGluon

    automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.” • Gluon͸AWSͱMicrosoft͕ڞಉ։ൃ͍ͯ͠ΔɺσΟʔϓϥʔχϯ άϥΠϒϥϦ • TensorFlow/Keras 㱻 MXNet/Gluon ͳରԠ͔ͳ? • ී௨ͷPythonϥΠϒϥϦͳͷͰɺSageMakerͰͳͯ͘΋ಈ࡞͢Δ 
  17. AutoGluon Example  # First install package from terminal: pip

    install mxnet autogluon from autogluon import TabularPrediction as task train_data = task.Dataset(file_path='https:// autogluon.s3.amazonaws.com/datasets/Inc/train.csv') test_data = task.Dataset(file_path='https:// autogluon.s3.amazonaws.com/datasets/Inc/test.csv') predictor = task.fit(train_data=train_data, label='class') performance = predictor.evaluate(test_data) ग़యIUUQTHJUIVCDPNBXTMBCTBVUPHMVPO
  18. AutoGluon Example (Image Classification with FashionMNIST)  import autogluon as

    ag from autogluon import ImageClassification as task filename = ag.download('https://autogluon.s3.amazonaws.com/datasets/ shopee-iet.zip') ag.unzip(filename) dataset = task.Dataset('data/train') test_dataset = task.Dataset('data/test', train=False) if ag.get_gpu_count() == 0: dataset = task.Dataset(name='FashionMNIST') test_dataset = task.Dataset(name='FashionMNIST', train=False) ग़యIUUQTBVUPHMVPONYOFUJPUVUPSJBMTJNBHF@DMBTTJpDBUJPOCFHJOOFSIUNM
  19. AutoGluon Example (Image Classification with FashionMNIST)  # train classifier

    = task.fit(dataset, epochs=5, ngpus_per_trial=1, verbose=False) print('Top-1 val acc: %.3f' % classifier.results['best_reward']) # predict # skip this if training FashionMNIST on CPU. if ag.get_gpu_count() > 0: image = 'data/test/BabyShirt/BabyShirt_323.jpg' ind, prob, _ = classifier.predict(image, plot=True) print('The input picture is classified as [%s], with probability %.2f.' % (dataset.init().classes[ind.asscalar()], prob.asscalar())) image = 'data/test/womenchiffontop/womenchiffontop_184.jpg' ind, prob, _ = classifier.predict(image, plot=True) print('The input picture is classified as [%s], with probability %.2f.' % (dataset.init().classes[ind.asscalar()], prob.asscalar())) ग़యIUUQTBVUPHMVPONYOFUJPUVUPSJBMTJNBHF@DMBTTJpDBUJPOCFHJOOFSIUNM
  20. ଞʹ΋͜Μͳ࢖͍ํ͕͋Δ  IUUQTBVUPHMVPONYOFUJPUVUPSJBMTUPSDIIQPIUNM

  21. ·ͱΊ 

  22. ·ͱΊ • Amazon SageMaker AutopilotͰAutoMLΛ؆୯ʹ ࢖͑Δ • ݱࡏ͸ςʔϒϧܗࣜͷσʔλ͔͠ѻ͑ͳ͍͕ɺ SageMaker͸Ξοϓσʔτ͕ଟ͍ͷͰɺAWS re:Inventͷൃදʹظ଴͍ͨ͠

    • ଴ͯͳ͍ํ͸AutoGluonΛ࢖͏ͷ΋໘ന͍͔΋ 
  23. օ͞Μ΋ૉఢͳ AutoMLϥΠϑΛ 

  24. ࢀߟ Amazon SageMaker Autopilot ؔ࿈ • Amazon Sagemaker Autopilot |

    Amazon Sagemaker • https://aws.amazon.com/jp/sagemaker/autopilot/ • Amazon SageMaker Autopilot ͷ։࢝ํ๏ - Amazon SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/autopilot-automate-model- development-get-started.html • Amazon SageMaker AutopilotΛ࢖ͬͯΈͨ | Developers.IO • https://dev.classmethod.jp/articles/try-amazon-sagemaker-autopilot/ • Amazon SageMaker Autopilot ͷαϯϓϧͰϞσϧΛ࡞੒ͯ͠Έͨ - Qiita • https://qiita.com/NomuK3/items/fb4c8bd2e4c7724457c3 • Amazon SageMaker Studio ͷಋೖͱ Autopilot ʹ͍ͭͯˌ̍ - Qiita • https://qiita.com/SatoshiGachiFujimoto/items/eecb66f5d57cb50324c7 • Amazon SageMaker Studio ͷಋೖͱ Autopilot ʹ͍ͭͯ ˌ̎ - Qiita • https://qiita.com/SatoshiGachiFujimoto/items/95c9e5877959bfe3b7ca 
  25. ࢀߟ AutoGluon ؔ࿈ • AutoGluon: AutoML Toolkit for Deep Learning

    ʔ AutoGluon Documentation 0.0.1 documentation • https://autogluon.mxnet.io/index.html • AutoGluonΛ࢖ͬͯΈͨ | Developers.IO • https://dev.classmethod.jp/articles/yoshim_autogluon_sagemaker/