Amazon SageMaker Autopilot を試す / pyml-niigata-20200620-automl

2020/06/20 Python機械学習勉強会 in 新潟 で発表した資料です。

Transcript

1. Amazon SageMaker Autopilot Λࢼ͢ ʙ AutoGluonΛఴ͑ͯ ʙ 2020-06-20 Pythonػցֶशษڧձ in ৽ׁ ּݪ ޺ (@kasacchiful)
2. Hiroshi Kasahara @kasacchiful @kasacchiful Software Developer Favorite: Community: • JAWS-UG Niigata • JaSST Niigata • ASTER • SWANII • etc. 
3. ຊ೔ͷ಺༰ • Amazon SageMaker AutopilotΛ࢖ͬͯ AutoMLΛମݧ͢Δ • AutoGluonͷ঺հ 
4. Amazon SageMaker ͷ AutoML ػೳ IUUQTBXTBNB[PODPNKQTBHFNBLFSBVUPQJMPU 
5. Amazon SageMaker • AWSͷػցֶशϓϥοτϑΥʔϜαʔϏε  ग़య
   "NB[PO
   4BHF.BLFS
   μογϡϘʔυ
6. Amazon SageMaker Autopilot • Amazon SageMaker ͷ AutoML ػೳ • τϨʔχϯά࣌ͷΞϧΰϦζϜɺύϥϝʔλ ௐ੔ΛࣗಈԽ͠ɺ࠷దͳϞσϧΛ࡞੒Ͱ͖Δ IUUQTBXTBNB[PODPNKQTBHFNBLFSBVUPQJMPU 
7. ͬͦ͘͞΍ͬͯΈΔ 
8. ΍Δ͜ͱ • Amazon SageMaker AutopilotͷαϯϓϧΛಈ͔͢ • αϯϓϧίʔυ͸ҎԼͷϊʔτϒοΫ • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/ autopilot/sagemaker_autopilot_direct_marketing.ipynb • ࢖༻σʔληοτ: UCI Machine Learning Repository: Bank Marketing Data Set • https://archive.ics.uci.edu/ml/datasets/bank+marketing • ϙϧτΨϧͷۚ༥ػؔͷμΠϨΫτϚʔέςΟϯάΩϟ ϯϖʔϯʹؔ࿈͠ɺΫϥΠΞϯτ͕ఆظ༬ۚΛαϒεΫ ϥΠϒ͢Δ͔Ͳ͏͔Λ༧ଌ͢Δɻ 
9. ΍Δ͜ͱ௥Ճ • Kaggleͷॅ୐Ձ֨༧ଌίϯϖͷσʔλͰࢼ͢ • https://www.kaggle.com/c/house-prices-advanced-regression-techniques/overview • ࣮ߦ಺༰͸ҎԼͷϊʔτϒοΫʹهࡌ • https://github.com/kasacchiful/autopilot-sample-kaggle-house-prices • λΠλχοΫίϯϖ͸σʔλ݅਺͕଍Γͳͯ͘ AutopilotͰΤϥʔʹͳͬͯ͠·ͬͨɻ • trainσʔλ݅਺: 891݅ (Autopilot͸1,000݅Ҏ্ඞཁ) 
10. σϞ 
11. Kaggleͷ݁Ռ 
12. ྑ͍఺ • JupyterNotebook্ͷPythonιʔε͔Βɺ͍ Ζ͍Ζ͍͡ΕΔ • νϡʔχϯά·ͰશࣗಈͰ΍ͬͯ͘Εͯɺͱͯ ΋ϥΫ 
13. վળ΄͍͠఺ • ςʔϒϧܗࣜҎ֎ͷσʔλ΋ѻ͑ΔΑ͏ʹͯ͠΄͍͠ • ͲΕ͘Β͍͓͔͔ۚΔͷ͔ɺ͍ͭAutopilotδϣϒ͕ ऴΘΔͷ͔Α͘Θ͔ͬͯͳ͍ͷͰɺϋϥϋϥυΩυΩ ͠ͳ͕Β࣮ߦ͠ͳ͍ͱ͍͚ͳ͍ɻ • ֶश࣌͸ΠϯελϯεՔಇ࣌ؒͰ՝ۚ͞ΕΔ͕ɺνϡʔχϯάͷτ Ϩʔχϯάδϣϒ͕ΨϯΨϯ૿͍͑ͯ͘ͷ͕ɺͳΜ͔ා͍... → ݁ہ2 ࣌ؒ͘Β͍͔͔ͬͨ • ࠓճ͸ΦϨΰϯϦʔδϣϯͰ"ml.m5.4xlarge"ΠϯελϯεΛ࢖͍ͬͯ ΔͷͰɺ1࣌ؒ͋ͨΓ1.075USD͔͔Δɻ 
14. ςʔϒϧҎ֎ͷσʔλ΋ ѻ͍͍ͨ… 
15. ը૾ͷAutoMLࢼ͍ͨ͠Μ͕ͩ... • Amazon SageMaker Autopilotͷػೳ֦ு·Ͱ ଴ͭ • ଴ͯͳ͍ํ͸ɺGCPͷAutoMLͳͲΛࢼͯ͠Έ Δ͔ɺʮAutoGluonʯͰ΋ࢼͯ͠ΈΔ? ςʔϒϧσʔλҎ֎ͷ΋ͷΛAutoMLͰ͖ͳ͍͔ ୳͍ͯͨ͠Βۮવݟ͚ͭͨʮAutoGluonʯ 
16. AutoGluon • AutoML Toolkit for Deep Learning • https://github.com/awslabs/autogluon “AutoGluon automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.” • Gluon͸AWSͱMicrosoft͕ڞಉ։ൃ͍ͯ͠ΔɺσΟʔϓϥʔχϯ άϥΠϒϥϦ • TensorFlow/Keras 㱻 MXNet/Gluon ͳରԠ͔ͳ? • ී௨ͷPythonϥΠϒϥϦͳͷͰɺSageMakerͰͳͯ͘΋ಈ࡞͢Δ 
17. AutoGluon Example  # First install package from terminal: pip install mxnet autogluon from autogluon import TabularPrediction as task train_data = task.Dataset(file_path='https:// autogluon.s3.amazonaws.com/datasets/Inc/train.csv') test_data = task.Dataset(file_path='https:// autogluon.s3.amazonaws.com/datasets/Inc/test.csv') predictor = task.fit(train_data=train_data, label='class') performance = predictor.evaluate(test_data) ग़య
    IUUQTHJUIVCDPNBXTMBCTBVUPHMVPO
18. AutoGluon Example (Image Classification with FashionMNIST)  import autogluon as ag from autogluon import ImageClassification as task filename = ag.download('https://autogluon.s3.amazonaws.com/datasets/ shopee-iet.zip') ag.unzip(filename) dataset = task.Dataset('data/train') test_dataset = task.Dataset('data/test', train=False) if ag.get_gpu_count() == 0: dataset = task.Dataset(name='FashionMNIST') test_dataset = task.Dataset(name='FashionMNIST', train=False) ग़య
    IUUQTBVUPHMVPONYOFUJPUVUPSJBMTJNBHF@DMBTTJpDBUJPOCFHJOOFSIUNM
19. AutoGluon Example (Image Classification with FashionMNIST)  # train classifier = task.fit(dataset, epochs=5, ngpus_per_trial=1, verbose=False) print('Top-1 val acc: %.3f' % classifier.results['best_reward']) # predict # skip this if training FashionMNIST on CPU. if ag.get_gpu_count() > 0: image = 'data/test/BabyShirt/BabyShirt_323.jpg' ind, prob, _ = classifier.predict(image, plot=True) print('The input picture is classified as [%s], with probability %.2f.' % (dataset.init().classes[ind.asscalar()], prob.asscalar())) image = 'data/test/womenchiffontop/womenchiffontop_184.jpg' ind, prob, _ = classifier.predict(image, plot=True) print('The input picture is classified as [%s], with probability %.2f.' % (dataset.init().classes[ind.asscalar()], prob.asscalar())) ग़య
    IUUQTBVUPHMVPONYOFUJPUVUPSJBMTJNBHF@DMBTTJpDBUJPOCFHJOOFSIUNM
20. ଞʹ΋͜Μͳ࢖͍ํ͕͋Δ  IUUQTBVUPHMVPONYOFUJPUVUPSJBMTUPSDIIQPIUNM
21. ·ͱΊ 
22. ·ͱΊ • Amazon SageMaker AutopilotͰAutoMLΛ؆୯ʹ ࢖͑Δ • ݱࡏ͸ςʔϒϧܗࣜͷσʔλ͔͠ѻ͑ͳ͍͕ɺ SageMaker͸Ξοϓσʔτ͕ଟ͍ͷͰɺAWS re:Inventͷൃදʹظ଴͍ͨ͠ • ଴ͯͳ͍ํ͸AutoGluonΛ࢖͏ͷ΋໘ന͍͔΋ 
23. օ͞Μ΋ૉఢͳ AutoMLϥΠϑΛ 
24. ࢀߟ Amazon SageMaker Autopilot ؔ࿈ • Amazon Sagemaker Autopilot | Amazon Sagemaker • https://aws.amazon.com/jp/sagemaker/autopilot/ • Amazon SageMaker Autopilot ͷ։࢝ํ๏ - Amazon SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/autopilot-automate-model- development-get-started.html • Amazon SageMaker AutopilotΛ࢖ͬͯΈͨ | Developers.IO • https://dev.classmethod.jp/articles/try-amazon-sagemaker-autopilot/ • Amazon SageMaker Autopilot ͷαϯϓϧͰϞσϧΛ࡞੒ͯ͠Έͨ - Qiita • https://qiita.com/NomuK3/items/fb4c8bd2e4c7724457c3 • Amazon SageMaker Studio ͷಋೖͱ Autopilot ʹ͍ͭͯˌ̍ - Qiita • https://qiita.com/SatoshiGachiFujimoto/items/eecb66f5d57cb50324c7 • Amazon SageMaker Studio ͷಋೖͱ Autopilot ʹ͍ͭͯ ˌ̎ - Qiita • https://qiita.com/SatoshiGachiFujimoto/items/95c9e5877959bfe3b7ca 
25. ࢀߟ AutoGluon ؔ࿈ • AutoGluon: AutoML Toolkit for Deep Learning ʔ AutoGluon Documentation 0.0.1 documentation • https://autogluon.mxnet.io/index.html • AutoGluonΛ࢖ͬͯΈͨ | Developers.IO • https://dev.classmethod.jp/articles/yoshim_autogluon_sagemaker/