amazon-dsstne-2016-07

Transcript

1. DL for Recommendations OpenHyperScale ษڧձ 16.07.28
2. ͦͷલʹ 2
3. 3 V1.0! http://mesos.apache.org/blog/mesos-1-0-0-released/ (Yesterday, Jul 27 2016)
4. 4 GPU SUPPORT!! https://github.com/apache/mesos/blob/1.0.0/CHANGELOG
5. ࢒՝୊΋͋ͱগ͠ 5 https://issues.apache.org/jira/browse/MESOS-4424
6. 6 WHAT ABOUT DC/OS? https://mesosphere.com/blog/2016/07/27/apache-mesos-reaches-1-0/
7. ଓ͖͸࣍ճ.. 7
8. ຊ୊ 8
9. Ϩίϝϯυ 9
10. “͜ͷ঎඼Λങͬͨਓ͸͜Μͳ঎඼΋ങ͍ͬͯ·͢” 10
11. 11 http://aws.typepad.com/sajp/2016/07/generating-recommendations-at-amazon-scale-with-apache-spark-and-amazon-dsstne.html AMAZON ͷύʔιφϧϨίϝϯυΛ AWS Ͱ࣮૷
12. 12 ֶशσʔλੜ੒ɿ Amazon EMR (cluster) + Apache Spark (CPU) ֶशɾਪ࿦ɿ ECS (cluster) + ECR (docker image) + Amazon DSSTNE (GPU) σʔλసૹɾڞ༗ɿ Amazon S3 Ϩίϝϯυੜ੒؀ڥͷߏ੒
13. 13 ೖྗɾग़ྗϨΠϠʹ Amazon ͷ঎඼਺ेສ఺Λ഑ஔ͠ɺ ݸਓ͝ͱʹ͜ͷϞσϧΛ༻͍ͯϨίϝϯυ͢΂͖঎඼Λਪଌɻ Ϩίϝϯσʔγϣϯੜ੒ DNN Ϟσϧ ※ ࠨਤɺޡղΛট͘ֆͰ͢Έ·ͤΜɻ ࣮ࡍʹ͸ೖྗ૚ͱग़ྗ૚͸ಉ͡ϊʔυ਺
14. DSSTNE 14
15. 15 εύʔεσʔλʹ࠷దԽ͠ɺ଎౓ͱεέʔϧΛॏࢹͨ͠ DNN with GPUs ͳϥΠϒϥϦɻAmazon ͕ OSS ͱͯ͠ެ։ɻ ࣗࣾͷ DL ʹ·ͭΘΔ՝୊ղܾͷͨΊʹ࡞੒͞Εͨ΋ͷɻ AMAZON DSSTNE https://github.com/amznlabs/amazon-dsstne/
16. 16 ɾ੡඼ͱͯ͠࢖͑Δ͚ͩͷύϑΥʔϚϯεΛॏࢹ ɾϨίϝϯυੜ੒΍ࣗવݴޠཧղॲཧͰѻ͏εύʔεσʔλͰಛʹߴ଎ ɹɹʢg2.8xlarge Πϯελϯε 1 ୆Ͱ͸ TensorFlow ͷ໿ 2.1 ഒʣ ɾαʔό 1 ୆ × ෳ਺ GPU Ͱͷԋࢉ͕͔ͳΓಘҙ ɹɹ-> GPU ͝ͱʹλεΫΛࣗಈ෼ࢄ ɹɹ-> GPU Λ૿΍ͤ͹ॲཧ଎౓ΛߴΊΒΕΔʢεέʔϧΞ΢τʣ ɹɹɹ& ΑΓେ͖ͳϞσϧ͕ѻ͑Δ ɾωοτϫʔΫͷఆ͕ٛ Caffe ΑΓ΋γϯϓϧ ɾ৞ΈࠐΈ૚ʹ͸ະରԠ ɾϦΧϨϯτ૚΋ݱࡏ͸ݶఆతαϙʔτɻ͍ͣΕ΋ϩʔυϚοϓʹ͸͋Δɻ ͦͷଞͷ DL ϥΠϒϥϦͱͷҧ͍ͳͲ https://github.com/amznlabs/amazon-dsstne/blob/master/FAQ.md