JAWS-UG AI #9
 ήʔϜӡӦͱػցֶश
 [SageMaker RL Λ࢖ͬͯΈͯ] ݹ৓ लོ mixi, Inc.

ݹ৓लོ ։ൃຊ෦ͨΜΆΆάϧʔϓ (ೖࣾ11೥໨) αʔό։ൃ
 |> ΞϓϦӡ༻
 |> ήʔϜ։ൃ
 |> ػցֶश ࣗݾ঺հ

Agenda ήʔϜͷ঺հͱ๊͑Δ໰୊఺ ػցֶशͷར༻ߏ੒ SageMaker RL Λ࢖ͬͨײ૝ͱཁ๬ P3 P9 P14

ήʔϜͷ঺հ

ϑΝΠτϦʔάͱ͸ LEADER HANDS LEADER LEADER HANDS LEADER HANDS

• ໿2िؒͷγʔζϯ੍ • γʔζϯ͝ͱʹΩϟϥΫλʔ͕ ௥Ճ͞ΕΔ ϑΝΠτϦʔάͱ͸

• ৽نϑΝΠλʔ௥ՃͰήʔϜόϥϯε่͕ΕΔ͜ͱ͕ൃੜ • ༧૝֎ͷ૊Έ߹Θͤͯγφδʔ͕ൃੜ͢Δέʔε • ૝ఆҎ্ʹڧ͗ͨ͢Χʔυ • ৽͍͠ϧʔϧมߋΛ͢ΔࡍʹશͯͷΧʔυͷӨڹΛௐࠪ͠ͳ͚Ε͹͍͚ͳ͍ ๊͑Δ໰୊ • ਓखͰҡ࣋͢Δͷ͸ίετ͕ߴ͍ • ػցͷྗʹཔΓ͍ͨ • ࠷ڧͷAIΛ࡞ͬͯόϥϯεௐ੔ͷςετϓϨΠΛ͓ئ͍͍ͨ͠

ػցֶशͷಋೖ؀ڥ

γεςϜߏ੒ Client Application(Ϣʔβ୺຤) APP Server(Socket Layer) APP Server(Logic Layer) RPC AWS • Logic Server͕
 ήʔϜϩδοΫΛ͍࣋ͬͯΔ • Client͸ԋग़෦෼ͷΈ • ϢʔβͷήʔϜதͷߦಈ͸
 ϩάͱͯ͠อଘՄೳ

൫໘৘ใͷCNN ωοτϫʔΫ εΩϧ৘ใͷ ϕΫτϧԽ पล৘ใͷDNN બ୒Մೳߦಈͷ֬཰

γεςϜߏ੒ Client Application(Ϣʔβ୺຤) APP Server(Socket Layer) APP Server(Logic Layer) RPC AWS • ϓϨΠϠʔϩάΛ࢖ͬͨ
 ڭࢣ͋ΓֶशͷϞσϧ • SageMaker EndpointͰ
 ഑ஔ݁ՌΛਪ࿦ • ϞσϧͷAPIΤϯυϙΠϯτԽ • ඇఀࢭʹΑΔϞσϧ੾Γସ͑ • εέʔϧ΋Մೳ SageMaker Endpoint

ˇ πʔϧԽ

ڧԽֶश΁ͷऔΓ૊Έ

ڧԽֶश • ΑΓΩϟϥΫλʔʹ߹ΘֶͤͨशΛ͢ΔͨΊʹڧԽֶशͷΞϓϩʔν΋ඞཁ • ࣮૷ͨ͠εΩϧΛ౤ೖલʹ͖ͪΜͱݕূ͍ͨ͠ • ৽͍͠ରઓ؀ڥͷӨڹΛڭࢣ͋Γ͚ͩͰ͸ֶशͰ͖ͳ͍ • ຊ౰ʹτοϓϓϨΠϠʔͱಉ͡ڧ͞ʹ͸౸ୡͰ͖͍ͯͳ͍ • ࣗલͰ࣮૷͍͕ͯͨ͠Ͱ͖Δ͜ͱͳΒେ͖ͳྗʹ૬৐Γ͔ͨͬͨ͠ • SageMaker RLͷbetaςετʹࢀՃͤͯ͞΋Β͑ͨͷͰࢼͯ͠Έͨ

SageMaker RL beta (ਫฏεέʔϧ) Docker image Training Job Docker image Docker image Docker image (Trainer) Docker image S3 bucket Notebook
 Instance Redis Weights Experience Replay

ݕূͨ͠಺༰ • ϑϨʔϜϫʔΫʹ৐Βͣʹࣗ෼ͷ؀ڥΛͦͷ·· • ͜ͷ৔߹͸࿈ܞ͕Ͱ͖ͳ͍ͷͰ1node͔͠ಈ͔ͤͳ͍ Training Job Docker image Notebook
 Instance LogicServer
 (elixir) GameEnv (python) ࣗ࡞RL ϩδοΫ
 (pyton) http S3 bucket model & checkpoint

ݕূͨ͠಺༰ • ਪ঑ͷCoach & GymܗࣜͰ࠶࣮૷ • ͜ͷܗͩͱ؆୯ʹਫฏ෼ׂͷԸܙΛड͚ΕΔ Training Job Docker image Notebook
 Instance LogicServer
 (elixir) GymEnv (python) http S3 bucket model & checkpoint Intel Coach

ݕূͰͷײ૝ • े෼࣮༻Ͱ͖ΔϨϕϧ • ͨͩ͠ɺ·ͩ໨ʹݟ͑Δൣғ಺ʹటष͍෦෼͕ͨ͘͞Μ࢒͍ͬͯΔ • ग़ྗ͞ΕΔσʔλ͸Tensorflowͷ΋ͷͳͷͰɺ৭ʑऔΓճ͕͠ޮ͘ • CoachͱGymͷྲّྀʹ৐ΔͱΞϧΰϦζϜͷ࣮૷ͳͲΘ͔Βͳ͍Ͱ΋࢖͑Δ • ٯʹGymͳͲͷྲّྀʹ৐Βͳ͍ͱ͍͚ͳ͘ͳΔ(single agentͳͲ) • Ray RLib+ multi agentͳͲ΋ࢼͯ͠Έ͍ͨ • ߋ৽ͷεϐʔυ͕ͱʹ͔͘ૣ͍ • ·ͩ·ͩൃలதͷͨΊɺॻ͖ํ͕ͲΜͲΜ৽͘͠ͳ͍ͬͯ͘ • ·ͩαϯϓϧίʔυಡΈղ͍࣮ͯ૷͍ͯ͘͠ײ͸͋ΔͷͰ
 υΩϡϝϯτྨͷॆ࣮ʹ΋ظ଴

Thank you !!