ϓϩάϥϛϯάݴޠ Tlangͷ։ൃ Takato Yamada

͍͖ͳΓͰ͕͢ɺ ϓϩάϥϜΛߴ଎Խ͸େมͰ͢

΅͔͠ॲཧ (ߴ଎Խͳ͠)

΅͔͠ॲཧ (खͰߴ଎Խ)

΅͔͠ॲཧ (HalideͰߴ଎Խ)

΅͔͠ॲཧ (HalideͰߴ଎Խ) ·ͩਓखͰߴ଎Խ (εέδϡʔϦϯά)͕ඞཁ

Tlang (Tensor language) ͱ͸ • ΞϓϦέʔγϣϯࢥߟͳςϯιϧϕʔεɺϓϩάϥϛϯάݴޠ • ΞϧΰϦζϜͱεέδϡʔϦϯάΛ෼཭ͯ͠ॻ͚Δ • εέδϡʔϦϯάΛࣗಈͰߦ͑Δ (΋ͪΖΜखॻ͖΋Ͱ͖Δ) • ϔςϩδχΞεͳ؀ڥͰಈ͖ɺϢʔβ͸ࠩҟΛҙࣝ͠ͳͯ͘ྑ͍ • ϞμϯͳϓϩάϥϛϯάݴޠͰ࣮૷͢Δ͜ͱͰՄಡੑɺ։ൃޮ཰Λվળ • هड़ՄೳͳυϝΠϯΛը૾ॲཧ΍σΟʔϓϥʔχϯά͚ͩͰͳͯ͘ɺ
 HPC·Ͱ޿͍͛ͨ

ࠩผԽ • Tlang:
 ։ൃݴޠ͕Ϟμϯ(Swift, Rust)ɺϧʔϓ఻ൖґଘ͕هड़ՄೳͳϧʔϓϨϕϧDSL
 ΦʔτνϡʔχϯάΛ౥ࡌ
 ෼ࢄܭࢉ΍ܭࢉΧʔωϧͷඇಉظ࣮ߦʹରԠ͢Δ (શͯئ๬) • Tensorflow, Pytorch: 
 ։ൃݴޠ͕C++ ɺϧʔϓϨϕϧهड़Ͱ͖ͳ͍ (XLA͸Մೳ?) • Halide: 
 ։ൃݴޠ͕C++ɺϧʔϓ఻ൖґଘ͕ॻ͚ͳ͍(ϑϩϯτΤϯυͰ͸)ɺ
 Φʔτνϡʔχϯάͦ͜·Ͱڧ͘ͳ͍ɺ෼ࢄܭࢉͰ͖ͳ͍ • TVM: 
 ։ൃݴޠ͕C++ͱPythonɺϧʔϓ఻ൖґଘ͕ॻ͚ͳ͍ɺ෼ࢄܭࢉͰ͖ͳ͍ • Tiramisu: ։ൃݴޠ͕C++ɺΦʔτνϡʔχϯά͕ͳ͍

• Tlang:
 ։ൃݴޠ͕Ϟμϯ(Swift, Rust)ɺϧʔϓ఻ൖґଘ͕هड़ՄೳͳϧʔϓϨϕϧDSL
 ΦʔτνϡʔχϯάΛ౥ࡌ
 ෼ࢄܭࢉ΍ܭࢉΧʔωϧͷඇಉظ࣮ߦʹରԠ͢Δ (શͯئ๬) • Tensorflow, Pytorch: 
 ։ൃݴޠ͕C++ ɺϧʔϓϨϕϧهड़Ͱ͖ͳ͍ (XLA͸Մೳ?) • Halide: 
 ։ൃݴޠ͕C++ɺϧʔϓ఻ൖґଘ͕ॻ͚ͳ͍(ϑϩϯτΤϯυͰ͸)ɺ
 Φʔτνϡʔχϯάͦ͜·Ͱڧ͘ͳ͍ɺ෼ࢄܭࢉͰ͖ͳ͍ • TVM: 
 ։ൃݴޠ͕C++ͱPythonɺϧʔϓ఻ൖґଘ͕ॻ͚ͳ͍ɺ෼ࢄܭࢉͰ͖ͳ͍ • Tiramisu: ։ൃݴޠ͕C++ɺΦʔτνϡʔχϯά͕ͳ͍ ࠩผԽ (স) ݸਓͰશͯΛ࣮૷͢Δ͜ͱ͸΄΅ෆՄೳ. ݁ہͷॴͳʹ͔࡞Γ͍͚ͨͩস

ϥ ϯ λ Π Ϝ ߏ੒ ؔ਺ܕελΠϧ ϑϩϯτΤϯυ ίʔυੜ੒ LLVM ݴޠ಺ϑϩϯτΤϯυ ߴϨϕϧ IR ࠷దԽػ 1 ௿Ϩϕϧ IR ࠷దԽػ 2 GPU CPU ෼ࢄ ୯Ұ ࣗಈ࠷దԽػ ϥ Π ϒ ϥ Ϧ

ؔ਺ܕελΠϧ ϑϩϯτΤϯυ ίʔυੜ੒ ݴޠ಺ϑϩϯτΤϯυ ߴϨϕϧ IR ࠷దԽػ 1 ௿Ϩϕϧ IR ࠷దԽػ 2 ࣗಈ࠷దԽػ ߏ੒ ϥ ϯ λ Π Ϝ ϥ Π ϒ ϥ Ϧ LLVM GPU CPU ෼ࢄ ୯Ұ

ϥ ϯ λ Π Ϝ ϥ Π ϒ ϥ Ϧ ؔ਺ܕελΠϧ ϑϩϯτΤϯυ ίʔυੜ੒ ݴޠ಺ϑϩϯτΤϯυ ߴϨϕϧ IR ࠷దԽػ 1 ௿Ϩϕϧ IR ࠷దԽػ 2 ࣗಈ࠷దԽػ ͜͜Λ௥Ճ͍ͨ͠! ݟੵ΋Γ LLVM GPU CPU ෼ࢄ ୯Ұ

ϥ ϯ λ Π Ϝ ϥ Π ϒ ϥ Ϧ ؔ਺ܕελΠϧ ϑϩϯτΤϯυ ίʔυੜ੒ ݴޠ಺ϑϩϯτΤϯυ ߴϨϕϧ IR ࠷దԽػ 1 ௿Ϩϕϧ IR ࠷దԽػ 2 ࣗಈ࠷దԽػ ͕͜͜ݮͬͨ ݁Ռ LLVM GPU CPU ෼ࢄ ୯Ұ

• ࢀߟϓϩδΣΫτΛॻ͖௚͍ͯͨ͠ॴɺDeep copy͕Ͱ͖ͳ͍͜ͱ ͕෼͔ͬͨ
 →ίϯύΠϥͷIR͸ϥϕϧ෇͖༗ޮάϥϑͰද͞Ε͍ͯΔ͕ɺάϥ ϑΛίϐʔ͢ΔͨΊʹ͸ɺઌߦ఺Λશͯίϐʔ͠ͳ͍ͱ͍͚ͳ͍… • ϊʔυͷछྨ͕ଟ͍͠ɺࠓޙ΋૿͑Δͷख࣮૷͸େม ࣗಈͰੜ੒ͯ͠΍Ζ͏! ͑ͬͳʹ΍ͬͯͨͷʁ

• ࢀߟϓϩδΣΫτΛॻ͖௚͍ͯͨ͠ॴɺDeep copy͕Ͱ͖ͳ͍͜ͱ ͕෼͔ͬͨ
 →ίϯύΠϥͷIR͸ϥϕϧ෇͖༗ޮάϥϑͰද͞Ε͍ͯΔ͕ɺάϥ ϑΛίϐʔ͢ΔͨΊʹ͸ɺઌߦ఺Λશͯίϐʔ͠ͳ͍ͱ͍͚ͳ͍… • ϊʔυͷछྨ͕ଟ͍͠ɺࠓޙ΋૿͑Δͷख࣮૷͸େม ࣗಈͰੜ੒ͯ͠΍Ζ͏! ͑ͬͳʹ΍ͬͯͨͷʁ ѱເͷ࢝·Γ

ͱΓ͋͑ͣਐḿ D {Add(C[Var(i), Var(j)], {C {Add(A[Var(i), Var(j)], {Input(A)}, B[Var(i), Var(j)], {Input(B)})}}, IntConst(3))}

ίʔυΛੜ੒͢ΔͨΊͷίʔυ(ίϯύΠϥ)Λ
 ੜ੒͢ΔίʔυΛॻ͘৬ਓʹͳΓ·ͨ͠ ݁࿦