Microsoft の AI 技術 をベースに構成される Craft Copilot

https://azureai.connpass.com/event/319491/ でお話した
Craft Copilot での Microsoft の AI 技術 をベースに構成される アーキテクチャについて

Transcript

1. © PLAID, Incɹ Microsoft AI Craft Copilot 2 024 . 0 7 . 2 5 Azure OpenAI Service Dev Day
2. © PLAID, Incɹ 竹 　 行 VP of Engineering & Product, Ecosystem 2014 KARTE 用 PaaS KARTE Craft 心 PLAID Ecosystem 20 22 APN AWS Top Engineers 支 士 2
3. © PLAID, Incɹ Craft Copilot Craft Copilot ? 3
4. © PLAID, Incɹ Craft Copilot 4 KARTE Craft (PaaS) Ͱͷ ։ൃࢧԉΛͯ͘͠ΕΔ AI Ξγελϯτ ݱࡏ͸ Beta ൛ͱͯ͠ఏڙத
5. © PLAID, Incɹ KARTE Craft KARTE Craft ? 5
6. © PLAID, Incɹ KARTE Craft 6 KARTE ͷ֦ு͔ΒಠࣗͷΞϓϦέʔγϣϯ·Ͱɺ ΧελϚʔσʔλυϦϒϯͳػೳΛ։ൃͰ͖ΔPaaS
7. © PLAID, Incɹ KARTE Craft 7 https://solution.karte.io/tags/craft-functions/
8. © PLAID, Incɹ Craft Copilot Craft Copilot ͸۩ମతʹԿΛͯ͘͠Ε Δ ? 8
9. © PLAID, Incɹ Craft Copilot 9 https://solution.karte.io/blog/2024/07/generate-html-with-sites-copilot/
10. © PLAID, Incɹ Craft Copilot 10
11. © PLAID, Incɹ Craft Copilot ͷཪଆ Craft Copilot ͷཪଆ 11
12. © PLAID, Incɹ Craft Copilot ͷཪଆ 12 ίʔυվળͷཪଆ HTMLੜ੒ͷཪଆ
13. © PLAID, Incɹ HTMLੜ੒ͷཪଆ HTMLੜ੒ 13 ը૾Λೖྗ͢Δͱɺ Craft Sites Ͱར༻Ͱ͖ΔHTMLΛੜ੒͢Δػೳ
14. © PLAID, Incɹ HTMLੜ੒ͷཪଆ 14
15. © PLAID, Incɹ GPT-4o • ࠷ऴతͳHTMLϑΝΠϧͷੜ੒ • HTMLͷੜ੒ਫ਼౓Λ͋͛ΔͨΊʹɺ ◦ System messageͰٕज़ελοΫ΍ϑϨʔϜΛࢦఆ ◦ Few shot ͰγϯϓϧͳHTMLੜ੒ͷྫ • Stream Completion Λ࢖ͬͯɺମݧͷ޲্ ◦ 15 sec -> 4 Sec GPT-4o mini • ޙଓͷHTMLੜ੒ͷॲཧͰ࢖͍΍͍͢Α͏ʹɺςΩετͳͲΛ੔ཧ • JSON Mode Ͱޙଓʹ౉͠΍͍͢σʔλߏ଄Ͱग़ྗ શମ • جຊతʹɺMicrosoft ެ։ͷʮϓϩϯϓτ ΤϯδχΞϦϯάͷख๏ʯʹै͏ • λεΫΛ෼ׂ͢Δ͜ͱͰޮ཰తͳϞσϧ/ϓϩϯϓτͷ࢖͍ํΛ͢Δ • ϞσϧͷਐԽΛظ଴ͯ͠ɺϓϩϯϓτɺϑϩʔΛ࡞ΓࠐΈ͗͢ͳ͍ ◦ աڈʹGPT3.5 turboͰؤு͍ͬͯͨػೳ͕ҰॠͰࡴ͞Εͨ HTMLੜ੒ͷཪଆ (Azure Open AI Service) 15
16. © PLAID, Incɹ ͳͥ AOAI ͷ GPT-4o ͔? 16 • AI ͷτοϓϥϯφʔͰ͋Δ OpenAI ͷϞσϧ • ։ൃணखͷλΠϛϯάͰɺ֎෦ͷධՁ + ࣗ෼ͨͪͰ৮ͬͯΈͨධՁ ◦ ϕϯμʔ͕ग़͢score͸৴͡ͳ͍ͱ͍͏ελϯε ◦ શମతͳύϑΥʔϚϯε͕Α͍Ϟσϧ ◦ ೔ຊޠͷύϑΥʔϚϯε͕Α͍Ϟσϧ • αϙʔτମ੍ͷ҆ఆײ ◦ Microsoft Japanͷ҆ఆײ ◦ Azure AOAI ͷαϙʔτͷ҆ఆײ • API ͷ҆ఆੑ ◦ Ϟσϧͱͯ͠ͷύϑΥʔϚϯε͸ྑ͍͕ɺAPIͱͯ͠͸ෆ҆ఆ ͰɺΤϥʔ͕ى͜Γ΍͍͢Ϟσϧ΋͋Δ Chatbot Arena leaderBoard(As of July 23, 2024): https://chat.lmsys.org/?leaderboard
17. © PLAID, Incɹ HTMLੜ੒ͷཪଆ (Azure API Management) 17 Մ༻ੑɺΩϟύγςΟɺϝϯςφϏϦςΟͷ޲্Λ໨తʹར༻ Introducing GenAI Gateway Capabilities in Azure API Management Backend Pool - AOAIͷϦιʔε͝ͱʹ࡞੒͞ΕͨBackend͕ඥ͚ͮΔ - priorityɺweight Λࢦఆͯ͠ɺॊೈʹίϯτϩʔϧ - (࠷ۙGAʹͳͬͨͷͰTerraform͸ະରԠ) αʔΩοτϒϨʔΧʔ - { failure Condition: { count: 1 interval: 'PT10S' statusCodeRanges: [ { min: 429 max: 429 } ] } ೝূ - AOAIͱApiMؒ - AOAI͕ॴଐ͢ΔResourceGroup΁ͷݖݶ෇༩ͰͷηΩϡΞͳೝূ
18. © PLAID, Incɹ ίʔυվળͷཪଆ ίʔυվળ 18 ίʔυΛࢦఆͯ͠Copilotʹࢦࣔ͢Δͱɺ CraftͷϧʔϧΛ౿·͑ɺίʔυΛमਖ਼ͯ͘͠ΕΔػೳ
19. © PLAID, Incɹ ίʔυվળͷཪଆ 19
20. © PLAID, Incɹ ίʔυվળͷཪଆ (Azure Open AI Service) 20 GPT-4o mini - 1. ޙଓͷॲཧͰ࢖͍΍͍͢Α͏ʹɺςΩετͳͲΛ੔ཧ - JSON Mode Ͱޙଓʹ౉͠΍͍͢σʔλߏ଄Ͱग़ྗ - 2. Azure AI ServiceͷσʔλΛ࢖ͬͯճ౴Λੜ੒ - Stream Completion Λ࢖ͬͯɺମݧͷ޲্ text-embedding-3 - υΩϡϝϯτͷ৘ใ΍ɺݕࡧʹ͓͚ΔςΩετΛϕΫτϧԽ - large(3072࣍ݩ)ɺsmall(1536࣍ݩ)Λ྆ํར༻ - smallͰ΋ΫΦϦςΟ͸ग़͍ͯΔ͕ɺࠓޙͷ֦ுΛ౿·͑ͯ
21. © PLAID, Incɹ ίʔυվળͷཪଆ (Azure AI Search 1/2) 21 AI ػೳ͕౷߹͞ΕͨɺRAG ߏ੒ͷத৺ʹͳΔ ڧྗͳݕࡧΤϯδϯ OpenAI ͷཪଆͰ΋࢖ΘΕ͍ͯΔ͜ͱ͕ൃද͞Ε͍ͯΔ อଘϕΫτϧσʔλ - Craft Developer Portal ͷ Markdown σʔλ ͕ιʔε - ݱ࣌఺ͰɺνϟϯΫ͸͍ͯ͠ͳ͍ - ݱ࣌఺Ͱ΋ਫ਼౓͕ҰఆͰ͍ͯΔ - ࠓޙͷٕज़ਐԽʹظ଴͠ɺͰ͖Δ͚ͩγϯϓϧʹ - large(3072࣍ݩ)ɺsmall(1536࣍ݩ)Λ྆ํอଘ
22. © PLAID, Incɹ ίʔυվળͷཪଆ (Azure AI Search 2/2) 22 ηϚϯςΟοΫϋΠϒϦουݕࡧ - ςΩετͱϕΫτϧͷϋΠϒϦουݕࡧ - Microsoft ͷݴޠཧղϞσϧͰݕࡧͷؔ࿈ੑΛߴΊΔ࠶ϥϯΫॲཧ - ΫΦϦςΟ্͕͕Δ͕ɺlatency͕૿͑Δ͜ͱ͕͋Δ - Azure AI Search ͷύϑΥʔϚϯε޲্ʹؔ͢Δώϯτ Ͱνϡʔχϯ ά ʮAI ʹ͖͘ʯ Ϙλϯ - Azure AI Search͸ຖճຬ఺ͷ݁ՌΛಘΕΔΘ͚Ͱͳ͍ - ؔ࿈͢ΔυΩϡϝϯτΛදࣔ͠ɺҙਤ͠ͳ͍݁Ռ͕དྷͨ৔߹ʹਓ https://qiita.com/nohanaga/items/95a4780757e89b29e93e
23. © PLAID, Incɹ ·ͱΊ 23 ίʔυվળͷཪଆ HTMLੜ੒ͷཪଆ
24. © PLAID, Incɹ ·ͱΊ 24 ίʔυվળͷཪଆ HTMLੜ੒ͷཪଆ - Craft Copilot ͸ Microsoft ͷ AIٕज़Λத৺ʹߏ੒͞Ε͍ͯΔ - Azureʹ͸ OpenAIΛ׆͔͍͢Ζ͍ΖͳαʔϏε͕ఏڙ͞Ε͍ͯΔ - ߏ੒͸ɺحΛͯΒΘͣɺϕετϓϥΫςΟεʹै͏ - ٕज़ਐԽΛظ଴͠ɺ࡞ΓࠐΉ෦෼ͱਐԽʹҕͶΔ෦෼ͷόϥϯεΛऔΔ - ·ͩ·ͩ β൛