Summary of "Meet the Foundation Models framework"

https://love-swift.connpass.com/event/355976/

Transcript

1. Meet the Foundation Models framework Takeshi Tanaka (@p0dee) ू·ΕSwift޷͖ʂSwiftѪ޷ձεϐϯΦϑ WWDC25ηογϣϯཁ໿ձ @ DeNA 2025/06/26
2. Tanaka Takeshi X: p0dee iOS/Android Engineer झຯ͸δϟζͱےτϨͰ͢ɻ
3. ΦϯσόΠεͰͷɺେن໛ݴޠϞσϧͷར༻ΛՄೳʹ͢ΔϑϨʔϜϫʔΫ • macOSɺiOSɺiPadOSɺvisionOSͰར༻Մೳ • ΦϯσόΠεͷͨΊɾɾɾ • ϓϥΠόγʔอޢͱAIମݧͷ૊ΈࠐΈཱ͕྆Մ 👍 • ߴ౓ͳਪ࿦ʹ͸ෆ޲͖ 😕 • ஌͕ࣝݶΒΕɺࣄ࣮ݕূ͕ඞཁ 😕 Foundation Models framework
4. Foundation Models framework • ίϯςϯπੜ੒ʗੜ੒తର࿩ʗΞϓϦ಺ϢʔβʔΨΠυʗੜ੒తΫΠζʗ ύʔιφϥΠζʗ෼ྨʗཁ໿ʗηϚϯςΟοΫݕࡧʗચ࿅ɾਫ਼៛Խʗ࣭໰Ԡ ౴ʗΧελϚΠζֶशʗλάੜ੒ʗΤϯςΟςΟநग़ʗτϐοΫݕग़ • ηογϣϯϏσΦͰ͸ɺཱྀఔܭը΍ɺήʔϜ಺ͰͷNPCͱͷձ࿩
5. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: ɹɹɹɹɹɹ“WWDCͷ๲େͳηογϣϯϏσΦΛޮ཰Α͘ফԽ͢ΔΞΠσΞΛ͍ͩ͘͞ʂ" ) } Foundation Models framework 🤖💭 “ޮ཰Α͘WWDCͷηογϣϯϏσΦΛফԽ͢Δํ๏͸͍͔ͭ͋͘Γ·͢ɻ·ͣɺڵ ຯͷ͋ΔτϐοΫΛࣄલʹௐ΂ͯɺಈըͷλΠτϧ΍಺༰ΛνΣοΫ͢Δͱྑ͍Ͱ ͠ΐ͏ɻ࣍ʹɺಈըΛ1ͭͷηογϣϯͰ·ͱΊΔಈը΍ཁ໿ಈըΛ୳͢ͷ΋༗ޮͰ ͢ɻ·ͨɺཁ఺Λ·ͱΊࠐΜͩεϥΠυ΍هࣄΛಡΉͱɺཧղ͕ਂ·Γ΍͍͢Ͱ͢ɻ ࠷ޙʹɺࣗݾ൓ল͠ͳ͕Βɺ࣍ͷΞοϓσʔτʹ޲͚ͯ४උΛ੔͑ͯΈ·͠ΐ͏ɻ”
7. • ߏ଄Խ͞Εͨग़ྗΛอূ͢Δػೳ • SwiftͷܕγεςϜΛ׆༻ͯࣗ͠વݴޠϓϩϯϓτΛิڧ • @GenerableɿϞσϧʹੜ੒ͯ͠ཉ͍͠ܕΛఆٛ • @GuideɿϓϩύςΟʹઆ໌Λ෇༩͠ɺੜ੒஋Λ੍ޚ͢Δ • ϓϦϛςΟϒܕɺ഑ྻɺGenerableܕɺ࠶ؼతͳߏ଄΋αϙʔτ Guided generation
8. @Generable(description: "ΦϒδΣΫτͷઆ໌Ͱ͢") struct ObjectDescription { @Guide(description: “આ໌จͷ഑ྻ", .maximumCount(3)) var descriptions: [String] } —- let session = LanguageModelSession( instructions: “͋ͳͨ͸ɺ༩͑ΒΕͨΦϒδΣΫτͷ໊લΛ΋ͱʹɺ ͦͷΦϒδΣΫτͷҙ֎ͳ౾஌ࣝΛڭ͑ͯ͘ΕΔത࢜Ͱ͢ɻ” ) ΦϒδΣΫτͷ໊લ͔Βɺ؆୯ͳઆ໌จΛ࠷େ̏ͭग़ྗ͍ͤͨ͞
9. @Guided ͰࢦఆՄೳͳ੍໿ \w+/
10. @Generable(description: "ΦϒδΣΫτͷઆ໌Ͱ͢") struct ObjectDescription { @Guide(description: “આ໌จͷ഑ྻ”, .maximumCount(3)) var descriptions: [String] } —- let session = LanguageModelSession( instructions: “͋ͳͨ͸ɺ༩͑ΒΕͨΦϒδΣΫτͷ໊લΛ΋ͱʹɺ ɹɹɹɹɹɹɹͦͷΦϒδΣΫτͷҙ֎ͳ౾஌ࣝΛڭ͑ͯ͘ΕΔത࢜Ͱ͢ɻ” )
11. Snapshot streaming { greenting _to : _Cup τʔΫϯ : … let stream = session.streamResponse( to: text, generating: Message.self) for try await partial in stream { print(partial) // Snapshot streaming Message.PartiallyGenerated(greeting: nil) Message.PartiallyGenerated(greeting: "") Message.PartiallyGenerated(greeting: "Welcome") Message.PartiallyGenerated(greeting: "Welcome to") Message.PartiallyGenerated(greeting: "Welcome to Cup") }
12. // Expanded @Generable Macro nonisolated struct PartiallyGenerated: Identifiable, ConvertibleFromGeneratedContent { var id: GenerationID var descriptions: [String].PartiallyGenerated? nonisolated init(_ content: FoundationModels.GeneratedContent) throws { self.id = content.id ?? GenerationID() self.descriptions = try content.value(forProperty: "descriptions") } } @Generable ϚΫϩʹΑΓࣗಈੜ੒͞ΕΔ PartiallyGenerated
13. Tool calling • ֎෦৘ใݯͱͷ౷߹ɺϞσϧ͕ΞϓϦ಺ͷίʔυ(Tool)Λ࣮ߦՄೳ • ֎෦৘ใݯͱͷ΍ΓऔΓΛ࣮૷ʢe.g. δΦίʔυ͔ΒॅॴɺҐஔ৘ใ͔Βఱؾʣ • πʔϧͷઆ໌จʢdescriptionʣΛఆٛ͢Δ͜ͱͰɺϞσϧ͕πʔϧΛࣗ཯ తʹݺͼग़͠ɺ࠷ऴϨεϙϯεʹ૊ΈࠐΉ • ಈతੜ੒εΩʔϚʹΑΓɺμΠφϛοΫͳఆ͕ٛՄೳʢDeep dive in… Ͱઆ໌ʣ
14. import FoundationModels struct GetWeatherTool: Tool { let name = "getWeather" let description = "Retrieve the latest weather information for a city" @Generable struct Arguments { @Guide(description: "The city to fetch the weather for") var city: String } func call(arguments: Arguments) async throws -> ToolOutput { let places = try await CLGeocoder().geocodeAddressString(arguments.city) let weather = try await WeatherService.shared.weather(for: places.first!.location!) let temperature = weather.currentWeather.temperature.value let content = GeneratedContent(properties: ["temperature": temperature]) let output = ToolOutput(content) return output } } … let session = LanguageModelSession( tools: [GetWeatherTool()], instructions: "Help the user with weather forecasts." )
15. Stateful session • LanguageModelSession ͸ঢ়ଶΛอ࣋ • Ϟσϧͱͷର࿩͸ίϯςΩετʹอ࣋͞ΕΔ • ηογϣϯ಺ͷաڈͷ΍ΓͱΓΛࢀরɺཧղͰ͖Δ • ͨͩ͠ɺίϯςΫετͷ্ݶ͸ 4096 τʔΫϯ • ௒ա࣌͸৽͍͠ηογϣϯΛ࡞੒͠Ҿ͖ܧ͙ඞཁ͕͋Δʢʣ • session.transcript Ͱաڈͷ΍ΓऔΓΛग़ྗՄೳ
16. ؔ࿈ηογϣϯ - Deep dive into the Foundation Models framework - Code-along: Bring on-device AI to your app using the Foundation Models framework - Explore prompt design & safety for on-device foundation models - Explore large language models on Apple silicon with MLX
 ↑ Foundation Models Ͱ͸ͳ͘ɺΦϯσόΠεLLM ͰࣗલσʔλΛ௥Ճֶश͍ͤͨ͞έʔε
17. Meet the Foundation Models framework Takeshi Tanaka (@p0dee) ू·ΕSwift޷͖ʂSwiftѪ޷ձεϐϯΦϑ WWDC25ηογϣϯཁ໿ձ @ DeNA 2025/06/26