What If Your App Could Debug Itself?

Meet try! Swift@CommunityKit
https://communitykit.social

Transcript

1. akkey What If Your App Could Debug Itself? Meet try! Swift@CommunityKit What If Your App Could Debug Itself?
 Today, I want to share a real use case of Foundation Models.
 Thank you for joining me.
2. try! Swift Tokyo Organizer akkey Me! My name is Akkey. I’m an organizer of try! Swift Tokyo. This is one of my favorite photos.
 It was taken when Apple Vision Pro launched in Japan. Many people say, “Wow, that’s amazing!” This photo was even used in an Apple press release. But actually, I’m just a customer.
3. Please report bugs after the session 🧪Experimental English Support You may have noticed already… My English support is still experimental. If you fi nd any bugs, please report them after the session. And I would really appreciate your support today.
4. Please report bugs after the session 🧪Experimental English Support Wait… what did they say? 💦 By the way, this photo was taken during my fi rst Apple Vision Pro demo in Hawaii. I was so nervous speaking English. I even got sweat on a brand-new Apple Vision Pro.
5. My app
6. ɾɾɾ ɾɾɾ First, let me introduce my app. There are two apps. One runs on Apple Vision Pro. The other runs on Mac. They work together.
7. What is that?
8. What is that? AVFoundation Persona Persona Viewer Multipeer Persona Anywhere What does this app do? Do you know Persona on Apple Vision Pro?
 In simple words, it’s your avatar.
9. What is that? AVFoundation Persona Persona Viewer Multipeer Connectivity
 Framework Persona Anywhere Normally, Persona only works inside visionOS apps. My app brings Persona to the Mac.
 This is useful when using Mac Virtual Display.
10. Store Reviews But this app received many harsh reviews.
11. One star.
12. App never connects to my Mac :( I’m running the Persona Viewer on my Mac and enabled the virtual camera as instructed. I’m not running any extra security software and they are on the same network but I never get the option to connect. “I followed the instructions, but it doesn’t work.”
13. One star.
14. This app worked before sequoia which led to the positive reviews. But since then, local network access and communication between Mac/AVP has broken. “It stopped working after an OS update.”
15. One star.
16. This app is broken on Mac - doesn’t ask for permission to use camera, thus can not be added through Privacy settings. Either fi x it or just delete it from App Store “It should be removed from the App Store.”
17. This app is broken on Mac - doesn’t ask for permission to use camera, thus can not be added through Privacy settings. Either fi x it or just delete it from App Store 😭 These issues are very hard to investigate.
 And honestly, I don’t enjoy reading reviews like that.
18. Let "Foundation Models" Debug Your App So I built a new feature with Foundation Models.
19. The app can help debug itself. It looks like this.
 When a problem happens, the app suggests what the user should try. All of this advice is generated by Foundation Models.
20. 1. Collect Diagnostic Logs import OSLog let logger = Logger( subsystem: “com.akkeylab.xxx", category: “networking", ) logger.info("Disconnected from session") Let’s look at the implementation.
 First, record logs when errors happen.
 Many of you already do this.
21. 2. Analyze the Issue let store = try! OSLogStore( scope: .currentProcessIdentifier ) let position = store.position( timeIntervalSinceLatestBoot: TimeInterval(-60) ) let entries = try! store.getEntries(at: position) let logText = entries .compactMap { $0 as? OSLogEntryLog } .prefix(10) .map { log -> String in "\(log.category):\(log.composedMessage)" } .joined(separator: "\n") Next, collect those logs.
 This code may look complex.
 But it mostly converts logs into text for a prompt.
22. 3. Generate User-Friendly Guidance import FoundationModels let prompt = “ ... \(logText) ...” let instructions = Instructions(" ... ") let session = LanguageModelSession( instructions: instructions ) let response = try? await session.respond( to: prompt, generating: Advice.self ) let content = response?.content Finally, use the logs and other information to generate user-friendly guidance. This is where Foundation Models are used.
 I skipped some details here. The prompt includes many small tricks and improvements.
23. After that, I simply display the result with SwiftUI.
24. We're Seeing More Positive Reviews So, what happened?
25. Maybe it’s not only because of this feature…
 But we are getting more positive reviews now.
26. Thank you !! Thank you.