Voice to Intelligence

A presentation about SpeechAnalyzer and Foundation Models with examples and best practices.

Transcript

1. Krishna Venkatramani 16 June 2026 Voice Transcription & Foundation Models: from Speech To Intelligence
2. Chapter 1: Speech Analyzer
3. What is SpeechAnalyzer? • New API in iOS 26, iPadOS 26, macOS 26, tvOS 26, visionOS 26 • Great for: Long-form audio , word-by-word transcription • 100% on-device — audio never leaves the device • No Siri or Dictation settings required • Built for Swift concurrency: async/await + AsyncSequence
4. Two Key Pieces • SpeechAnalyzer = Session manager • SpeechTranscriber = On-device speech-to-text module
5. Volatile vs. Finalized Results • Volatile: fast, rough guesses (result.isFinal == false) • Show immediately with low opacity • Gives users ‘I am being heard’ feedback • Finalized: accurate, permanent text (result.isFinal == true) • Non-overlapping, safe to persist • Delivered in CMTimeRange segments without overlap
6. Transcribing a File • Create SpeechTranscriber(locale, preset: .o ffl ineTranscription) • Collect results concurrently with async let and reduce • Initialize SpeechAnalyzer and feed audio fi le • Finalize or cancel analyzer session based on last sample • Await collected string — pure async/await, no callbacks
7. Word-Level Timestamps • Enable with attributeOptions: [.audioTimeRange] • Each word carries a CMTimeRange attribute • AttributedString integrates natively — no index math
8. Chapter 2: Foundation Models Framework
9. What Is the Foundation Models Framework? • Direct access to the on-device large language model powering Apple Intelligence. • Optimized for content generation, summarization, and extraction
10. Guided Generation — Structured Output • Constrained decoding prevents invalid output • De fi ne schemas with @Generable and @Guide • Guarantees structural correctness and improves accuracy
11. Snapshot Streaming • Streams valid partial struct states instead of token di ff s • Each @Generable type has a PartiallyGenerated mirror • SwiftUI binds snapshots for smooth progressive updates
12. Tool Calling • Model autonomously invokes on-device functions • Arguments validated via @Generable for safety • Tool outputs re-injected into transcript seamlessly
13. Prompt Design Best Practices • Use clear, single-command prompts to control output • Provide fewer than fi ve examples or one-shot inputs • Leverage PromptBuilder and "DO NOT" to suppress unwanted text
14. Performance & Optimization • Prewarm the model on user intent for instant responses • Exclude schemas from prompts to reduce token count • Use greedy sampling to ensure consistent tool calls
15. Demo Voice + Foundation Models
16. Architecture.
17. Q&A