Building a Karaoke Machine with Angular and HTML5 Speech Recognition

Learn how I built a karaoke machine using Angular, the HTML5 Web Speech API, and RxJS. The app lets you select songs, sync lyrics, and grade singers as they sing along.

Transcript

1. Web Karaoke BUILDING A KARAOKE MACHINE FOR THE BROWSER
2. Web Karaoke App
3. Sergio
 Cruz @hashtagserg ! About Me • Instructor • Software Consultant • Sr. Karaoke Advocate ORLANDO, FL
4. Karaoke Machines ORIGIN STORY
5. How The Karaoke Was Invented http://content.time.com/time/world/article/0,8599,2054546,00.html • Live music scene in Japan in 1970’s was very popular • A mistake led Daisuke Inoue to invent what we today know as Karaoke
6. The 80’s Came Along • When innovation took off • Karaoke machine with lyrics was invented • Machines started calculating karaoke scores
7. Web Karaoke HOW I BUILT IT
8. 1. Synced audio with song lyrics 2. Interpret text from speech 3. Combined both to calculate scores Web Karaoke 3 Steps I Took To Build It
9. Component Architecture Review PlayerComponent Lyrics
 Component Audio
 Component Speech
 Component CONTAINER C C C
10. 1. Syncing Music & Lyrics 0:37 I just wanna tell you how
 I’m feeling 0:39 Gotta make you understand 0:43 Never gonna give you up
11. File Format Lyrics File Format: .lrc There were 2 reasons why LRC was awesome for this [ti:Never gonna give you up] [ar:Rick Astley] [00:02]Never Gonna Give You Up [00:16]Rick Astley [00:19]We're no strangers to love [00:23]You know the rules and so do I
12. Reason 1: Easy To Work With { ti: "Never gonna give you up", ar: "Rick Astley", lines: [ { time: 19, text: "We're no strangers to love" }, { time: 23, text: "You know the rules and so do I" }, { time: 27, text: "A full commitment's what I'm thinking of" } ] }
13. Reason 2: This Person’s GitHub • Bulleted list • Bulleted list • Bulleted list • Bulleted list • Bulleted list
14. Syncing
 Lyrics const audio = new Audio(src); Observable.fromEvent(audio, 'timeupdate') .subscribe((currentSecond) => { // find current lyric line // in lrc json }); Syncing Audio with LRC
15. @Component({ /* ... */ }) export class AudioComponent implements OnInit { @Output() onTimeUpdate = new EventEmitter<number>(); @Input() src: string; private audio: Audio; public currentTime = '00:00'; ngOnInit() { this.audio = new Audio(this.src); Observable.fromEvent(this.audio, 'timeupdate') .subscribe((currentSecond) => { this.onTimeUpdate.emit(currentSecond); this.currentTime = formatTime(currentSecond); }); } } 00:37
16. Component Architecture Review PlayerComponent Lyrics
 Component Audio
 Component CONTAINER C C Speech
 Component C
17. Interpreting Text From Speech The HTML5 Web Speech Recognition API is a great solution for this Very easy to use — check it out!
18. Speech Recognition const recognition = new SpeechRecognition(); Observable.fromEvent(recognition, 'result') .subscribe((e) => { console.log(e.results[0][0].transcript); }); recognition.start(); Using The Recognition API Constructor may vary depending on browser used
 ie.: new webkitSpeechRecognition()
19. @Component({ /* ... */ }) export class SpeechComponent implements OnInit { @Output() onSpeechFound = new EventEmitter<number>(); private recognition: SpeechRecognition; ngOnInit() { this.recognition = new SpeechRecognition(); Observable.fromEvent(this.recognition, 'result') .subscribe((event) => { const { transcript } = event.results[0][0]; this.onSpeechFound.emit(transcript); }); } } Send latest transcript to parent container
20. Data Is Emitted Back To Container PlayerComponent Current
 Lyric Line Recognized
 Speech CONTAINER Gives us enough info to calculate score
21. How Scoring Is Calculated • We get the transcript from Recognition API • Match that transcript to previous few lyric lines • Give points for each word that was matched Let’s take a better look at the demo!
22. Demo Time!
23. Sergio
 Cruz @hashtagserg ! Thank you! https://sergiocruz.github.io/ng-karaoke ORLANDO, FL Check out the live app: