Concurrency in Mopidy

Short presentation of how we solved concurrency in Mopidy. This is the precursor to what later became the Pykka actor library for Python. Presented at the Iterate Weekly Breakfast Meeting 2010-09-01

Transcript

1. Mopidy Concurrency in
2. Mopidy is If you speak enterprisey: a music integration platform
3. Mopidy is A program that connects all your music stuff, so that you can use it from any computer or smartphone
4. Six months has passed
5. Changes MPD server From 40% to 80% complete Now works 98% with most MPD clients Spotify integration From 2x 50% complete to 1x 80% complete We've dropped despotify in favour of libspotify
6. Changes License From GPLv2 to Apache License 2.0 You are now free to build anything on top of Mopidy Releases From 0 to 5 releases 0.1.0 released August 23 0.1.1 and 0.2.0 in the works
7. Changes Developers From 0 to 3 active contributors Users From 0 to 5 active users reporting bugs About 400 downloads from PyPI
8. Changes Code More than 1100 new commits From 2700 to 6200 lines, including tests Tests From 200 tests in 0.4s to 550 tests in 0.8s 55% of the code is tests
9. Changes Design We still create the design as we walk Refactoring early and often with the confidence provided by the tests
10. The result
11. WARNING The following script actually works
12. 1.Slide finger on Android or iPhone
13. 1.Slide finger on Android or iPhone 2.Watch the volume knob on your NAD or Denon amplifier turn
14. 1.Slide finger on Android or iPhone 2.Watch the volume knob on your NAD or Denon amplifier turn 3.Hear the sound of your music, either from Spotify or local storage
15. Concurrency
16. We need to respond timely to clients, get data from Spotify, play sound, turn physical volume knobs through serial links, all at the same time
17. Concepts Client | Frontend | Backend /------+-------\ | | | Spotify Output Mixer | | GStreamer ...
18. Multiple processes respond timely to clients (frontend), get data from Spotify (backend) play sound (output), turn physical volume knobs through serial links (mixer), all at the same time
19. Communication Generally, parallell processing is known to be hard to get right, and harder to debug. These processes are not independent, but need to communicate. How do we manage this?
20. Solution No shared state Communication through messaging
21. Solution Actors reacting to messages Multiple processes/threads (actors) that are working together by sending serializable messages to each other
22. Messages The messages are Python dicts {'command': 'play_uri', 'uri': uri}
23. Messages A message is a command with attached data Think of it like a method call with arguments, which can be serialized and passed around
24. Actors The message reactors are if/else statements (for now) def process_message(self, message): if message.get('to') == 'output': self.output.process_message(message) elif message.get('to') == 'frontend': for frontend in self.frontends: frontend.process_message(message) elif message['command'] == 'end_of_track': self.backends.playback.on_end_of_track() ...
25. Loose coupling Ignorance is bliss Most of our code is ignorant of the concurrency. We call regular methods, which do the multiprocessing work.
26. Loose coupling If we wanted to, we could replace said methods, and make the processes run on different machines Communicating over the network Nice, eh?
27. Next release(s) Last.fm scrobbling Submit your played tracks to Last.fm Multi-backend Tracks from Spotify and local storage in the same playlist OS X Support intentionally broken in 0.1.0
28. Later WIMP backend DNLA/UPnP frontend/backend REST WS frontend with JavaScript client Audio streaming over HTTP
29. www.mopidy.com
30. @jodal & @knutz3n, 2010-09-01