PyConZA 2015: Transcript of "Kivy Showcase: a short exploration of how Kivy is changing the world" by Richard Larkin

In this talk, we will look at some real-world success stories using Kivy. From exhibition-size interactive displays to race car telemetry systems to robotics, Kivy is being used to deliver many compelling and innovative applications. We'll use these applications to discuss some of the features and abilities of the framework that make it such a good choice for a wide range of uses.

We'll briefly cover some exciting projects using Kivy, namely:

* Project Liatris: a new, open source project using Kivy touch devices to control robots.

* RaceCapture Pro: A race car telemetry system that uses Kivy interactive displays both in-car and in post-session analytics interaction.

* Brain Trainer Plus: A mental dexterity trainer deployed to old age homes and medical treatment facilities that has been shown effective in treating dementia and other mental disorders, as well as bringing a greatly enhanced quality-of-life to the aged and mentally ill.

* CAMI Educational products: The rich feature set and effortless animation abilities of Kivy make developing compelling, responsive desktop and mobile interactions easy.

* The Icarus touch and Touch live projects: creating radically new and exciting interactive musical instruments.

All of these applications are being delivered using Kivy, a free, open source, full stack, multi-touch, cross-platform Python/Cython framework that runs on a watch. I kid you not.

Transcript

1. 1. Museum of Natural History, Lille.mp4 So, lets start with a bit of history. This is a project called 'Table Atomineral Touch', which was installed at the Museum of Natural History in Lille, France in 2011. This project is part PyMT, part Kivy. So, lets explain that. In the beginning there was PyMT, and it was good. It was multi-touch, pretty, and Pythonic. But it was sub-optimal. The developers saw there was a better way to do things, but that it would mean breaking compatibility. Thus was born Kivy. All the experience and learning gained from PyMT was rolled into Kivy as the next-generation framework. In addition, the hardware requirements were fixed at OpenGL ES 2, so mobile devices could be easily accommodated. This helps to explain my Kivy is both fresh and exciting, yet mature and stable. It inherits a rich legacy of multi-touch and large-scale application learning, yet has mobile and lightweight processors squarely in it's aim. It the best of both worlds: scalable, portable and fast. Another take away from this project is that Kivy isn't just touch. That why we can it a Novel User Interface. It's different, it's unique, it's whatever you want it to be. Kivy widgets use lightweight Mixin class to define behavior. It de-couples the presentation of widgets from their appearance, and makes it easy to create your own widgets. You can define you own look, then just mixin the behaviours you want e.g. the ButtonBehaviour class to get on_press and on_release events. This user input here consists of discs which you can twist and turn. You can associate elements by moving discs closer. It's more than touch, and this flexibility arises because Kivy uses an abstraction called an InputProvider to isolate input dependencies. By using such an abstraction, it allows for a wide range of hardware and devices to use used as input yet isolates these differences from your code. But we'll return to this subject later.
2. 2. Plane White by Christina Ow This is project that was presented in Geneva, Switzerland by Christina Ow as a student in the Master of Arts in Design in 2011. The artwork intends to explore and challenge the idea of exhibition spaces being white cubes, and introducing, volatile digital spaces. This particular project was developed over a period of 10 days by a team totally unfamiliar with Kivy. No previous Kivy experience, but they did this in 10 days. So, this piece not only gives one a good idea of the scale of applications possible with Kivy, but also how rapidly then can be developed. As you can see, this is a massive display. I could not get hold of the technical details of the hardware involved, in terms of processors and screen size, but it's clearly very big. And it got flashing lights and blankies and stuff. That's pretty cool. So, that's one thing to keep in mind. These kind of projects are old hat for Kivy. It's been doing large scale apps and multi-user, multi-touch input for a long time.
3. But lets move on. How about some particle physics? 3. Particle Panda by Chaos Buffalo Labs Particle panda is a particle effect simulator for particle systems. It allows you to develop custom animations for games, artistic expression or just because you're bored. The animations are created with particle panda can be exported to “.pex” files, video files compatible with the 'Starling', Flash, 'Cocos2d' and Kivy engines. Using the interactive GUI, particle effects can be changed and edited in real time. Notice how even during rendering loads, the app remains responsive. It also showcases a lot of standard kivy widgets. Sliders, buttons, tabbed panels, scrollable list views, colour pickers etc. Particle physics aside, this gives and good indication of the look and feel and a standard Kivy app. Of course, this can all be changed, but at least it's professional and modern by default. The look and feel of Kivy widget are based on the Moblin mobile spec from the Maemo and Meego Operating Systems, but is fully customizable. We'll see some concrete examples shortly.
4. So, Kivy in education. 4. CAMI Speedtest by CAMI Education This is a quick preview of CAMI's next generation education platform built using Kivy. Now, the first thing to notice here is that although it looks totally different, it's actually mostly a matter of artwork. Kivy widgets are built to be flexible and theme-able, and most of what you see here are standard Kivy widgets with the image properties changed. 'button.background_down = x'. button.background_normal = y' etc. It's trivial to integrate you own look and feel into the pre-rolled toolkit. Another thing you'll see here for the first time is Screens. Kivy apps run in a single Windows. Pretty natural for mobile, but one might think limited for desktop. Not so. The concept of multiple forms or windows is simply replaced by multiple screens. And they're much sexier. Notice the fluid transitions between them. That's for free, and changing the transition is as simple a specifying another transition class. SlideTransition, SwapTransition, FadeTransition, WipeTransition and more. Or roll you own. Up to you. Popups? Yes. Modal popups? Yes. It's all possible, and easy to use. The last thing I want to point out here is the use of custom keyboards. You'll see in the exercises here we present a small, movable and re-sizable keyboard. Even where words are requested, we try to present a list of options, either of words or letters, rather than have the keyboard swallow half of the screen. We do this in order to try and maximize the screen space for content. A keyboard taking half of the screen is often overkill and can really impede interaction with the bottom half of the screen. Predictably enough, this is one of the only areas we've had to care about desktop vs. mobile. But it's an inherent difference, and one that is easily handled in Kivy. Software and hardware keyboards are handled virtually identically, so catering for these different keyboard scenario's is simple. Also, you can use the Kivy config module to force a software keyboard on a desktop, meaning you don't have to run it on mobile to test. You can simulate the software keyboard on a desktop for testing, and then just do your final tests on the actual touch device. That really saves time. Oh, one more thing. These animations and non-blocking url fetches happen thanks to the asynchronous UrlRequest function, and these image downloads happen asynchronously thanks to the AsyncImage widget. These kinds of built in classes make it very easy to develop interfaces which respond instantly even whilst requesting and waiting for non-local resources. It really helps and makes to easy to keep the app responsive.
5. So, lets look at another app that demonstrates Kivy's responsiveness. 5. This is a small project developed using Flat Kivy. Flat Kivy is a collection of experimental flat/material design widgets for Kivy. The are inspired by Google's Material Design guidelines and make use of binding to implement what web-devs would call a 'responsive design', where the UI adapts naturally to screen resolution and orientation changes. Notice how the text resizes and re-flows instantly when the window size changes. At almost any size and orientation, it remains clear and readable. So, you can use these widgets instead of the standard ones. You'll see from the pops that they also implement the 'ripple effect' around your touch, thus making it clear on a mobile exactly where you have touched. What we also can learn here is that Kivy does a great job, even in Line-Of- Business apps. This is an very functional, practical and utilitarian tool. It's fast, lightweight and native. Another interesting aspect of this project is it's author, Jacob Kovak. He is the resident OpenGL guru for Kivy and also develops a highly-performant gaming engine for building Kivy games called KivEnt. So, to explain that, Kivy widgets are built for interaction and sub-optimal when you start talking hundreds of sprites. Kivent gives you a way to handle many more sprites in a highly optimized way. It means Kivy can extend to much more intensive gaming scenario's whilst still maintaining it's ease-of-use. What Kovak seems to do is spend his spare time profiling various scenario's and then rewriting the slow parts in Cython to speed up the bottlenecks so they run at C speeds. These optimizations allow KivEnt to handle roughly ~100,000 game objects and we have seen speedups of 5 to 50 times in some cases. He also helps optimize the Kivy widgets if he discovers more optimal ways of doing things. Keep in mind though, standard Kivy is perfectly suitable for a vast amount of games. It's just that KivEnt allows you to scale up if performance does become a problem.
6. Okay, so onto more fun stuff. Robotics. 6. This is a project called TicTacTouch by Dominique Burnand and Livio Conzett. It consists of a multitouch screen used to control a PLC hardware machine via Libnodave. Libnodave is a library that provides functions to connect to and exchange data with Siemens S7 300/400 PLCs. It exposes a C interface, as does almost any pluggable hardware, and that makes it very easy for Kivy to talk to. Being Cython/Python based, this kind of low-level conversation between Kivy and hardware devices is relatively straight forward. So, as you can see, we are playing tic-tac-toe with a robot. And why move the pieces if a bot can? It's totally pointless yet profound, all at the same time because it hints at some of the possibilities that arise when integrating touch screens into the world around us.
7. 7. Enter Liatris. Liatris is an open-source project that determines object’s exact positions and identities using a touch screen and RFID reader. The beauty of this solution is cost and sophistication. To elaborate, one of the major challenges in robotics is visual analysis. It's a hard problem. Determining what an object is, how it's positioned and how to use purely from camera images typically requires loads of processing time and learning. So Mark Silliman has has devised an approach which vastly reduces the amount of visual processing required. Appliances or objects are fitted with an RFID tag which contains a 3D model of that object, along with data describing how the object can be manipulated. The bottom of the object is fitted with triangular markers that, when placed on a touch screen, allow the robot to determine the exact orientation of the object. This orientation, together with the information obtained from the RFID tag, gives the robot all the information it needs to build a 3D model and manipulate and the object in question. It can work with multiple object on a screen and multiple screens. It's an open spec, so manufacturers can freely define new tags and objects that the robot can recognize and interact with. Project Liatris is free and open project the could significantly lower the cost and increase the usefulness of everyday robotics, and it's doing it using using Kivy.
8. Okay, so robots are cool, but how about race cars? 08. Race Capture Pro by Autosport labs is a powerful, expandable and customizable lap timer, data logger and telemetry system for your race or street car. It allows you to plug a mobile device or computer into your car and get detailed telemetry readings in real time. You see here the hardware device that gathers the data. The gps plugs in at the right and the bluetooth transmitter on the left. It has an SD card reader and a micro-usb port. Kivy talks to the device over bluetooth and presents a fully customizable, real-time readout of this data. RPM, oil temperature, oil pressure, engine speed, acceleration, yaw, accelerometer, wheel pressure or temperature. Autosport labs provide a vast range of sensors you can choose and fit as you will. This all gets fed in the RaceCpature pro unit from which Kivy pulls all this data. You then use the app to choose how and where the visualizations for the sensors are presented. We don't have time to go into the details of this app, but it's really incredibly impressive. One of the things I do want to mention, however, it that it uses Kivy's newly introduced support for SVG in version 1.9.0. Brent Picasso, one of the developers, was experimenting with ways to increase the speed of the gauge rendering. These sensors typically have a sample rate in excess of 1000hz. That's 1000 samples per second, so having a display with fast-reaction time is vital. It also requires that each gauge have 'thresholds'. As the limit of a gauge is reached, it should turn yellow for warning, red for danger etc. Anyway, what he discovered was that SVG's have a “current colour” property, which effectively determines the default colour used for drawing. He then added a binding so that when these limits were reached and the current_color changed, this would trigger a re-parsing of the SVG tree. So, effectively, as SVG is just an XML node structure, changing the node value of the SVG object automatically updates the rendering, and voilla, colour changed. The response time is virtually instantaneous. In a testimony to the power of open source, Brent submitted pull request, number #3518 for this feature, and it was merged by Tito himself on the 20th of July. Now we can all enjoy these dynamically updating SVG displays in the next release of Kivy.
9. From cars to aeroplanes. 10. Flightgear TQ panel is application by d-laser that provides you with touch panel controllers for the Open Source Flight Gear aircraft simulator. It provides you with throttle controls, speed brakes, elevators and trim controls, flaps, field of view (zoom), parking brake and more, all from the comfort of your phone, tablet or PC. Apart from being ridiculously cool, one of the interesting things was that his initial prototype was just 80 lines of kivy code. So, Flightgear supports the idea of 'generic protocols defined by XML files. You define you own protocol and command support in the XML file, drop it into a specific location and Flightgear then implements the protocol, sending and receiving packets as specified. Here, it works over the UDP protocol. The video may not be the best quality, but come on, we are flying planes using Kivy! Of course, there are many other games that you could build your own, custom controller for. And it requires almost 0 investment. You can use your phone or tablet starting creating any type of touch controller you want.
10. And the rasberry pi. 11. This is small demo by Mark Richardson of Kivy using the Official Rasberry Pi touch screen. Here, he makes it control and respond via to a few LED's and a button using the GPIO pins on the Pi. Control and feedback is bi-directional, with kivy buttons switching the lights on and off, yet also responding to state changes on the board. So, GPIO stands for General Purpose Input and Output devices, and as you probably already know, the is a large range of such devices that you can plug into the py. That means you can control and visualize all of these devices on your Pi using Kivy. If your Pi is connected to a network, this can extend beyond local devices into an Internet of Things. You can do such things as monitor air-quality, camera's, sensors, control quad-copters, drones, and of course, robots. Think home automation. Think device control. The low cost of the raspberry Pi opens all of these possibilities up to amatuers and tinkerers, and Kivy gives us a rich, open and mature software stack to achieve it with.
11. So, whilst on the subject on controllers, we hinted earlier that Kivy is much more than just a multi-touch framework. 12. Here we see the amazing Mathieu Virbil demonstrating his ninja skills via the Leap Motion. Mathieu, also known to us lower mortals at tito, is kind of the Benevolent Dictator For Life of Kivy. He fathered Kivy and seems to produce amazing libraries as easily as most of us produce body odours. Anyway, Kivy provides full support for 3-D input, currently via the Leap Motion or Kinect devices. On the screen, you can see each of his fingers are represented as independent points with varying colours and brightness depending on the X, Y and Z positions and it picks these up instantly. Kivy's uses an InputProvider abstraction to support a vast array of input devices and of course, you can add your own. Luckily, they are all handled in a very generic way, which means you don't have to care about them unless you want to. They all just provide a MotionObject, which is a rich and detailed object providing a your with a complete profile of both the motion event and the device that provided it. But that's fine. Your code does not have to care. You can just use the x and y like a mouse if you want to.
12. Okay, let's touch on the 'full-stack framework' claim. 13. Here we see the Kivy inspector, one of a few a few debugging tools provided with Kivy. To use the inspector, you simply launch you main python script with a '-m inspector' argument. It can then be activated and any time via a keystroke and allows you to inspect you entire GUI widget hierarchy in real-time. Making full use of Python's powerful introspective abilities, it allows you traverse child widgets and parent widgets, as well as change their properties on the fly. We see this kind of thing in Visual Studio, Eclipse and other IDE's, but I have yet see one able to inspect and modify a live, running interface. Just look at some of the things that are happening here. A full, editable listing of the objects properties. Changing them then and there and have it reflect instantly. It's an incredibly powerful debugging tool. Not sure why a widget is placed there? Behaving that way? Just activate the inspector, look at it's properties and even try changing them. That's possible at any point in your live, running application. This is just one of the tools Kivy also provide. There is a webdebugger, basically a pretty c-profiler served to your webbrowser via an embedded Flask server. Then there is also the monitor module, which adds a small topbar showing FPS and input activity, the keybinding module, the recorder (for the recording and playback a sequence of events) and the screen module, for simulating various screen resolutions and DPI settings. And these modules just really cover the debugging side. There is no time to go into details, but Kivy provides a host of other modules and classes that cover many aspects of development cycle. It's very rare that you ever need to look outside of the framework supporting tools. If Python provides the batteries, Kivy brings in the nuclear power plant. Just without all the death and radiation and stuff.
13. Now onto on of my favourites. With Kivy, because it's so easy to design GUI's quickly without knowing a stitch of OpenGL, it's easy to forget you are actually building 3-D applications. 14. Kivy 3-D GUI is a project by Karel Piorno Charchabal and it reminds us that Kivy is in a 3-D world. Here, we see a bunch of stock standard Kivy widgets running on a surface suspended in a 3-D environment. Notice that he's still interacting seamlessly with all these widgets. Sliders, buttons, video players, textboxes wrapped around balls, Restructured text widgets. Scrollviews. All of these widgets respond as fluently and easily as if they were on a flat surface. Their physics and behaviour are all perfectly maintained. What he's done here is just provided us with another layout, just like a box layout, or a grid layout or any of the numerous Kivy layout types, but this one handles the 3rd dimension. Because OpenGL supports the loading of wavefront 'obj' files, we can also load 3-D object models and meshes into this Kivy scene and use the objects and surfaces from our application. Blender, Maya, Google Sketup, most decent 3-D programs support exporting to obj files, so you can use these tools to design your objects. At the end of the video, we see kivy image widgets being dragged over the floors, walls and roof of the building. It's perhaps not revolutionary in itself, but what is revolutionary is how simply this is. That we can start with a simple, 2D gui then, but just by dropping it in 3-D layout, take our app into populated a 3D space, interacting with 3D objects built in blender, all without knowing anything about 3D graphics languages. I not sure how I would actually use this, but the 5 year old inside me wants to use it anyway. It's just ridiculously cool. It's as if apps become toys in a parallel, cyber universe that we can play in. And by doing that, it totally blurs the divide between apps and games. Work and play. Art and machine.
14. Okay, so we've seen Kivy with massive screens and driving impressive hardware. 15. Here we see Gabriel Pettier, known affectionately to us minions as tshirtman, demonstrating the game 2048 of his Galaxy gear watch. Yes, the labels are too big, and yes, it's awkward to move tiny tiles on a phone, but it works. And it's fluid. It's responsive. And it's a watch, dammit! It runs a 1Ghz processor with 512 megs of RAM, running Android Wear, a lightweight Android distribution built for wearable devices, and it runs Kivy easily. So, let's think about this. We've seen Kivy educating children, moving robots, helping the aged and mentally ill. We've flown aircraft simulators, held art exhibitions, raced cars, handled ninja chops in 3-D, we've made lights twinkle using our raspberry pi, simulated particle physics and swiped our widgets over 3-D monuments. Sorry, but if you don't think that's awesome, you're either dead, boring or probably both.
15. So, I'm going to shut-up now and end this presentation with one of my favourite Kivy projects. 16. It's called the Icarus Touch and it's a continuous keyboard written by Cyril Stoller of the band 'Liquid Rain'. I think it demonstrates the creativity and beauty that Kivy enables better than my ranting. Before I go, I have to offer up one massive lump of goey thanks to all the Kivy devs. Tito, Tshirtman, Qua-non, Ben Roush, Kovak, Matham. Kived. Inclement. Dessant. Thopiekar. And all the others. Thank you for the awesomeness.