3-Way Web: Getting Javascript, C++ and Python to interact together all at once

Due to internal constraints, we’ve had to replace our old C++ UI made of spaghetti code with a new Javascript UI. We’ve also had to create Python bindings to our C++ backend so that mathematicians, physicists and biologists could pilot and script our software.

But Python also makes a great backend for a web platform!

In this talk, we’ll explore how we can get those 3 languages to interact in an event-driven manner, and I’ll share what worked and didn’t work with our approach.

Speaker
Paul Habfast
Lead Engineer & Head of Cloud Services by Nanolive (SA)

Transcript

1. 3-Way Web GETTING JAVASCRIPT, C++ AND PYTHON TO INTERACT TOGETHER ALL AT ONCE
2. A little bit of context: + =
3. Outline - A little bit more context - Python and C++ - Javascript and C++ - Python and Javascript - The good, the bad, and the ugly - What’s next? - Questions?
4. A little bit more context:
 Technology stack* *Mainly:
5. A little bit more context:
 Our challenges
6. Python and C++:
 Calling C++ from Python We used pybind11: #include <pybind11/pybind11.h> int add(int i, int j) { return i + j; } PYBIND11_MODULE(example, m) { m.doc() = "pybind11 example plugin"; // optional module docstring m.def("add", &add, "A function which adds two numbers"); }
7. Python and C++:
 Calling python from C++ We embedded a python interpreter! Now we can call C++ from python from C++!
8. Javascript and C++:
 Calling javascript from C++ - Qt comes with an embedded Chromium - Gives you high-level access to many controls and APIs QWebSocketServer server server.listen(QHostAddress::LocalHost, 0) prerunScript.setInjectionPoint(QWebEngineScript::DocumentCreation); prerunScript.setSourceCode( QString{"window.cx_channel_url = 'ws://localhost:%1’;”} .arg(server.serverPort()) ); page()->scripts().insert(prerunScript);
9. Javascript and C++:
 Calling C++ from javascript Qt helps us again: QWebChannel is a micro-library that wraps Qt Objects and exposes them in javascript It’s essentially a wrapper on top of WebSockets
10. Python and Javascript:
 just use C++ as a middle man!
11. The good, the bad and the ugly:
 The good: the devtools We have a chromium, hence we can use: - chrome devtools - react devtools - redux devtools Makes debugging a whole lot easier! Also: our javascript only has to target one platform!
12. The good, the bad and the ugly:
 The good: numpy integration Thanks to pybind11 and numpy’s architecture, our volumes of data are directly usable using numpy 0 overhead, memory or CPU!
13. The good, the bad and the ugly:
 The good: JS Promises & Redux-Saga - We modified QWebChannel so that it would work using JS Promises instead of callbacks - Redux-saga is a library that aims to make application side effects easier to manage, more efficient to execute, easy to test, and better at handling failures.
14. function* getItems(itemsApi) {
 const registeredItems = yield call(itemsApi.registeredItems);
 yield put(itemsFetched(registeredItems));
 } function* addItem(itemsApi, { payload: { name, value } }) {
 const success = yield call(itemsApi.registerItem, name, value);
 yield success
 ? put(addItemSuccessful(name, value))
 : put(addItemRejected(name));
 } function* deleteItem(itemsApi, { payload: { name } }) {
 const success = yield call(itemsApi.unregisterItem, name);
 yield success
 ? call(getItems, itemsApi)
 : put(deleteItemRejected(name));
 } export default function* configurationSaga() {
 const qchannel = yield call(initializeChannel);
 const itemsApi = qchannel.items;
 yield call(getItems, itemsApi);
 yield all([
 takeEvery(ITEM_DELETE_REQUESTED, deleteItem, itemsApi),
 takeEvery(ITEM_ADD_REQUESTED, addItem, itemsApi),
 ]);
 }
15. The good, the bad and the ugly:
 The bad: Image stream performance You need to send a real-time live feed from your camera to the javascript, with instant initialization time, low memory usage: 50 fps at 1MB per frame, without overloading the js - you dedicate one thread to compression - and another one to data upload - and you reduce quality (png instead of raw) to save bandwidth - and you reduce resolution (256kB instead of 1MB) to save processing time and bandwidth - and you reduce framerate (25fps instead of 50) to save processing time and bandwidth - and you bypass redux and react to avoid killing the devtools
16. The good, the bad and the ugly:
 The bad: Qt update cycle Our version of Chromium is bound to the version of C++ - Oh but we have a visual bug in the CSS filters - Ha it’s because we use an old version of chromium, let’s upgrade it - Then we need to upgrade Qt - Oh, but the newest Qt only embeds chromium if we use MSVC 2017 - But all our dependencies are compiled with MSVC 2015, and they’re not binary compatible! - Deal with it
17. The good, the bad and the ugly:
 The ugly: C++ setup vs Python setup You want to keep your sources together In C++ your build directory is in a different place than your sources In Python your execution directory is the same as your source directory But you want your python to refer to your built dlls, and to execute python scripts from within your console
18. The good, the bad and the ugly:
 The ugly: Main thread ownership In a GUI environment, different threads can have various handles on multiple event sources (network, USB, etc), but only the main thread can receive GUI events In a python REPL, the python interpreter is waiting for user inputs on the main thread In a python web server environment, the main thread is generally taken by the event loops waiting for network events How do you solve this?
19. The good, the bad and the ugly:
 The ugly: Software architecture
20. What’s next? Ship that product! Integrate the volumes with an ML framework such as Tensorflow or PyTorch Write that C++ WSGI gateway to create a proper python server, for example django Create a better console, using JS
21. Questions? Thank you for your attention
22. Quizz - What’s the name of the tool used to expose our C++ to python Boost::python / pybind11 / python extender / PyObject - Who’s Nanolive designer? Calvin Klein / Karl Lagerfeld / Donatella Versace / Giorgio Armani - How do you invoke a saga from another saga? Invoke / Call / Spawn / Action - How does the javascript communicate with the C++: Natively / Shared memory / REST Api / websockets
23. Quizz - How do we get the result of an API call in javascript A promise / a callback / the return value / ES6 async functions - Was this talk too technical? Yes / No / I fell asleep / I prefer poneys
24. Quizz - How does python communicate with C++: Natively / Shared memory / REST Api / websockets - How do you test your python bindings: Unit tests / integration tests / QA / you don’t - How do you solve bugs in the embedded Chromium: Bug report / Upgrade Chromium / Download source, fix and recompile / you don’t - Who owns the main thread? Python / C++ / Javascript / the user