Integrating C++ with Python:
How, When, and Why
Austin Bingham
Roxar Software Solutions
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
Named
attributes
Slide 4
Slide 4 text
Serializable
values
Slide 5
Slide 5 text
Metadata
&
introspection
Slide 6
Slide 6 text
Accessible
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Named attributes
Serializable
values
Metadata &
introspection
Accessible
Slide 9
Slide 9 text
”Any sufficiently complicated C or Fortran
program contains an ad hoc, informally-
specified, bug-ridden, slow implementation
of half of Common Lisp.”
– Greenspun's 10th Rule
#include
void initLogging() {
basicConfig();
Logger log = getLogger();
// This handler will log to stderr
log.addHandler(handlers::StreamHandler());
// This will log to stdout
log.addHandler(
handlers::StreamHandler(
boost::python::import("sys")
.attr("stdout")));
}
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
Why & Where
Slide 22
Slide 22 text
print('Hello, world.')
vs.
#include
int main(int, char**) {
std::cout << "Hello, world."
<< std::endl;
return 0;
}
Expressiveness and productivity
Slide 23
Slide 23 text
Edit, compile, run, debug
Slide 24
Slide 24 text
Access to Python modules
●
logging
●
uuid
●
re
●
difflib
●
shutil
●
template
engines
...and so on
C++
Slide 25
Slide 25 text
Existing C++ Code
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
But
where
does
Python
fit?
Slide 28
Slide 28 text
Everywhere
Pareto Principle
Slide 29
Slide 29 text
Filesystem interaction
Writing "servers"
Logging
Access the standard
library
Slide 30
Slide 30 text
Reinventing the Wheel
Slide 31
Slide 31 text
Making the Case
Slide 32
Slide 32 text
Rapid prototyping and high productivity allow
you to build a proof-of-concept in your ”spare
time”
Just do it!
Slide 33
Slide 33 text
Zero-cost
● Software
● Strong community
support
● Transparent, open-
source, fixable
Be cognizant of issue like training
cost, finding developers, etc.
Zero-cost
Slide 34
Slide 34 text
High Productivity
L. Prechelt. An empirical comparison of seven programming
languages. IEEE Computer, 2000.
High Productivity
Slide 35
Slide 35 text
Understand Performance
● Python when you can, C++ when you must
● Proof-by-demonstration (again!)
● Pareto Principle
...PLUS...
Weave
Understand Performance
Type conversion
How do we get objects across the Python-
C++ boundary?
The key is often to remember that
PyObjects are wholly legitimate C-level
entities. There is no magic.
Slide 47
Slide 47 text
Type conversion: A simple model
C++ class
bp::object obj_;
PyObject
Slide 48
Slide 48 text
Type conversion: to C++
inspection
Python to C++
if (PyObject_IsInstance(obj, class_obj))
return new T(
object(
handle<>(
borrowed(
obj))));
Slide 49
Slide 49 text
Type conversion: to Python
templates
C++ to Python
template
struct to_python_object_
{
static PyObject* convert(const T& t)
{
return boost::python::incref(
t.obj().ptr());
}
};
Slide 50
Slide 50 text
Type conversion: registration
// Register from-python converter
boost::python::converter::registry::push_back(
&convertible,
&construct,
boost::python::type_id());
// Register to-python converter
boost::python::to_python_converter<
MyType,
to_python_object_ >();
// Covert from Python to C++
MyType x = boost::python::extract(obj);
// Convert from C++ to Python (implicit in bp::object
constructor)
python_class.attr("doit")(x);
Slide 51
Slide 51 text
Exceptions: with C-API
PythonAPI_Foo();
if (PyErr_Occurred())
respondToPythonException();
PythonAPI_Bar();
if (PyErr_Occurred())
respondToPythonException();
Slide 52
Slide 52 text
Exceptions: error_already_set
Python
exception bp::error_already_set
try {
some_class.attr("method")(3);
}
catch (const bp::error_already_set&) {
// Some exception has been thrown in Python
throw SomeException();
}
Slide 53
Slide 53 text
Exception translation
PyObject *t, *v, *tb;
PyErr_Fetch(&t, &v, &tb);
// Check if it's a ValueError
if (PyErr_GivenExceptionMatches(
value_error_class_obj, t))
{
throw ValueError();
}
Iteration
There's a very natural mapping between C++ and
Python iterators
● C++ iterator holds a Python iterator reference
● Incrementing iterator calls next(iter)and stores
values
● StopIteration is caught in increment
● The "end" iterator simply has a None Python
iterator
Slide 56
Slide 56 text
Duck Typing vs. Static Typing
mypackage::Database
C++
Python
mypackage.DB
mypackage.DBProxy
pkg2.FancyDB
Debugging the full stack
C++ application
C++ wrapper library
Python module
C/C++ library
debug?!
● pdb.set_trace()
● gdb with python
extensions
● VisualStudio python
support (?)
● SIGINT to break
debugger
● Common logging
● Debugging individual
layers
Slide 60
Slide 60 text
Debugging: Pythonizing the App
main() extension
C++ wrapper library
Python module
C/C++ library
Python application pdb
gdb
attach
Slide 61
Slide 61 text
Performance
● Python function call overhead can cause
problems.
● Be aware of unnecessary data copies.
● Learn the buffer and memoryview APIs
In general, the 80/20 principle works in
your favor. Performance is not an issue for
most code, and when it is you can bring
many tools to bear.
Slide 62
Slide 62 text
● Lutz Prechelt. An empirical comparison of seven programming languages. IEEE
Computer 33(10):23-29, October 2000.
● Hobbs, Jeff. ”5 Python Pluses for the Enterprise” http://www.linuxinsider.
com/rsstory/70337.html
● misspent.wordpress.com/2009/09/27/how-to-write-boost-python-converters/
● misspent.wordpress.com/2009/10/11/boost-python-and-handling-python-
exceptions/
● misspent.wordpress.com/2012/03/24/debugging-cc-and-cpython-using-gdb-7s-
new-python-extension-support/
● Ackward: code.google.com/p/ackward/
Links & References