Proceedings of the 8th Annual Python in Science Conference

More documents

Recommendations

Info

Convert-XY: type-safe interchange of C++ and Python containers for NumPy extensionsCopyThis compound type specifies that the string keys andvector elements be copied but the buffer pointer to thearrays should be reused. Figure 5 illustrates how thisrecursive instantiation works.DefaultToCPPAction::ActionCopyKeyActionDefaultToCPPAction::ActionActionCopyValueActionDefaultToCPPAction::ActionActionCopyElementActionDefaultToCPPAction Action::ActionActionAllocateCopyCopyFigure 5 The process of conversion is deducedat compile time via recursive instantiation ofDefaultToCPPAction.The process of conversion can be overridden by passingas a template argument a different Action type. Forexample, suppose the target BasicImage was allocatedfrom elsewhere (e.g. a special malloc function thatproperly aligns buffers), we can override the defaultconversion action (Reuse) so Copy is used as theAction instead. The ConvertXY::convert_overridefunction can be used for this purpose:void example(PyObject *x, BasicImage &y) {convert_override(x, y);}Suppose a C++ object y of a compound container typealready exists, e.g. an object of type map. In this case, the keys shouldnot be copied but we must ensure that the C++ mapcontains exactly the same set of keys as the Pythondictionary. This is done by first checking that the sizeof the dictionary is the same as the std::map and thenmaking sure each unique key in the dictionary is equivalentto some unique key in the map.The contents of a dictionary of arrays can be copiedinto a std::map of existing BasicImage buffers as follows:void example(PyObject *x,map &y) {convert_override(x, y);}The CheckSize simple checks that the size ofthe std::map and Python map are equivalent.CheckExists ensures each key in the Python map isalso in the STL map.Reference counting is automatically handled duringconversion with PyCXX. In cases where the flow ofexecution may return to Python after conversion, Py-CXX can be used to ensure objects that are in use byC++ routines don’t get prematurely destroyed.:void example(PyObject *x,BasicImage &y) {// The line below is safe.convert(x,y);// However, if it can’t be guaranteed that program// control flow will never return to Python// for the rest of this function, use PyCXX’s// Py::Object to increment the reference count.Py::Object obj(x);// If the flow of execution returns to Python while// executing the function below, y will always point// to a valid buffer.compute(y);}The Other Way Around: from C++ to PythonWhen converting from C++ to Python, the typestructure of the target Python object must beprespecified at compile time (e.g. the type of thePython object to be created is static). This is notstraightforward since there is sometimes more thanone compatible Python type for a C++ type. Forexample, an STL vector can be converted to a Pythonlist, tuple, or NumPy object array. In Convert-XYthis is deduced with a compile-time-only Structuremetatype deduced by recursive instantiation ofDefaultToPythonStructure::Structure.Figure 6illustrates how DefaultToPythonStructureis recursively instantiated to yield a default Pythontarget type.ToPythonDefaultStructure::StructureStructurePyDictKeyStructureToPythonDefaultStructure::StructureStructurePyStringNon-terminal Structure TypeTerminal Structure TypeExpanded typedefValueStructureToPythonDefaultStructure::StructureElementStructurePyListElementStructureDefault::StructureStructurePyArrayPyDictFigure 6 The default Python type when convertingfrom C++ to Python is deduced by recursive instantiationof ToPythonDefaultStructure.As the figure illustrates, the default type for a Pythontarget given a source object that’s an STL map mappingstring to vectors of images is PyDict. If a tuple is preferredover a list, the default structure can be overridenas follows:c○2009, D. Eads, E. Rosten 32
Proceedings of the 8 th Python in Science Conference (SciPy 2009)PyObject *y= convert_override(x)BasicImagedata float*PyArrayObject*data float*If an attempt is made to convert a C++ object to aPython object that’s incompatible, meaningful compilermessages can be generated via a trick involvingincomplete types:in not_found.hpp:88:’unsupport_type_or_structure’has incomplete typeCannotConvertToPython.Allocating Result ArraysOne approach to allocating result arrays is to copythe result (e.g. image or matrix) into a new NumPyarray. This overhead can be avoided using theConvertXY::allocate_numpy_array function. Thisfunction is templated, and pattern matching is performedon the C++ type to generate an allocatorat compile time. In this example, when theallocate_numpy_array function is called, NumPyarray and BasicImage object are allocated at thesame time, sharing the same buffer pointer. Theimage is smoothed using the LIBCVD functionconvolveGaussian, as illustrated on figure 7:PyObject *smooth_image(PyObject *x, double radius) {}// First convert the NumPy array input image// to a BasicImage.Holder y;convert(x, y);// Now allocate enough memory to store the// result. Use the same buffer pointer for// both the NumPy array and BasicImage.Holder result;PyObject *pyResult =allocate_numpy_array(result,y.getReference().size());// Using LIBCVD, convolve a Gaussian on the// converted input, store the result in the// buffer pointed to by both pyResult and// result.CVD::convolveGaussian(y.getReference(),result.getReference(),radius);return pyResult;Libraries SupportedConvert-XY supports conversion between Pythonand objects from several C++ libraries includingTooN, LIBCVD, and STL. The library is split intothree different headers, all of which are optional.• ConvertXY/TooN.hpp: for converting betweenNumPy arrays and TooN matrices and vectors.• ConvertXY/STL.hpp: for converting between STLstructures and Python objects.C++ Computationfloat bufferPythonComputationFigure 7 allocate_numpy_array constructs a newPython and C++ array at the same time, to wit, aNumPy array and a BasicImage object. The NumPyarray owns the buffer allocated so when its referencecount goes to zero, its memory buffer will be freed.• ConvertXY/CVD.hpp: for converting betweenNumPy arrays and CVD::Image C++ objects.The compiler can convert combinationsof STL, CVD, and TooN structures (e.g.vector)only when their respective headers are includedtogether.Tom’s object-oriented Numerics Library (TooN)TooN is a header-only linear algebra library thatis very fast on small matrices making it particularlyuseful for real-time Computer Vision applications[Dru03]. TooN provides templated classes for staticand dynamically-sized vectors and matrices, and usestemplated-based pattern matching to catch for commonlinear algebra errors at compile time. For example,when converting the result of an a matrix multiplicationinvolving two matrices X and Y of incompatibledimensions:Matrix X;Matrix Y;PyObject *Z;Z = convert(X * Y);an easy-to-understand compiler error results,“dimension_mismatch has incomplete typeSizeMismatch”.Dimension checking is performed at compile time evenwhen some dimensions are not known beforehand asthe following example shows:Matrix X;Matrix Y;PyObject *Z;Z = convert(X * Y);If a dimension check passes at compile time, it is omittedat run time. A buffer pointer of a NumPy arraycan be reused in a TooN::Matrix by specifying a layouttype as the third template argument to Matrix(which is matched with DefaultAction):33 http://conference.scipy.org/proceedings/SciPy2009/paper_4
Page 6 and 7: Convert-XY: type-safe interchange o

Proceedings of the 8th Annual Python in Science Conference

Create successful ePaper yourself

Delete template?

Save as template?