Build systems¶
Building with setuptools¶
For projects on PyPI, building with setuptools is the way to go. Sylvain Corlay has kindly provided an example project which shows how to set up everything, including automatic generation of documentation using Sphinx. Please refer to the [python_example] repository.
[python_example] | https://github.com/pybind/python_example |
Building with cppimport¶
[cppimport] is a small Python import hook that determines whether there is a C++ source file whose name matches the requested module. If there is, the file is compiled as a Python extension using pybind11 and placed in the same folder as the C++ source file. Python is then able to find the module and load it.
[cppimport] | https://github.com/tbenthompson/cppimport |
Building with CMake¶
For C++ codebases that have an existing CMake-based build system, a Python extension module can be created with just a few lines of code:
cmake_minimum_required(VERSION 2.8.12)
project(example)
add_subdirectory(pybind11)
pybind11_add_module(example example.cpp)
This assumes that the pybind11 repository is located in a subdirectory named
pybind11
and that the code is located in a file named example.cpp
.
The CMake command add_subdirectory
will import the pybind11 project which
provides the pybind11_add_module
function. It will take care of all the
details needed to build a Python extension module on any platform.
A working sample project, including a way to invoke CMake from setup.py
for
PyPI integration, can be found in the [cmake_example] repository.
[cmake_example] | (1, 2) https://github.com/pybind/cmake_example |
pybind11_add_module¶
To ease the creation of Python extension modules, pybind11 provides a CMake function with the following signature:
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
[NO_EXTRAS] [SYSTEM] [THIN_LTO] source1 [source2 ...])
This function behaves very much like CMake’s builtin add_library
(in fact,
it’s a wrapper function around that command). It will add a library target
called <name>
to be built from the listed source files. In addition, it
will take care of all the Python-specific compiler and linker flags as well
as the OS- and Python-version-specific file extension. The produced target
<name>
can be further manipulated with regular CMake commands.
MODULE
or SHARED
may be given to specify the type of library. If no
type is given, MODULE
is used by default which ensures the creation of a
Python-exclusive module. Specifying SHARED
will create a more traditional
dynamic library which can also be linked from elsewhere. EXCLUDE_FROM_ALL
removes this target from the default build (see CMake docs for details).
Since pybind11 is a template library, pybind11_add_module
adds compiler
flags to ensure high quality code generation without bloat arising from long
symbol names and duplication of code in different translation units. It
sets default visibility to hidden, which is required for some pybind11
features and functionality when attempting to load multiple pybind11 modules
compiled under different pybind11 versions. It also adds additional flags
enabling LTO (Link Time Optimization) and strip unneeded symbols. See the
FAQ entry for a more detailed explanation. These
latter optimizations are never applied in Debug
mode. If NO_EXTRAS
is
given, they will always be disabled, even in Release
mode. However, this
will result in code bloat and is generally not recommended.
By default, pybind11 and Python headers will be included with -I
. In order
to include pybind11 as system library, e.g. to avoid warnings in downstream
code with warn-levels outside of pybind11’s scope, set the option SYSTEM
.
As stated above, LTO is enabled by default. Some newer compilers also support
different flavors of LTO such as ThinLTO. Setting THIN_LTO
will cause
the function to prefer this flavor if available. The function falls back to
regular LTO if -flto=thin
is not available.
Configuration variables¶
By default, pybind11 will compile modules with the C++14 standard, if available
on the target compiler, falling back to C++11 if C++14 support is not
available. Note, however, that this default is subject to change: future
pybind11 releases are expected to migrate to newer C++ standards as they become
available. To override this, the standard flag can be given explicitly in
PYBIND11_CPP_STANDARD
:
# Use just one of these:
# GCC/clang:
set(PYBIND11_CPP_STANDARD -std=c++11)
set(PYBIND11_CPP_STANDARD -std=c++14)
set(PYBIND11_CPP_STANDARD -std=c++1z) # Experimental C++17 support
# MSVC:
set(PYBIND11_CPP_STANDARD /std:c++14)
set(PYBIND11_CPP_STANDARD /std:c++latest) # Enables some MSVC C++17 features
add_subdirectory(pybind11) # or find_package(pybind11)
Note that this and all other configuration variables must be set before the
call to add_subdirectory
or find_package
. The variables can also be set
when calling CMake from the command line using the -D<variable>=<value>
flag.
The target Python version can be selected by setting PYBIND11_PYTHON_VERSION
or an exact Python installation can be specified with PYTHON_EXECUTABLE
.
For example:
cmake -DPYBIND11_PYTHON_VERSION=3.6 ..
# or
cmake -DPYTHON_EXECUTABLE=path/to/python ..
find_package vs. add_subdirectory¶
For CMake-based projects that don’t include the pybind11 repository internally,
an external installation can be detected through find_package(pybind11)
.
See the Config file docstring for details of relevant CMake variables.
cmake_minimum_required(VERSION 2.8.12)
project(example)
find_package(pybind11 REQUIRED)
pybind11_add_module(example example.cpp)
Note that find_package(pybind11)
will only work correctly if pybind11
has been correctly installed on the system, e. g. after downloading or cloning
the pybind11 repository :
cd pybind11
mkdir build
cd build
cmake ..
make install
Once detected, the aforementioned pybind11_add_module
can be employed as
before. The function usage and configuration variables are identical no matter
if pybind11 is added as a subdirectory or found as an installed package. You
can refer to the same [cmake_example] repository for a full sample project
– just swap out add_subdirectory
for find_package
.
Advanced: interface library target¶
When using a version of CMake greater than 3.0, pybind11 can additionally
be used as a special interface library . The target pybind11::module
is available with pybind11 headers, Python headers and libraries as needed,
and C++ compile definitions attached. This target is suitable for linking
to an independently constructed (through add_library
, not
pybind11_add_module
) target in the consuming project.
cmake_minimum_required(VERSION 3.0)
project(example)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
add_library(example MODULE main.cpp)
target_link_libraries(example PRIVATE pybind11::module)
set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}"
SUFFIX "${PYTHON_MODULE_EXTENSION}")
Warning
Since pybind11 is a metatemplate library, it is crucial that certain
compiler flags are provided to ensure high quality code generation. In
contrast to the pybind11_add_module()
command, the CMake interface
library only provides the minimal set of parameters to ensure that the
code using pybind11 compiles, but it does not pass these extra compiler
flags (i.e. this is up to you).
These include Link Time Optimization (-flto
on GCC/Clang/ICPC, /GL
and /LTCG
on Visual Studio) and .OBJ files with many sections on Visual
Studio (/bigobj
). The FAQ contains an
explanation on why these are needed.
Embedding the Python interpreter¶
In addition to extension modules, pybind11 also supports embedding Python into
a C++ executable or library. In CMake, simply link with the pybind11::embed
target. It provides everything needed to get the interpreter running. The Python
headers and libraries are attached to the target. Unlike pybind11::module
,
there is no need to manually set any additional properties here. For more
information about usage in C++, see Embedding the interpreter.
cmake_minimum_required(VERSION 3.0)
project(example)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
add_executable(example main.cpp)
target_link_libraries(example PRIVATE pybind11::embed)
Building manually¶
pybind11 is a header-only library, hence it is not necessary to link against any special libraries and there are no intermediate (magic) translation steps.
On Linux, you can compile an example such as the one given in Creating bindings for a simple function using the following command:
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
The flags given here assume that you’re using Python 3. For Python 2, just
change the executable appropriately (to python
or python2
).
The python3 -m pybind11 --includes
command fetches the include paths for
both pybind11 and Python headers. This assumes that pybind11 has been installed
using pip
or conda
. If it hasn’t, you can also manually specify
-I <path-to-pybind11>/include
together with the Python includes path
python3-config --includes
.
Note that Python 2.7 modules don’t use a special suffix, so you should simply
use example.so
instead of example`python3-config --extension-suffix`
.
Besides, the --extension-suffix
option may or may not be available, depending
on the distribution; in the latter case, the module extension can be manually
set to .so
.
On Mac OS: the build command is almost the same but it also requires passing
the -undefined dynamic_lookup
flag so as to ignore missing symbols when
building the module:
$ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
In general, it is advisable to include several additional build parameters that can considerably reduce the size of the created binary. Refer to section Building with CMake for a detailed example of a suitable cross-platform CMake-based build system that works on all platforms including Windows.
Note
On Linux and macOS, it’s better to (intentionally) not link against
libpython
. The symbols will be resolved when the extension library
is loaded into a Python binary. This is preferable because you might
have several different installations of a given Python version (e.g. the
system-provided Python, and one that ships with a piece of commercial
software). In this way, the plugin will work with both versions, instead
of possibly importing a second Python library into a process that already
contains one (which will lead to a segfault).
Generating binding code automatically¶
The Binder
project is a tool for automatic generation of pybind11 binding
code by introspecting existing C++ codebases using LLVM/Clang. See the
[binder] documentation for details.
[binder] | http://cppbinder.readthedocs.io/en/latest/about.html |