C++20 modules and Boost: a prototype

CMake

Every library provides a module interface unit defining its exports. For instance, in Boost.Mp11, this file is modules/boost_mp11.cppm. It’s installed to CMAKE_INSTALL_DATADIR, which places it in /usr/local/share/boost_mp11.cppm by default.

Libraries that want to support C++20 module builds include conditional logic in their CMake to react to BOOST_USE_MODULES. For example, this is what Boost.Mp11 would look like. Libraries that don’t support C++20 module builds are built and installed as they are today.

In C++20 module builds, binary artifacts are generated even for previously header-only builds. For instance, the above CMake generates a libboost_mp11.a in Linux. In most cases, these libraries only contain module initializers. I’ve made these libraries unconditionally static, to reduce overhead. They are installed along other compiled Boost libraries.

The libraries can be consumed from CMake with add_subdirectory and find_package, as usual. However, due to CMake current limitations, the find_package workflow is more sensitive to build flags than with headers.

Mixing includes and imports

At the time of writing, standard library implementations support including standard headers first, then importing std. This is relevant because some standard library headers still need to be included for macros to be visible.

In general, we’ve chosen not to support this in the general case: you should either include or import Boost, but not both. Compatibility headers help maintain this consistency across your project.

This choice allows libraries to attach their declarations to their named module ^[2]. This makes ODR violations easier to detect and may speed up compilation.

Some libraries may still want to support mixing imports and includes. Compiled libraries with tests that require access to implementation details are an example of this. See this section for more info.

Some libraries need to make macros available to users. Macros must always be exported using traditional includes, since modules don’t know anything about macros. In the prototype, compatibility headers make public macros available in addition to importing the relevant module. For example, Boost.Core has a lightweight testing framework used in unit tests that relies on macros. The boost.core module exports the required C++ entities, with the header performing the relevant imports and macro definitions.

All public headers have been converted into compatibility headers. This is what a compatibility header could look like:

// File: boost/mp11/list.hpp

// Conditionally skip declarations. BOOST_MP11_INTERFACE_UNIT is only defined
// in boost_mp11.cppm
#if defined(BOOST_USE_MODULES) && !defined(BOOST_MP11_INTERFACE_UNIT)

#include <boost/mp11/version.hpp> // Declares the BOOST_MP11_VERSION macro

// Boost libraries might need to define this because of certain limitations
// on where imports can be located in module units
#ifndef BOOST_MP11_SKIP_IMPORT
import boost.mp11;
#endif

#else

namespace boost::mp11 { /* regular declarations */ }

#endif

The idea is that:

We’ve also created a bunch of standard library compatibility headers in Boost.Config that follow the same principle. For example:

// File: boost/config/std/type_traits.hpp
#ifdef BOOST_USE_MODULES
#ifndef BOOST_CONFIG_SKIP_IMPORT_STD
import std;
#endif
#else
#include <type_traits>
#endif

I’d like to thank Peter Dimov for proposing the idea on compatibility headers.

Writing module interface units (boost_mp11.cppm)

We first need to make sure that our headers don’t include any third-party code when BOOST_USE_MODULES is defined. Standard library headers can be replaced by the equivalent Boost.Config compatibility headers. Boost dependencies don’t need to be updated. Some other headers may need to be ifdef’ed-out and included in the global module fragment.

For example: ^[3]

// File: boost/mp11/list.hpp

#if defined(BOOST_USE_MODULES) && !defined(BOOST_MP11_INTERFACE_UNIT)
// Compatibility header section: omitted for brevity
#else

// Includes
#include <boost/mp11/detail/config.hpp>     // Our own includes stay as they are
#include <boost/config/std/type_traits.hpp> // Replace stdlib includes
                                            // by compatibility headers

namespace boost::mp11 { /* regular declarations */ }

#endif

We now need to mark C++ entities in the public interface as exported. The first solution to this is to create a BOOST_MP11_MODULE_EXPORT macro that expands to export in module builds, and to nothing otherwise. This is similar to what we do today to handle DLL exports today. Some code samples:

// File: boost/mp11/list.hpp
// Compatibility header and includes skipped for brevity

BOOST_MP11_MODULE_EXPORT // defined to export if BOOST_USE_MODULES is defined, to nothing otherwise
template<class... T> struct mp_list
{
};

The module interface becomes:

// File: boost_mp11.cppm

module; // Global module fragment
#define BOOST_MP11_INTERFACE_UNIT     // We want headers to actually declare entities
#define BOOST_CONFIG_SKIP_IMPORT_STD  // Don't import std in compatibility headers
#include <cassert> // Some standard library headers need to be included for their macros

export module boost.mp11;

import std;               // Import should be first
#include <boost/mp11.hpp> // All entities declared here get attached to the named module
                          // This issues a compiler warning that should be suppressed

This allows attaching the declared entities to the boost.mp11 module, but has the following drawbacks:

We can use the export using technique as an alternative. Dependencies should still be ifdef’ed-out or replaced by compatibility headers, but no BOOST_MP11_MODULE_EXPORT macro is required:

// File: boost/mp11/list.hpp
// Compatibility header and includes skipped for brevity

template<class... T> struct mp_list // No export macro required
{
};

The interface unit becomes:

// File: boost_mp11.cppm
module; // Global module fragment
#define BOOST_MP11_INTERFACE_UNIT // We want headers to actually declare entities
                                  // No BOOST_CONFIG_SKIP_IMPORT_STD: import std is fine in the global module fragment
#include <cassert>        // Some standard library headers need to be included for their macros
#include <boost/mp11.hpp> // All entities are attached to the global module.

export module boost.mp11;

// List all symbols we want to export
export namespace boost::mp11 {
using mp11::list;
}

This technique doesn’t attach names to the named module, with the pros and cons this brings. Additionally, it hits two troublesome bugs in MSVC:

Compiled libraries

As with header-only libraries, compiled libraries should also provide a .cppm file stating the functions exported by the module. For Charconv, I’ve converted .cpp files in module implementation units in module builds.

In Windows, when shared libraries are enabled, a CMake limitation makes module interfaces within the same project always build with __declspec(dllexport). This has the effect of introducing an extra indirection when calling library functions. This limitation is expected to be lifted in the future.

Note that module exports need not match with DLL exports. DLL exports define the library’s ABI, while module exports define its API.

Some tests in Boost.Charconv need to access implementation details (i.e. entities in the detail namespace). If it was a header-only library, such tests could just include the relevant detail header instead of importing the module. This does not work for compiled libraries because detail headers might reference functions defined in the module implementation units. In other words, these tests need to mix includes and imports. For this reason, I’ve used the export using technique for Boost.Charconv.