Freestanding Feature-Test Macros and Implementation-Defined Extensions

Document number: P2198R7
Date: 2022-12-14
Reply-to: Ben Craig <ben dot craig at gmail dot com>
Audience: Library Working Group

Change history

R7

R6

R5

R4

R3

R2

R1

R0

Introduction

Users of freestanding implementations would like to be able to detect which library facilities are available for use. With the C++20 feature-test macros, all the feature-test macros are required to be defined in both freestanding and hosted implementations, even when the associated hosted facilities are not available. This paper will make the presence of feature-test macros for hosted facilities implementation-defined.

This paper also clarifies that freestanding implementations need not stop at the bare minimum support for freestanding. Freestanding implementations may provide additional facilities from the standard library, so long as those facilities meet the associated hosted requirements.

Motivation and Design

Beyond the bare minimum

This paper grants implementations the freedom to include more than just the facilities that are marked //freestanding. There will be platforms where, for example, floating point is both available and frequently used (e.g. GPU environments). On those platforms, it is desirable for implementations to provide many of the facilities in the <cmath> and <complex> headers. We must be careful here though, as we don't want to allow divergence of implementation, and we want to be able to add facilities to the required subset of freestanding. To permit both vendor extension and future freestanding growth, this paper will require that additionally included facilities must meet the same requirements as for a hosted implementation, with a few exceptions.

Freestanding implementations may include additional macros and namespace scoped entities, beyond what is required in a minimal freestanding implementation. A freestanding implementation does not need to provide the entirety of a header. If a freestanding implementation supplies an additional class, the entirety of the class must be present and meet all the hosted requirements. This prevents some useful vendor extensions, notably at()-less array, value()-less optional, and similarly partial string_view and bitset. This does not prevent us from standardizing a partial class in the future, though it does prevent us from getting usage experience in a conforming implementation.

Freestanding implementations may make the use of any non-freestanding, namespace scoped function or function template ill-formed (e.g. generally by marking it =delete). The intent is to allow the freestanding implementation to control overload resolution, so that a library that works with a freestanding implementation will have the same semantics in a hosted implementation.

Users will be able to rely on the freestanding portions of the standard library. The freestanding portions have portable semantics, and are required to be present. Users may then decide to rely on implementation-defined additions. WG21 will be able to add hosted entities to the freestanding subset without fear of breaking implementation-defined additions, because the implementation-defined additions needed to meet the hosted requirements.

Detecting once-required features

P2013R3 makes the default allocating ::operator news optional. Users may want to detect this in their code so that they can fall-back to a fixed capacity container rather than a dynamically sized container. Some users may also want to (ab)use such a feature-test macro so that they can provide an ::operator new implementation when one is not provided by the standard library.

This paper recommends a macro to detect the presence of ::operator new definitions: __cpp_lib_freestanding_operator_new. Unlike other feature-test macros, this macro will be required to be set to 0 on freestanding implementations that do not provide useful ::operator new definitions. This will allow users to detect whether the library is an old standard library or a new standard library as well.

Unlike other feature-test macros, this macro will need normative wording to tie the feature-test macro to the feature that it is describing.

Meta feature-test macro

C++20 requires that all implementations (freestanding and hosted) define all of the library feature-test macros in the <version> header, even the feature-test macros that correspond to facilities not required to be present in freestanding implementations. This means that those feature-test macros provide misleading results on freestanding implementations. This isn't just a theoretical problem. Existing implementations are already deploying <version> headers that report support for std::filesystem facilities, std::chrono facilities, and many others, even though those feature-test macros indicate support for features that require the support of an operating system.

In order for users of freestanding implementations to be able to detect extensions of freestanding, the users need a way of distinguishing the C++20 macro requirements from an accurate expression of extension. This paper will provide a new macro, __cpp_lib_freestanding_feature_test_macros, so that users can distinguish between these cases. When __cpp_lib_freestanding_feature_test_macros is present, users will know that the C++20 feature-test macros aren't lying.

Freestanding feature-test macros

The following macros are now required to be present in freestanding implementations. The other library feature test macros in the <version> header are not required to be present on freestanding implementations. The corresponding facilities were either required to be present in C++20, or are added in P1642. The green feature-test macros are the ones added since the paper was forwarded from LEWG.

Post LEWG additions

These papers with freestanding facilities were approved after this paper left LEWG. This paper was included after a September 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote). These papers were included after a November 2021 reflector discussion, and they received five votes in favor, with no opposition (the author of this paper did not vote). These papers were included after an April 2022 reflector discussion, and they received six votes in favor, with no opposition (the author of this paper did not vote).

New feature-test macros

Users of freestanding implementations will want to know whether including a formerly hosted-only header will work or not. Users of freestanding implementations will also what to know if all the facilities that are required to be in freestanding have been made available yet. This is a concern for highly portable libraries, and for users that need to support old and new compilers.

These feature-test macros are provided at a per-header granularity. This enables implementations to advertise new capabilities more easily than a single feature-test macro for the entirety of this paper. This also follows the precedent set by the __cpp_lib_constexpr_* macros.

If new, pre-C++20 functionality is added to the freestanding subset of C++, then the respective feature-test macro for the header should be bumped. If the functionality is new in C++23 or later, then alternative approaches should be taken. These alternative approaches are discussed in the examples section.

Name Header
__cpp_lib_freestanding_feature_test_macros only <version>
__cpp_lib_freestanding_utility <utility>
__cpp_lib_freestanding_tuple <tuple>
__cpp_lib_freestanding_ratio <ratio>
__cpp_lib_freestanding_memory <memory>
__cpp_lib_freestanding_functional <functional>
__cpp_lib_freestanding_iterator <iterator>
__cpp_lib_freestanding_ranges <ranges>

Partial macro coverage

The following, existing feature-test macros cover some features that I am making freestanding, and some features that I am not requiring to be freestanding. These feature-test macros won't be required in freestanding, as they could cause substantial confusion when the hosted parts of those features aren't available. The per-header __cpp_lib_freestanding_* macros should provide a suitable replacement in freestanding environments.

Feature-test macro policy recommendations

This paper patches up many of the historical problems with freestanding and feature-test macros. The following are the guidelines I recommend to keep feature-test macros useful for freestanding in the future.

Examples

The following feature detection examples should ... The examples will take advantage of a helper macro as well: 
#if (__STDC_HOSTED__ || \
     (defined(__cpp_lib_freestanding_feature_test_macros) && \
      __cpp_lib_freestanding_feature_test_macros >= 202112))
#define TRUTHFUL_MACROS 1
#else
#define TRUTHFUL_MACROS 0
#endif

Detect C++20 (or older) feature that is not required in freestanding

Is filesystem::copy available? 
#if TRUTHFUL_MACROS && defined(__cpp_lib_filesystem) && __cpp_lib_filesystem >= 201703L
// hosted and freestanding extension success path
#else
// fallback path
#endif

Detect C++20 (or older) feature that is now entirely required in freestanding

Is ssize available? 
#if TRUTHFUL_MACROS && defined(__cpp_lib_ssize) && __cpp_lib_ssize >= 201902L
// hosted and freestanding success path
#else
// fallback path
#endif

Detect C++20 (or older) feature that is now partially required in freestanding

Is the non-parallel version of uninitialized_default_construct available? 
#if TRUTHFUL_MACROS && defined(__cpp_lib_raw_memory_algorithms) && __cpp_lib_raw_memory_algorithms >= 201606L
// hosted and freestanding extension success path
#elif defined(__cpp_lib_freestanding_memory) && __cpp_lib_freestanding_memory >= 202007L
// freestanding success path
#else
// fallback path
#endif

Detect pre-feature-test-macro feature that is required in freestanding

Is tuple available? 
#if defined(__cpp_lib_freestanding_tuple) && __cpp_lib_freestanding_tuple >= 202007L
// freestanding and future hosted success path
#elif __STDC_HOSTED__
// flakey hosted success path.  Assume tuple works here
#else
// fallback path
#endif

Detect pre-feature-test-macro feature that is not required in freestanding

Is thread available? 
// No good answers here.  Currently calling this out of scope.

Detect C++23 feature that is entirely freestanding in the initial paper

#if defined(__cpp_lib_always_freestanding_feature) && __cpp_lib_always_freestanding_feature >= 202202L
// hosted and freestanding success path
#else
// fallback path
#endif

Detect C++23 feature that is hosted in the initial paper, but made entirely freestanding later (possibly in another version of C++)

For this category of features, the recommendation will be to bump the integer constant in the version test macro. 
#if defined(__cpp_lib_eventually_freestanding_feature)
    #if __cpp_lib_eventually_freestanding_feature >= 202702L
    // freestanding success path.  Will also trigger for new hosted toolchains
    #elif __STDC_HOSTED__ && __cpp_lib_eventually_freestanding_feature >= 202102L
    // Interim hosted success path
    #else
    // Interim freestanding fallback path
    #endif
#else
// fallback path
#endif

// Alternative that will work 99% of the time
#if defined(__cpp_lib_eventually_freestanding_feature)
// freestanding and hosted success, probably
#else
// fallback path
#endif

Detect freestanding part of C++23 feature that is partially freestanding in the initial paper

For this category of features, the recommendation will be to have two macros, one for the freestanding portion, and one for the entire paper. The macros should follow the convention __cpp_lib_foo for the full paper, and __cpp_lib_freestanding_foo for the freestanding portion.
Is the freestanding portion of feature foo available? 
#if defined(__cpp_lib_freestanding_foo) && __cpp_lib_freestanding_foo >= 202202L
// hosted and freestanding success path
#else
// fallback path
#endif

Detect freestanding part of C++23 feature that is made freestanding after the initial paper (possibly in a different C++ release)

For this category of features, the recommendation will be to add a new freestanding macro for the feature, following the convention of __cpp_lib_freestanding_bar for the freestanding portion.
Is the freestanding portion of feature bar available? 
#if defined(__cpp_lib_bar) && __cpp_lib_bar >= 202202L
// Old hosted toolchain path and freestanding extension path
#elif defined(__cpp_lib_freestanding_bar) && __cpp_lib_freestanding_bar >= 202702L
// freestanding success path.  Will also trigger for new hosted toolchains
#else
// fallback path
#endif

Detect absence of a previously required freestanding feature

Is ::operator new's definition in the standard library? 
#if defined(__cpp_lib_freestanding_operator_new)
    #if __cpp_lib_freestanding_operator_new >= 202009L
    // ::operator new is available
    #else
    // No ::operator new available
    static_assert(__cpp_lib_freestanding_operator_new == 0)
    #endif
#elif (/*vendor specific test*/)
// old, non-conforming freestanding "success" path.  No ::operator new available.
#else
// old hosted and heap-capable freestanding code path.  ::operator new is available.
#endif

Rejected alternatives

Bump existing macros instead of provide a new macro

Users need a bit of information to know whether the C++20 macros are lying or not. Prior revisions of this paper provided that bit by bumping the version number of the C++20 macros. This works, but it expands the scope of this change. It also interacts poorly when there are multiple changes to the same macro.

Instead, we provide the bit of information with the new __cpp_lib_freestanding_feature_test_macros macro.

Allow partial classes in conforming vendor extensions to freestanding

Allowing partial classes gives implementers a great deal of freedom to experiment and to provide extensions. It also permits abuse by the implementers. On the positive side, an implementation could provide std::array without at. This helps replace C facilities with better facilities from C++. On the negative side, an implementation could =delete the copy constructor. Removing the wrong functions would also mean that a class may not satisfy the same concepts in hosted vs. freestanding.

Update old feature-test macros to indicate freestanding status

Rather than have a __cpp_lib_freestanding_ratio (for example), we could instead update __cpp_lib_chrono. Users could test the value of __cpp_lib_chrono to determine if it is freestanding safe or not.

There are at least two big flaws with this approach.

First, not all facilities (particularly older ones) have existing feature-test macros to bump (e.g. unique_ptr). Users would still like to be able to detect the availability of the features in freestanding.

Second, implementers don't always implement features in the same order that they are added to the working draft. If a new feature were added to the <chrono> header that necessitated a feature-test macro bump, and an implementer addressed that feature before making the <ratio> header freestanding safe, then the implementer would either need to stick with the pre-freestanding feature-test macro version, or provide a misleading feature-test macro version.

Per-freestanding paper feature-test macro granularity

Rather than introduce a feature-test macro per header in this paper, I could instead introduce one feature-test macro... perhaps __cpp_lib_freestanding. Each paper that adds old facilities to freestanding could then introduce it's own macro, or bump the old one.

This approach can work, but it restricts the order in which implementers can meaningfully implement freestanding features. All of the old paper needs to be done before any of the old paper's progress can be advertised. All of the old paper needs to be done before advertising any newer papers.

Per-facility feature-test macro granularity

This paper could choose to add a feature-test macro for __cpp_lib_freestanding_unique_ptr, __cpp_lib_freestanding_pair, and __cpp_lib_freestanding_iterator_categories. There would be some value to users in that they could express exactly what it is they need, and see if that very specific facility is available.

However, this approach is an implementation hassle, and prone to endless wg21 debates on how to partition and name the facilities. Grouping facilities by header provides a natural partitioning and naming scheme.

Rely on __has_include

Some have suggested using __has_include to detect whether the <tuple> header (for example) is usable on a particular freestanding implementation. This doesn't work in practice for multiple reasons.

The first example is the <iterator> header. In older versions of Visual Studio, a user could attempt to #include <iterator>. The streaming iterators in the header result in compiler errors. We need feature-test macros to indicate whether including the header is well-formed.

The second example is headers that do standard versions checks inside. The libstdc++ implementation of the <ranges> header is mostly empty if the language version is less than C++20. __has_include will still report the header as present though.

Make __cpp_lib_freestanding_* macros freestanding only

This complicates the client feature-test code for features that were marked freestanding in their initial papers. It doesn't make any of the client feature-test code any simpler. It also violates the principle that freestanding should be a subset of hosted.

Define __cpp_lib_freestanding_* macros to 0 in hosted

Users of the freestanding macros would still need to account for the case where the macro isn't present. Defining the macro to zero doesn't provide any information beyond what __STDC_HOSTED__ provides.

Detect once-required features with a feature-test "absence" macro

We could define a feature-test macro in the negative, for example __cpp_lib_freestanding_no_operator_new. This would be different from every other feature-test macro in the standard and SD-6 (SD-6's __cpp_rtti and __cpp_exceptions are defined in the positive). Hosted implementations would never define this macro. Freestanding implementations may or may not define the macro. When the user detects this "absence" macro, they could take action confidently. When a user fails to detect a "presence" macro, the user would need to carefully consider whether it is because of an old standard library or a missing ::operator new definition.

This approach would still be useful "soon", but it would leave the ambiguity of the pre-adoption and post-adoption states.

Detect once-required features with a traditional feature-test macro

We could have a "presence" feature-test macro that is undefined when not set, as opposed to set to 0, as is proposed. Unfortunately, it would take a long time for this macro to be useful in practice. Most implementations today have an ::operator new definition available, but don't have this macro defined. That means that this macro would only be useful in the distant future, where the absence of this macro is more likely to indicate a system without a heap than a system with an old standard library. Setting the macro to 0 when the feature is not present is critical to the usability.

Wording

The following wording is relative to N4917, and assumes that P2013 have been applied.

Additions to [compliance]

Please append the following paragraphs to [compliance].

5
The hosted library facilities are the set of facilities described in this document that are required for hosted implementations, but not required for freestanding implementations.
A freestanding implementation provides a (possibly empty) implementation-defined subset of the hosted library facilities.
Unless otherwise specified, the requirements on each declaration, entity, typedef-name, and macro provided in this way are the same as the corresponding requirements for a hosted implementation, except that not all of the members of the namespaces are required to be present.
6
A freestanding implementation provides deleted definitions ([dcl.fct.def.delete]) for a (possibly empty) implementation-defined subset of the namespace-scope functions and function templates from the hosted library facilities.
[ Note: An implementation may provide a deleted definition so that the result of overload resolution does not silently change when migrating a program from a freestanding implementation to a hosted implementation. -end note]

Change in [version.syn]

Remove the "// all freestanding" comment from the beginning of the synopsis.

Add "freestanding, " to the beginning of each comment in the following macro definitions in [version.syn]:

The changes should look similar to this:
2
#define __cpp_lib_addressof_constexpr               201603L // freestanding, also in <memory>
Add a "// freestanding" comment after the following macro definition in [version.syn]: Please add the following macros to [version.syn]/2. Use the current year-based constant instead of "new-val".

2
#define __cpp_lib_freestanding_feature_test_macros  new-val // freestanding
#define __cpp_lib_freestanding_functional           new-val // freestanding, also in <functional>
#define __cpp_lib_freestanding_iterator             new-val // freestanding, also in <iterator>
#define __cpp_lib_freestanding_memory               new-val // freestanding, also in <memory>
#define __cpp_lib_freestanding_operator_new       see below // freestanding, also in <new>
#define __cpp_lib_freestanding_ranges               new-val // freestanding, also in <ranges>
#define __cpp_lib_freestanding_ratio                new-val // freestanding, also in <ratio>
#define __cpp_lib_freestanding_tuple                new-val // freestanding, also in <tuple>
#define __cpp_lib_freestanding_utility              new-val // freestanding, also in <utility>
Please append the following paragraphs to [version.syn].
3
The macro __cpp_lib_freestanding_operator_new is defined to the integer literal new-val if all the default versions of the replaceable global allocation functions meet the requirements of a hosted implementation, and to the integer literal 0 otherwise ([new.delete]).
4
Recommended practice: Freestanding implementations should only define a macro from <version> if the implementation provides the corresponding facility in its entirety.