1. Revision History
1.1. Revision 3 - April 15th, 2021
-
Vastly improve examples.
-
Re-target motivation.
-
Adjust fixes for things that have changed since C++20.
1.2. Revision 2 - January 13th, 2020
-
Improve wording and add more explanation in the design sections for the changes.
1.3. Revision 1 - November 24th, 2019
-
Change to
concepts. (November 6th)snake_case -
Update wording to target [n4835]. (November 6th)
-
Moved by LWG! 🎉 (November 6th)
-
Last minute objection to this paper in plenary.
-
Withdrawn; targeting C++23 instead with better design.
-
Explored 3 different designs offline: settle on 1 (thanks, Oktal).
1.4. Revision 0 - August, 5th, 2019
-
Initial release.
2. Motivation
C++ 20 | With Proposal |
---|---|
❌ - Compilation error
⚠️ - Compiles, but |
✔️ - Compiles and works with no extra template instantiations
✔️ - Compiles and works with no extra templates. |
❌ - Compilation error ⚠️ - Compiles, but is and is
|
✔️ - Compiles and works, types match input. ✔️ - Compiles and works, where is and is .
|
Currently in C++, there is no Generic ("with a capital G") way to take a range apart with its iterators and put it back together. That is, the following code is not guaranteed to work:
template < typename Range > auto operate_on_and_return_updated_range ( Range && range ) { using uRange = std :: remove_cvref_t < Range > ; if ( std :: ranges :: empty ( range )) { // ⛔! // ... the below errors return uRange ( std :: forward < Range > ( range )); } /* perform some work with the iterators or similar */ auto first = std :: ranges :: begin ( range ); auto last = std :: ranges :: end ( range ); if ( * first == u '\0xEF ') { // ... std :: advance ( first , 3 ); // ... } // ... algorithm finished, // return the "updated" range! // ⛔! // ... but the below errors return uRange ( std :: move ( first ), std :: move ( last )); } int main () { std :: string_view meow_view = "나는 유리를 먹을 수 있어요. 그래도 아프지 않아요" ; // this line will error std :: string_view sub_view = operate_on_and_return_updated_range ( meow_view ); return 0 ; }
The current fix is to employ
to return a generic subrange:
template < typename Range > auto operate_on_and_return_updated_range ( Range && range ) { using uRange = std :: remove_cvref_t < Range > ; using I = std :: Iterator < uRange > ; using S = std :: Sentinel < uRange > ; using Result = std :: ranges :: subrange < I , S > ; if ( std :: ranges :: empty ( range )) { return Result ( std :: forward < Range > ( range )); } // perform some work with the // iterators or similar auto first = std :: ranges :: begin ( range ); auto last = std :: ranges :: end ( range ); if ( * first == u '\0xEF ') { // ... std :: advance ( first , 3 ); // ... } // ... algorithm finished, // return the "updated" range! // now it works! return Result ( std :: move ( first ), std :: move ( last )); } int main () { std :: string_view meow_view = "나는 유리를 먹을 수 있어요. 그래도 아프지 않아요" ; auto sub_view = operate_on_and_return_updated_range ( meow_view ); // decltype(sub_view) == // std::ranges::subrange<std::string_view::iterator,std::string_view::iterator> // which is nowhere close to ideal. return 0 ; }
This makes it work with any two pair of iterators, but quickly becomes undesirable from an interface point of view. If a user passes in a
or a
that interface and information is entirely lost to the user of the above function.
does not -- and cannot/should not -- mimic the interface of the view it was created from other than what information comes from its iterators: it is the barebones idea of a pair-of-iterators/iterator-sentinel style of range. This is useful in the generic sense that if a library developer must work with iterators, they can always rely on creation of a
of the iterator and sentinel.
2.1. Preservation of Properties
Unfortunately, always using
decreases usability for end users. Users who have, for example a
, would prefer to have the same type after calling an algorithm. These types have functions and code that are purpose-made for
-like code: losing that intent means needing to reconstruct that from the ground up all over again. There is little reason why the original type needs to be discarded if it supports being put back together from its iterators.
The break-it-apart-and-then-generic-
approach also discards any range-specific storage optimizations and layout considerations, leaving us with the most bland kind of range similar to the "pair of iterators" model. Consider, for example,
. The author of the Low Level File IO (LLFIO) library, Niall Douglass, has spent countless days submitting pull requests and change requests to MSVC, GCC, and Clang’s standard libraries to ask them to change their structural layout to match things like the
of their platform or their asynchronous buffer span types. This optimization matters because it realizes the performance difference between having to individually transformed a container of
s to a more suitable type, or being able to simply
the desired sequences of spans into the underlying asynchronous operations. If range operations were to be performed on the
type, a
would hold, in most cases, two iterators rather than an iterator and a size. This completely eliminates the
optimization. This is not the only place this happens, however: other types have different storages requirements that are not appropriately captured by their iterators in the most general sense, but may benefit from reconstruction and desired type information for the target range they should be constructed into.
2.2. Compile-Time and Interoperability
Compilation time goes up as well in the current paradigm. Users must spawn a fresh
for every different set of iterator/sentinel/kind triplet, or handle deeply nested templates in templates as the input types. This makes it impossible to compile interfaces as dynamic libraries without having to explicitly materialize or manually cajole a
into something more palatable for the regular world.
There is also a problem where there are a wide variety of ranges that could conceivably meet this criterion, but do not. The author of this paper was not the only one to see utility for this. [p1739r4] does much the same that this paper does, without the introduction of a concept to formalize the behavior it presents. In particular, it selects views which can realistically have their return types changed to match the input range and operations being performed (or a similarly powerful alternative) by asking whether they can be called with a function called
from a subrange of the iterators with the expressions acted upon.
-
Ranges should be reconstructible from their iterators (or subrange of their iterators) where applicable;
-
and, reconstructible ranges serve a useful purpose in generic algorithms, including not losing information and returning it in a much more cromulent and desirable form.
Therefore, this paper formalizes that concept and provides an opt-in, user-overridable way to return a related "reconstructed" type from an tag, an iterator, and a sentinel.
3. Design
The design is given in 2 concepts added to the standard:
template < class R , class It = ranges :: iterator_t < remove_reference_t < R >> , class Sen = ranges :: sentinel_t < remove_reference_t < R >>> concept pair_reconstructible_range = ranges :: range < R > && ranges :: borrowed_range < remove_reference_t < R >> && requires ( It first , Sen last ) { reconstruct ( in_place_type < remove_cvref_t < R >> , std :: move ( first ), std :: move ( last ) ); }; template < class R , class Range = remove_reference_t < R >> concept reconstructible_range = ranges :: range < R > && ranges :: borrowed_range < remove_reference_t < R >> && requires ( Range first_last ) { reconstruct ( in_place_type < remove_cvref_t < R >> , std :: move ( first_last ) ); };
It is the formalization that a range can be reconstructed from its begin iterator and end iterator/sentinel. It also provides a concept for allowing a range to be put back together from another range, which may be useful for some types of ranges where more than the iterator and sentinel are required. This allows a developer to propagate the input type’s properties after modifying its iterators for some underlying work, algorithm or other effect. This concept is also the basis of the idea behind [p1739r4], which was accepted in a lesser form for C++20 with the intent of this paper following up on it.
Both concepts require that the type with any references removed model the concept
. This ensures that the validity of the iterators is in fact independent of the lifetime of the range they originate from and that a "reconstructed" range does not depend on the original. We remove reference before performing this check, because all reference types that model
also model
and the intent of the proposed changes is narrower: (re)construction is assumed to be in constant time (this typically implies that
also models
, but it is sufficient to check
).
Note that this explicitly excludes types like
from being reconstructible.
3.1. Multiple, Safe Reconstruction Points
Consider a hypothetical
type. It is impossible to, from normal
iterator pair ascertain the range is null-terminated and is thus a "C string". Therefore, a reconstruct for
would not return a
, but a normal
.
On the other hand, if there was a
sentinel with a
iterator that was preserved through a series of actions and algorithms, that kind of information would allow us to return a
from a reconstruction operation.
The
extension point supports both of these scenarios:
class c_string_view { /* blah... */ // reconstruct: with [const char*, const char*) iterators // no guarantee it’s null terminated: decay to string_view friend constexpr std :: string_view reconstruct ( std :: in_place_type_t < c_string_view > , const char * first , const char * last ) { return std :: string_view ( first , last ); } // reconstruct: with static [const char*, c_string_sentinel) // static guarantee by type: return a c_string_view friend constexpr c_string_view reconstruct ( std :: in_place_type_t < c_string_view > , const char * first , c_string_sentinel ) { return c_string_view ( first ); } };
This is a level of flexibility that is better than the Revision 0 design, and can aid in providing better static guarantees while decaying gracefully in other situations.
3.2. Should this apply to all Ranges?
Not all ranges can meet this requirement. Some ranges contain state which cannot be trivially propagated into the iterators, or state that cannot be reconstructed from the iterator/sentinel pair itself. However, most of the common ranges representing unbounded views, empty views, iterations viewing some section of non-owned storage, or similar can all be reconstructed from their iterator/iterator or iterator/sentinel pair.
For example
contains a exposition *semiregular-box* template type (ranges.semi.wrap) which holds a value to iterate over. It would not be possible to reconstruct the exact same range (e.g., iterators pointing to the exact same object) with the semi-regular wrapper.
Finally, there are ranges which could be reconstructible by just the definition of a constructor that takes iterators or a
. While this was a benefit in that the majority of conforming ranges were reconstructible, it made a situation where someone could write a strange range. For example, consider this alternative implementation of
:
class pop_front_view { private : int * begin_ , * end_ ; public : pop_front_view () = default ; pop_front_view ( int * begin , int * end ) : begin_ ( begin == end ? begin : begin + 1 ), end_ ( end ) {} int * begin () const { return begin_ ; } int * end () const { return end_ ; } };
Reconstructing this range using the constructors is a surefire way to have a developer scratching at their head, wondering what is going on. Therefore, rather than require reconstruction through the constructor, we rely instead on an Customization Point Object design instead.
3.3. Applicability
There are many ranges to which this is applicable, but only a handful in the standard library need or satisfy this. If [p1391r2] and [p1394r2] are accepted (they were), then the two most important view types --
and
-- will model the _spirit_ of the concept (but lack the customization point to make it generic).
already fits this as well. By formalizing concepts in the standard, we can dependably and reliably assert that these properties continue to hold for these ranges. We intend to add a
function to all of the non-
types.
There are also upcoming ranges from [range-v3] and elsewhere that could model this concept:
-
[p1255r4]'s
;std :: ranges :: ref_maybe_view -
[p0009r9]'s
;std :: mdspan -
and, soon to be proposed by this author for the purposes of output range algorithms, [range-v3]'s
.ranges :: unbounded_view
And there are further range adaptor closure objects that could make use of this concept:
-
,views :: slice
,views :: take_exactly
andviews :: drop_exactly
from [range-v3]views :: take_last
Note that these changes will greatly aid other algorithm writers who want to preserve the same input ranges. In the future, the standard may provide an
algorithm for these types.
3.4. Two Concepts
By giving these ranges
, and/or
reconstruction functions, we can enable a greater degree of interface fidelity without having to resort to
for all generic algorithms or similar input/output iterator holders as we have in much of the current
library.
reconstruction is important for building
,
,
, and other types that go into algorithms.
because one-argument functions can have extremely overloaded meanings. It also produces less compiler boilerplate to achieve the same result of reconstructing the range when one does not have to go through
. However, it is important to attempt to move away from the iterator, sentinel model being deployed all the time:
offers a single type that can accurately represent the intent and can be fairly easy to constrain overload sets on (most of the time) due to being a single family of types (anyone can check for a the type being a specialization of
, whereas iterators span multiple different families of types).
This paper includes two concepts that cover both reconstructible methods.
4. Deployment Experience
This change was motivated by Hannes Hauswedell’s [p1739r4] and became very important for the ranges work done with text encoding. There is a C++17 implementation at the soasis/text repository here, which is mimicking the interface needed for [p1629r1].
5. Impact
As a feature that is opt-in thanks to the
Customization Point Design, no currently created range or previously made range is permanently affected.
Furthermore, this is a new and separate concept. It is not to be added to the base
concept, or added to any other concept. It is to be applied separately to the types which can reasonably support it for the benefit of algorithms and code which can enhance the quality of their implementation.
Finally, Hannes Hauswedell’s [p1739r4] with the explicit intention to mark certain ranges as reconstructible by hardcoding their behavior into the standard and come back with an opt-in fix during the C++23 cycle. This paper completes that promise.
6. Proposed Changes
The following wording is relative to the latest C++ Draft paper.
6.1. Feature Test Macro
This paper results in a concept to help guide the further development of standard ranges and simplify their usages in generic contexts. There is one proposed feature test macro,
, which is to be input into the standard and then explicitly updated every time a
from a is added to reflect the new wording. We hope that by putting this in the standard early, most incoming ranges will be checked for compatibility with
and
.
6.2. Intent
The intent of this wording is to provide greater generic coding guarantees and optimizations by allowing for a class of ranges and views that model the new exposition-only definitions of a reconstructible range:
-
add a new feature test macro for reconstructible ranges to cover constructor changes;
-
add a new customization point object for
,ranges :: reconstruct -
and, add two new concepts to [range.req].
If borrowed_range is changed to the
concept name, then this entire proposal will rename all its uses of borrowed_range.
For ease of reading, the necessary portions of other proposal’s wording is duplicated here, with the changes necessary for the application of reconstructible range concepts. Such sections are clearly marked.
6.3. Proposed Library Wording
Add a feature test macro
.
Insert into §24.2 Header
Synopsis [ranges.syn] a new customization point object in the inline namespace:
namespace std :: ranges {
inline namespace unspecified {
…
nline constexpr nspecified reconstruct = unspecified ;
…
}
}
Insert into §24.4.2 Ranges [range.range]'s after paragraph 7, one additional paragraph:
8 The conceptsand
pair_reconstructible_range concepts describe the requirements on ranges that are efficiently constructible from values of their iterator and sentinel types.
reconstructible_range template < class R , class It = ranges :: iterator_t < remove_reference_t < R >> , class Sen = ranges :: sentinel_t < remove_reference_t < R >>> concept pair_reconstructible_range = ranges :: range < R > && ranges :: borrowed_range < remove_reference_t < R >> && requires ( It first , Sen last ) { reconstruct ( in_place_type < remove_cvref_t < R >> , std :: move ( first ), std :: move ( last ) ); }; template < class R , class Range = remove_reference_t < R >> concept reconstructible_range = ranges :: range < R > && ranges :: borrowed_range < remove_reference_t < R >> && requires ( Range first_last ) { reconstruct ( in_place_type < remove_cvref_t < R >> , std :: move ( first_last ) ); }; 9 Let
be a range with type
r .
R
- 9.1 — Let
be the result of
re if such an expression is well-formed.
ranges :: reconstruct ( in_place_type < remove_cvref_t < R >> , ranges :: begin ( r ), ranges :: end ( r )) models
r if
ranges :: pair_reconstructible_range
- —
is true, and
ranges :: begin ( r ) == ranges :: begin ( re ) - —
is true.
ranges :: end ( r ) == ranges :: end ( re ) - 9.2 — Let
be the result of
sub_re , if such an expression is well-formed. Then
ranges :: reconstruct ( in_place_type < remove_cvref_t < R >> , std :: move ( r )) models
sub_re if
ranges :: reconstructible_range
- —
is true, and
ranges :: begin ( r ) == ranges :: begin ( sub_re ) - —
is true.
ranges :: end ( r ) == ranges :: end ( sub_re )
Insert a new sub-clause "§24.3.13
[range.prim.recons]", after "§24.3.12
[range.prim.cdata]":
24.3.12
[range.prim.recons]
ranges :: reconstruct 1 The name
denotes a customization point object.
reconstruct 2 The expression
for some type
ranges :: reconstruct ( in_place_type < R > , I , S ) and some sub-expressions
R and
I is expression-equivalent to:
S
- (2.1)
if it is a valid expression and
reconstruct ( in_place_type < R > , std :: move ( I ), std :: move ( S )) ,
R , and
decltype ( I ) model
decltype ( S ) .
pair_reconstructible_range - (2.2) Otherwise,
if it is a valid expression and
reconstruct ( in_place_type < R > , ranges :: subrange < remove_cvref_t < decltype ( I ) > , remove_cvref_t < decltype ( S ) >> ( std :: move ( I ), std :: move ( S ))) ,
R , and
decltype ( I ) model
decltype ( S ) .
reconstructible_range - (2.3)
if it is a valid expression.
ranges :: subrange < remove_cvref_t < decltype ( I ) > , remove_cvref_t < decltype ( S ) >> ( std :: move ( I ), std :: move ( S )) - (2.4) Otherwise,
is ill-formed.
ranges :: reconstruct ( std :: in_place_type < R > , I , S ) 3 Let
be some sub-expression. The expression
SR for some type
ranges :: reconstruct ( in_place_type < R > , SR ) and sub-expression
R is expression-equivalent to:
SR
- (2.1)
if it is a valid expression, and
reconstruct ( in_place_type < R > , std :: move ( SR )) and
R model
decltype ( SR ) .
reconstructible_range - (2.2) Otherwise,
if it is a valid expression and
reconstruct ( in_place_type < R > , ranges :: begin ( std :: move ( SR )), ranges :: end ( std :: move ( SR ))) ,
R , and
range :: iterator_t < decltype ( SR ) > model
range :: sentinel_t < decltype ( SR ) > .
pair_reconstructible_range - (2.3) Otherwise,
.
return SR
Add to "§21.4.3.1
" [string.view.template.general] a
function ADL extension point for
:
// [string.view.reconstruct] friend constexpr basic_string_view reconstruct ( const_iterator first , const_iterator last ) noexcept ;
Add a new subclause "§21.4.���
" [string.view.reconstruct] in §21.4:
21.4.��� Range Reconstruction [string.view.reconstruct]
friend constexpr basic_string_view reconstruct ( const_iterator first , const_iterator last ) noexcept ; 1 Returns:
.
basic_string ( std :: move ( first ), std :: move ( last ))
Add to "§22.7.3.1
" [span.overview] a
function ADL extension point for
:
// [span.reconstruct] template < class It , class End > friend constexpr auto reconstruct ( It first , End last ) noexcept ;
Add a new subclause "§22.7.3.����
" [span.reconstruct] in §22.7.3:
22.7.3.���� Range Reconstruction [span.reconstruct]
template < class It , class End > friend constexpr auto reconstruct ( const_iterator first , const_iterator last ) noexcept ; 1 Constraints: Let
be
U .
remove_ reference_ t < iter_ reference_ t < It >>
- (1.1) —
is
is_ convertible_ v < U ( * )[], element_ type ( * )[] > true
. [Note 2: The intent is to allow only qualification conversions of the iterator reference type to element_type. — end note]- (1.2) —
satisfies
It .
contiguous_ iterator - (1.3) —
satisfies
End .
sized_ sentinel_ for < It > - (1.4) —
is
is_ convertible_ v < End , size_ t > false
.2 Let
be an implementation-defined constant expression of type
MaybeExtent . If it is not equal to
size_t , then
dynamic_extent shall be a value equal to
MaybeExtent .
distance ( first , last ) 3 Returns:
.
span < ElementType , MaybeExtent > ( std :: move ( first ), std :: move ( last )) 4 Remarks: the return type can be promoted to a
with a static extent if the implementation is capable of deriving such information from the given iterator and sentinel.
span
Add to "§24.5.4.1
" [range.subrange.general] a
function ADL extension point for
:
template < class It , class End > friend constexpr auto reconstruct ( It first , End last ) noexcept ;
Add a new subclause "§24.5.4.����
" [range.subrange.reconstruct] in §24.5.4:
22.7.3.���� Range Reconstruction [range.subrange.reconstruct]
friend constexpr subrange reconstruct ( I first , S last ) noexcept ; 1 Returns:
.
subrange ( std :: move ( first ), std :: move ( last ))
Add to "§24.6.4.2 Class template
" [range.iota] a
function ADL extension point for
:
friend constexpr iota_view reconstruct ( iterator first , sentinel last ) noexcept ;
Add one new paragraph to "§24.6.4.2 Class template
" [range.iota]:
friend constexpr iota_view reconstruct ( iterator first , sentinel last ) noexcept ; �� Returns:
.
iota_view ( std :: move ( first ), std :: move ( last ))
Add to "§24.6.2.2 Class template
" [range.empty.view] a
function ADL extension point for
:
friend constexpr empty_view reconstruct ( iterator , sentinel ) noexcept { return empty_view (); }
Modify "§24.7.7.1 Overview " [range.take.overview] for
's paragraph 2:
2 The name
denotes a range adaptor object ([range.adaptor.object]). Let
views :: take and
E be expressions, let
F be
T , and let
remove_ cvref_ t < decltype (( E )) > be
D . If
range_ difference_ t < decltype (( E )) > does not model
decltype (( F )) ,
convertible_ to < D > is ill-formed. Otherwise, the expression views::take(E, F) is expression-equivalent to:
views :: take ( E , F )
- (2.1) — If
is a specialization of
T ([range.empty.view]), then
ranges :: empty_ view .
(( void ) F , decay - copy ( E )) - (2.2) — Otherwise, if
models
T and
random_ access_ range and is
sized_ range then
- (2.2.1) — a specialization of
([views.span]) where
span ,
T :: extent == dynamic_ extent - (2.2.2) — a specialization of
([string.view]),
basic_ string_ view - (2.2.3) — a specialization of
([range.iota.view]), or
ranges :: iota_ view - (2.2.4) — a specialization of
([range.subrange]),
ranges :: subrange , except that
T { ranges :: begin ( E ), ranges :: begin ( E ) + min < D > ( ranges :: size ( E ), F )} is evaluated only once.
E - (2.3) — Otherwise,
.
ranges :: take_ view { E , F }
- (2.1) — If
is a specialization of
T ([range.empty.view]), then
ranges :: empty_ view .
(( void ) F , decay - copy ( E )) - (2.2) — Otherwise, if
models
T ,
random_ access_ range , and
sized_ range , then
reconstructible_range , except that
T { ranges :: begin ( E ), ranges :: begin ( E ) + min < D > ( ranges :: size ( E ), F )} is evaluated only once.
E - (2.3) — Otherwise,
.
ranges :: take_ view { E , F }
Modify "§24.7.9.1 Overview " [range.take.overview] for
's paragraph 2:
2 The name
denotes a range adaptor object ([range.adaptor.object]). Let
views :: drop and
E be expressions, let
F be
T , and let
remove_ cvref_ t < decltype (( E )) > be
D . If
range_ difference_ t < decltype (( E )) > does not model
decltype (( F )) ,
convertible_ to < D > is ill-formed. Otherwise, the expression
views :: drop ( E , F ) is expression-equivalent to:
views :: drop ( E , F )
- (2.1) — If
is a specialization of
T ([range.empty.view]), then
ranges :: empty_ view .
(( void ) F , decay - copy ( E )) - (2.2) — Otherwise, if
models
T and
random_ access_ range and is
sized_ range then
- (2.2.1) — a specialization of
([views.span]) where
span ,
T :: extent == dynamic_ extent - (2.2.2) — a specialization of
([string.view]),
basic_ string_ view - (2.2.3) — a specialization of
([range.iota.view]), or
ranges :: iota_ view - (2.2.4) — a specialization of
([range.subrange]),
ranges :: subrange , except that
T { ranges :: begin ( E ) + min < D > ( ranges :: size ( E ), F ), ranges :: end ( E )} is evaluated only once.
E - (2.3) — Otherwise,
.
ranges :: drop_ view { E , F }
- (2.1) — If
is a specialization of
T ([range.empty.view]), then
ranges :: empty_ view .
(( void ) F , decay - copy ( E )) - (2.2) — Otherwise, if
models
T ,
random_ access_ range , and
sized_ range , then
reconstructible_range , except that
T { ranges :: begin ( E ) + min < D > ( ranges :: size ( E ), F ), ranges :: end ( E )} is evaluated only once.
E - (2.3) — Otherwise,
.
ranges :: drop_ view { E , F }
7. Acknowledgements
Thanks to Corentin Jabot, Christopher DiBella, and Hannes Hauswedell for pointing me to [p1035r7] and [p1739r4] to review both papers and combine some of their ideas in here. Thanks to Eric Niebler for prompting me to think of the generic, scalable solution to this problem rather than working on one-off fixes for individuals views.
Thank you to Oktal, Anointed of ADL, Blessed Among Us, and Morwenn, the ever-watching Code Guardian for suggesting improvements to the current concept form.