P2374R4
views::cartesian_product

Published Proposal,

This version:
http://wg21.link/P2374
Authors:
Sy Brand (sy.brand at microsoft dot com)
Michał Dominiak (griwes at griwes dot info)
Audience:
LWG
Project:
ISO/IEC JTC1/SC22/WG21 14882: Programming Language — C++

Abstract

This paper proposes std::ranges::cartesian_product_view for taking the cartesian product of multiple forward ranges.

1. Changelog

1.1. Changes from r3

LWG review comments:

LWG review comments, second round:

LWG review comments, mailing list review:

LWG review comments, third round:

1.2. Changes from r2

1.3. Changes from r1

1.4. Changes from r0

2. Motivation

Cartesian product is a fundamental mathematical construct. There should be a cartesian_product_view which generates the cartesian product of any number of ranges, e.g.:

Before After
std::vector<int> a,b,c;
for (auto&& ea : a) {
    for (auto&& eb : b) {
        for (auto&& ec : c) {
            use(ea,eb,ec);
        }
    }
}
std::vector<int> a,b,c;
for (auto&& [ea,eb,ec] : std::views::cartesian_product(a,b,c)) {
    use(ea,eb,ec);
}

This is especially useful for composing with other views, or dealing with parameter packs of ranges:

Before After
template <std::size_t N = 0, class F,
          class Res, class Tuple, class... Args>
auto find_tuples_satisfying_impl(
    F f, Res& res, Tuple const& ranges, Args const&... args)
requires(N == tuple_size_v<std::remove_cvref_t<Tuple>>) {
  if (std::invoke(f, args...)) {
    res.push_back(std::make_tuple(args...));
  }
}

template <std::size_t N = 0, class F,
          class Res, class Tuple, class... Args>
auto find_tuples_satisfying_impl(
    F f, Res& res, Tuple const& ranges, Args const&... args) {
  for (auto&& e : std::get<N>(ranges)) {
    find_tuples_satisfying_impl<N+1>(f, res, ranges, args..., e);
  }
}

template <class F, std::ranges::forward_range... Vs>
requires(std::regular_invocable<
          F, std::ranges::range_reference_t<Vs>...>)
auto find_tuples_satisfying(F f, Vs&&... vs) {
  std::vector<tuple<std::ranges::range_value_t<Vs>...>> res;
  find_tuples_satisfying_impl(
        f, res, tuple(std::forward<Vs&&>(vs)...));
  return res;
}
template <class F, std::ranges::forward_range... Vs>
requires(std::regular_invocable<F,
          std::ranges::range_reference_t<Vs>...>)
auto find_tuples_satisfying(F f, Vs&&... vs) {
  return std::views::cartesian_product(std::forward<Vs>(vs)...)
    | std::views::filter([f](auto&& tuple) { return std::apply(f, tuple); });)
    | std::ranges::to<std::vector>(); //given P1206
}

3. Design

3.1. Minimum Range Requirements

This paper requires all ranges, except the first one, passed to cartesian_product_view to be forward ranges. See Initial Range Argument Special-Casing for discussion on the first argument being an input range.

3.2. Tuple or pair

A potential design would be to use tuple as the value_type and reference_type of cartesian_product_view. This paper uses std::pair if two ranges are passed and tuple otherwise. See p2321 for motivation.

3.3. reference_type

This paper uses tuple-or-pair<ranges::reference_t<Vs>...> as the reference type. See p2214 for discussion of value types (particularly pair and tuple) as the reference_type for ranges, and p2321 for wording of improvements for key use-cases.

3.4. Empty cartesian product view

Trying to take the cartesian product view of 0 views will produce an empty_view<tuple<>>, in parity with Range-v3 and p2321.

3.5. Common Range

cartesian_product_view can be a common range if the first range either is common, or is sized and random access. This paper reflects this.

3.6. Bidirectional Range

cartesian_product_view can be a bidirectional range if the underlying ranges, except for the first one, are bidirectional and common, or if they are random access and sized. Non-common bidirectional ranges are not supported because decrementing when one of the iterators is at the beginning of its range would require advancing the iterator to end, which may be linear in complexity.

We don’t consider non-common, random access, sized ranges as worth supporting, so this paper requires bidirectional and common.

3.7. Random Access Range

cartesian_product_view can be a random access range if all the underlying ranges are random access and sized, with the second requirement not appling to the first range. Sized ranges are required because when the view is incremented, the new states of the iterators are calculated modulo the size of their views.

We can’t think of too many use-cases for this and it adds a fair bit of implementation burden, but this paper supports the necessary operations.

3.8. Initial Range Argument Special-Casing

The first range passed to cartesian_product_view can be treated specially, since it is only passed through a single time. Therefore, the specification relaxes several constraints on the first range passed:

Previous revisions of this paper didn’t propose this feature, however the Ranges SG has requested that it be added to the paper.

3.9. Sized Range

cartesian_product_view can be a sized range if all the underlying ranges are, in which case the size is the product of all underlying sizes. This is reflected in the paper.

3.10. Naming

An alternative name is std::ranges::product_view. This paper uses cartesian_product_view as we believe it is more explicit in what its semantics are.

3.11. Pipe Support

It may be possible to support syntax such as vec1 | views::cartesian_product(vec2) by either fixing the number of arguments allowed to the view, or adding a pipe operator to cartesian_product_view itself.

However, it’s problematic for the same reason as views::zip, in that one cannot distinguish between the usages in a | views::cartesian_product(b, c) and views::cartesian_product(b,c) on its own. As such, this paper does not support this syntax.

3.12. Borrowed Range?

It is possible to implement cartesian_product_view in such a way that it is a borrowed range if all the underlying ranges are borrowed. However, this requires that every iterator of the view stores two iterators per every range used for the cartesian product - one for the current position within that range, and another for the begin iterator, to allow wrapping around back to the first element when the end of that range is reached. (For random access underlying ranges, it is conceivable to store an iterator and an offset, but this is not applicable in general.) If cartesian_product_view is not a borrowed view, the iterators only require storing one iterator per every range, and a pointer to the view itself.

Due to this, this paper does not make cartesian_product_view a borrowed range.

4. Implementation

There are implementations of a cartesian product view in Range-v3, cor3ntin::rangesnext, and tl::ranges, among others.

5. Wording

5.1. Addition to <ranges>

Add the following to 24.2 [ranges.syn], Header <ranges> synopsis:

// ...
namespace std::ranges {
    // ...

    // [range.cartesian], cartesian product view
    template<input_range First, forward_range... Vs>
        requires (view<First> && ... && view<Vs>)
    class cartesian_product_view;

    namespace views { inline constexpr unspecified cartesian_product = unspecified; }
}

5.2. Range adaptor helpers [range.adaptor.helpers]

Add a new section after Non-propagating cache [range.nonprop.cache]. Remove the definitions of tuple-or-pair, tuple-transform, and tuple-for-each from Class template zip_view [range.zip.view] and add them to this new section:

namespace std::ranges {
    template <class... Ts>
    using tuple-or-pair = see-below;                     // exposition only

    template<class F, class Tuple>
    constexpr auto tuple-transform(F&& f, Tuple&& tuple) { // exposition only
        return apply([&]<class... Ts>(Ts&&... elements) {
            return tuple-or-pair<invoke_result_t<F&, Ts>...>(
                invoke(f, std::forward<Ts>(elements))...
            );
        }, std::forward<Tuple>(tuple));
    }

    template<class F, class Tuple>
    constexpr void tuple-for-each(F&& f, Tuple&& tuple) { // exposition only
        apply([&]<class... Ts>(Ts&&... elements) {
            (invoke(f, std::forward<Ts>(elements)), ...);
        }, std::forward<Tuple>(tuple));
    }
}

Given some pack of types Ts, the alias template tuple-or-pair is defined as follows:

  1. If sizeof...(Ts) is 2, tuple-or-pair<Ts...> denotes pair<Ts...>.

  2. Otherwise, tuple-or-pair<Ts...> denotes tuple<Ts...>.

5.3. Cartesian product view [range.cartesian]

5.3.1. Overview [range.cartesian.overview]

  1. cartesian_product_view takes any non-zero number of ranges n and produces a view of tuples calculated by the n-ary cartesian product of the provided ranges.

  2. The name views::cartesian_product denotes a customization point object. Given a pack of subexpressions Es..., the expression views::cartesian_product(Es...) is expression-equivalent to

[Example:

std::vector<int> v { 0, 1, 2 };
for (auto&& [a,b,c] : std::views::cartesian_product(v, v, v)) {
  std::cout << a << ' ' << b << ' ' << c << '\n';
  //0 0 0
  //0 0 1
  //0 0 2
  //0 1 0
  //0 1 1
  //...
}
-- end example ]

5.3.2. Class template cartesian_product_view [range.cartesian.view]

namespace std::ranges {
  template <bool Const, class First, class... Vs>
  concept cartesian-product-is-random-access = // exposition only
    (random_access_range<maybe-const<Const, First>> && ... &&
      (random_access_range<maybe-const<Const, Vs>>
        && sized_range<maybe-const<Const, Vs>>));

  template <class R>
  concept cartesian-product-common-arg = // exposition only
    common_range<R> || (sized_range<R> && random_access_range<R>);

  template <bool Const, class First, class... Vs>
  concept cartesian-product-is-bidirectional = // exposition only
    (bidirectional_range<maybe-const<Const, First>> && ... &&
      (bidirectional_range<maybe-const<Const, Vs>>
        && cartesian-product-common-arg<maybe-const<Const, Vs>>));

  template <class First, class... Vs>
  concept cartesian-product-is-common = // exposition only
    cartesian-product-common-arg<Const, First>>;

  template <class... Vs>
  concept cartesian-product-is-sized = // exposition only
    (sized_range<Vs> && ...);

  template<bool Const, template<class> class FirstSent, typename First, typename... Vs>
    concept cartesian-is-sized-sentinel = // exposition only
      (sized_sentinel_for<FirstSent<maybe-const<Const, First>,
          iterator_t<maybe-const<Const, First>> && ...
        && (sized_range<maybe-const<Const, Vs>>
          && sized_sentinel_for<iterator_t<maybe-const<Const, Vs>>,
              iterator_t<maybe-const<Const, Vs>>>));

  template <cartesian-product-common-arg R>
  constexpr auto cartesian-common-arg-end(R & r) {
    if constexpr (common_range<R>) {
      return ranges::end(r);
    }
    else {
      return ranges::begin(r) + ranges::distance(r);
    }
  }

  template <input_range First, forward_range... Vs>
    requires (view<First> && ... && view<Vs>)
  class cartesian_product_view
  : public view_interface<cartesian_product_view<First, Vs...>> {
  private:
    tuple<First, Vs...> bases_; // exposition only

    template<bool Const>
    struct iterator; // exposition only
  public:
    constexpr cartesian_product_view() = default;
    constexpr explicit cartesian_product_view(First first_base, Vs... bases);

    constexpr iterator<false> begin()
      requires (!simple-view<First> || ... || !simple-view<Vs>);
    constexpr iterator<true> begin() const
      requires (range<const First> && ... && range<const Vs>);

    constexpr iterator<false> end()
      requires ((!simple-view<First> || ... || !simple-view<Vs>) &&
        cartesian-product-is-common<First, Vs...>);
    constexpr iterator<true> end() const
      requires cartesian-product-is-common<const First, const Vs...>;
    constexpr default_sentinel_t end() const noexcept;

    constexpr see below size()
      requires cartesian-product-is-sized<First, Vs...>;
    constexpr see below size() const
      requires cartesian-product-is-sized<const First, const Vs...>;
  };

  template <class... Vs>
  cartesian_product_view(Vs&&...)->cartesian_product_view<all_t<Vs>...>;

  namespace views { inline constexpr unspecified cartesian_product = unspecified; }
}
constexpr explicit cartesian_product_view(First first_base, Vs... bases);
  1. Effects: Initialises bases_ with std::move(first_base), std::move(bases)....

constexpr iterator<false> begin()
  requires (!simple-view<First> || ... || !simple-view<Vs>);
  1. Effects: Equivalent to: return iterator<false>(tuple-transform(ranges::begin, bases_));

constexpr iterator<true> begin() const
  requires (range<const First> && ... && range<const Vs>);
  1. Effects: Equivalent to: return iterator<true>(tuple-transform(ranges::begin, bases_));

constexpr iterator<false> end()
  requires ((!simple-view<First> || ... || !simple-view<Vs>)
    && cartesian-product-is-common<First, Vs...>);
constexpr iterator<true> end() const
  requires cartesian-product-is-common<const First, const Vs...>;
  1. Let:

  1. Effects: Equivalent to:

iterator<is-const> it(tuple-transform(
  [](auto & rng){ return begin-or-first-end(rng); }, bases_));
return it;
constexpr default_sentinel_t end() const noexcept;
  1. Returns: default_sentinel.

constexpr see below size()
  requires cartesian-product-is-sized<First, Vs...>;
constexpr see below size() const
  requires cartesian-product-is-sized<const First, const Vs...>;
  1. The return type is an implementation-defined unsigned-integer-like type.

  2. Recommended practice: The return type should be the smallest unsigned-integer-like type that is sufficiently wide to store the product of the maximum sizes of all the underlying ranges, if such a type exists.

  3. Let p be the product of the sizes of all the ranges in bases_.

  4. Preconditions: p can be represented by the return type.

  5. Returns: p.

5.3.3. Class template cartesian_product_view::iterator [ranges.cartesian.iterator]

namespace std::ranges {
template<input_range First, forward_range... Vs>
requires (view<First> && ... && view<Vs>)
template<bool Const>
class cartesian_product_view<First, Vs...>::iterator {
public:
  using iterator_category = input_iterator_tag;
  using iterator_concept  = see below;
  using value_type = tuple-or-pair<range_value_t<maybe-const<Const, First>>,
    range_value_t<maybe-const<Const, Vs>>...>;
  using reference = tuple-or-pair<reference_t<maybe-const<Const, First>>,
    reference_t<maybe-const<Const, Vs>>...>;
  using difference_type = see below;

  iterator() requires forward_range<maybe-const<Const, First>> = default;

  constexpr iterator(iterator<!Const> i) requires Const &&
    (convertible_to<iterator_t<First>, iterator_t<maybe-const<Const, First>>> &&
      ... && convertible_to<iterator_t<Vs>, iterator_t<maybe-const<Const, Vs>>>);

  constexpr auto operator*() const;
  constexpr iterator& operator++();
  constexpr void operator++(int);
  constexpr iterator operator++(int) requires forward_range<maybe-const<Const, First>>;

  constexpr iterator& operator--()
    requires cartesian-product-is-bidirectional<Const, First, Vs...>;
  constexpr iterator operator--(int)
    requires cartesian-product-is-bidirectional<Const, First, Vs...>;

  constexpr iterator& operator+=(difference_type x)
    requires cartesian-product-is-random-access<Const, First, Vs...>;
  constexpr iterator& operator-=(difference_type x)
    requires cartesian-product-is-random-access<Const, First, Vs...>;

  constexpr reference operator[](difference_type n) const
    requires cartesian-product-is-random-access<Const, First, Vs...>;

  friend constexpr bool operator==(const iterator& x, const iterator& y)
    requires equality_comparable<iterator_t<maybe-const<Const, First>>>;

  friend constexpr bool operator==(const iterator& x, default_sentinel_t);

  friend constexpr auto operator<=>(const iterator& x, const iterator& y)
    requires all-random-access<Const, First, Vs...>;

  friend constexpr iterator operator+(const iterator& x, difference_type y)
    requires cartesian-product-is-random-access<Const, First, Vs...>;
  friend constexpr iterator operator+(difference_type x, const iterator& y)
    requires cartesian-product-is-random-access<Const, First, Vs...>;
  friend constexpr iterator operator-(const iterator& x, difference_type y)
    requires cartesian-product-is-random-access<Const, First, Vs...>;
  friend constexpr difference_type operator-(const iterator& x, const iterator& y)
    requires cartesian-is-sized-sentinel<Const, iterator_t, First, Vs...>;

  friend constexpr difference_type operator-(iterator i, default_sentinel_t)
    requires cartesian-is-sized-sentinel<Const, sentinel_t, First, Vs...>;
  friend constexpr difference_type operator-(default_sentinel_t, iterator i)
    requires cartesian-is-sized-sentinel<Const, sentinel_t, First, Vs...>;

  friend constexpr auto iter_move(const iterator& i) noexcept(see below);

  friend constexpr void iter_swap(const iterator& l, const iterator& r) noexcept(see below)
    requires (indirectly_swappable<iterator_t<maybe-const<Const, First>>> && ... &&
      indirectly_swappable<iterator_t<maybe-const<Const, Vs>>>);

private:
  maybe-const<Const, cartesian_product_view>* parent_ = nullptr; // exposition only
  tuple-or-pair<iterator_t<maybe-const<Const, First>>,
    iterator_t<maybe-const<Const, Vs>>...> current_; // exposition only

  template <size_t N = sizeof...(Vs)>
  constexpr void next(); // exposition only

  template <size_t N = sizeof...(Vs)>
  constexpr void prev(); // exposition only

  template <class Tuple>
  constexpr difference_type distance-from(Tuple t); // exposition only

  constexpr explicit iterator(tuple-or-pair<iterator_t<maybe-const<Const, First>>,
    iterator_t<maybe-const<Const, Vs>>...> current); // exposition only
};
}
  1. iterator::iterator_concept is defined as follows:

  1. iterator::difference_type is an implementation-defined signed-integer-like type.

  2. Recommended practice: iterator::difference_type should be the smallest signed-integer-like type that is sufficiently wide to store the product of the maximum sizes of all underlying ranges if such a type exists.

template <size_t N = sizeof...(Vs)>
constexpr void next(); // exposition only
  1. Effects: Equivalent to:

auto& it = std::get<N>(current_);
++it;
if constexpr (N > 0) {
  if (it == ranges::end(get<N>(parent_->bases_))) {
    it = ranges::begin(std::get<N>(parent_->bases_));
    next<N - 1>();
  }
}
template <size_t N = sizeof...(Vs)>
constexpr void prev(); // exposition only
  1. Effects: Equivalent to:

auto& it = std::get<N>(current_);
if (it == ranges::begin(std::get<N>(parent_->bases_))) {
   it = cartesian-common-arg-end(std::get<N>(parent_->bases_));
   if constexpr (N > 0) {
       prev<N - 1>();
   }
}
--it;
template <class Tuple>
constexpr difference_type distance-from(Tuple t); // exposition only
  1. Let:

  1. Preconditions: scaled_sum can be represented by difference_type.

  2. Returns: scaled_sum.

constexpr explicit iterator(tuple-or-pair<iterator_t<maybe-const<Const, First>>,
  iterator_t<maybe-const<Const, Vs>>...> current);
  1. Effects: Initializes current_ with std::move(current).

constexpr iterator(iterator<!Const> i) requires Const &&
  (convertible_to<iterator_t<First>, iterator_t<maybe-const<Const, First>>> &&
    ... && convertible_to<iterator_t<Vs>, iterator_t<maybe-const<Const, Vs>>>);
  1. Effects: Initializes current_ with std::move(i.current_).

constexpr auto operator*() const;
  1. Effects: Equivalent to:

return tuple-transform([](auto& i) -> decltype(auto) { return *i; }, current_);
constexpr iterator& operator++();
  1. Effects: Equivalent to:

next();
return *this;
constexpr void operator++(int);
  1. Effects: Equivalent to ++*this.

constexpr iterator operator++(int) requires forward_range<maybe-const<Const, First>>;
  1. Effects: Equivalent to:

auto tmp = *this;
++*this;
return tmp;
constexpr iterator& operator--()
  requires cartesian-product-is-bidirectional<Const, First, Vs...>;
  1. Effects: Equivalent to:

prev();
return *this;
constexpr iterator operator--(int)
  requires cartesian-product-is-bidirectional<Const, First, Vs...>;
  1. Effects: Equivalent to:

auto tmp = *this;
--*this;
return tmp;
constexpr iterator& operator+=(difference_type x)
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Let orig be the value of *this before the call.

  2. Let ret be:

  1. Precondition: x is in the range [ranges::distance(*this, ranges::begin(*parent_)), ranges::distance(*this, ranges::end(*parent_))].

  2. Effects: Sets the value of *this to ret.

  3. Returns: *this.

  4. Complexity: Constant.

constexpr iterator& operator-=(difference_type x)
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to:

*this += -x;
return *this;
constexpr reference operator[](difference_type n) const
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to: return *((*this) + n);

friend constexpr bool operator==(const iterator& x, const iterator& y)
  requires equality_comparable<iterator_t<maybe-const<Const, First>>>
  1. Effects: Equivalent to: return x.current_ == y.current_;

friend constexpr bool operator==(const iterator& x, default_sentinel_t);
  1. Returns: true if std::get<i>(x.current_) == ranges::end(std::get<i>(x.parent_->bases_)) is true for any integer 0 ≤ i ≤ sizeof...(Vs); otherwise, false.

friend constexpr auto operator<=>(const iterator& x, const iterator& y)
  requires all-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to: return x.current_ <=> y.current_;

friend constexpr iterator operator+(const iterator& x, difference_type y)
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to: return iterator(x) += y;

friend constexpr iterator operator+(difference_type x, const iterator& y)
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to: return y + x;

friend constexpr iterator operator-(const iterator& x, difference_type y)
  requires cartesian-product-is-random-access<Const, First, Vs...>;
  1. Effects: Equivalent to: return iterator(x) -= y;

friend constexpr difference_type operator-(const iterator& x, const iterator& y)
  requires cartesian-is-sized-sentinel<Const, iterator_t, First, Vs...>;
  1. Effects: Equivalent to: return x.distance-from(y.current_);

friend constexpr difference_type operator-(iterator i, default_sentinel_t)
  requires cartesian-is-sized-sentinel<Const, sentinel_t, First, Vs...>;
  1. Let end-tuple be an object of a type that is a specialization of tuple, such that:

  1. Effects: Equivalent to: return i.distance-from(end-tuple);

friend constexpr difference_type operator-(default_sentinel_t s, iterator i)
  requires cartesian-is-sized-sentinel<Const, sentinel_t, First, Vs...>;
  1. Effects: Equivalent to: return -(i - s);

friend constexpr auto iter_move(const iterator& i) noexcept(see below);
  1. Effects: Equivalent to: return tuple-transform(ranges::iter_move, i.current_);

  2. Remarks: The exception specification is equivalent to the logical AND of the following expressions:

friend constexpr void iter_swap(const iterator& l, const iterator& r) noexcept(see below)
    requires (indirectly_swappable<iterator_t<maybe-const<Const, First>>> && ... &&
        indirectly_swappable<iterator_t<maybe-const<Const, Vs>>>);
  1. Effects: For every integer 0 ≤ i ≤ sizeof...(Vs), performs: ranges::iter_swap(std::get<i>(l.current_), std::get<i>(r.current_)).

  2. Remarks: The exception specification is equivalent to the logical AND of the following expressions: noexcept(ranges::iter_swap(std::get<i>(l.current_), std::get<i>(r.current_))) for every integer 0 ≤ i ≤ sizeof...(Vs).

5.4. Feature-test macro

Add the following macro definition to 17.3.2 [version.syn], Header <version> synopsis, with the value selected by the editor to reflect the date of adoption of this paper:

#define __cpp_lib_ranges_cartesian_product 20XXXXL // also in <ranges>

6. Acknowledgements

Thank you to Christopher Di Bella, Corentin Jabot, Tim Song, and Barry Revzin for feedback and guidance.

Thank you to Tomasz Kamiński for help with getting the wording into a good shape.