P2300R2
std::execution

1. Introduction

This paper proposes a self-contained design for a Standard C++ framework for managing asynchronous execution on generic execution contexts. It is based on the ideas in [P0443R14] and its companion papers.

1.1. Motivation

Today, C++ software is increasingly asynchronous and parallel, a trend that is likely to only continue going forward. Asynchrony and parallelism appears everywhere, from processor hardware interfaces, to networking, to file I/O, to GUIs, to accelerators. Every C++ domain and every platform need to deal with asynchrony and parallelism, from scientific computing to video games to financial services, from the smallest mobile devices to your laptop to GPUs in the world’s fastest supercomputer.

While the C++ Standard Library has a rich set concurrency primitives (std::atomic, std::mutex, std::counting_semaphore, etc) and lower level building blocks (std::thread, etc), we lack a Standard vocabulary and framework for asynchrony and parallelism that C++ programmers desperately need. std::async/std::future/std::promise, C++11’s intended exposure for asynchrony, is inefficient, hard to use correctly, and severely lacking in genericity, making it unusable in many contexts. We introduced parallel algorithms to the C++ Standard Library in C++17, and while they are an excellent start, they are all inherently synchronous and not composable.

This paper proposes a Standard C++ model for asynchrony, based around three key abstractions: schedulers, senders, and receivers, and a set of customizable asynchronous algorithms.

1.2. Priorities

• Be composable and generic, allowing users to write code that can be used with many different types of execution contexts.

• Encapsulate common asynchronous patterns in customizable and reusable algorithms, so users don’t have to invent things themselves.

• Make it easy to be correct by construction.

• Support the diversity of execution contexts and execution agents, because not all execution agents are created equal; some are less capable than others, but not less important.

• Allow everything to be customized by an execution context, including transfer to other execution contexts, but don’t require that execution contexts customize everything.

• Care about all reasonable use cases, domains and platforms.

• Errors must be propagated, but error handling must not present a burden.

• Support cancellation, which is not an error.

• Have clear and concise answers for where things execute.

• Be able to manage and terminate the lifetimes of objects asynchronously.

1.3. Examples: End User

In this section we demonstrate the end-user experience of asynchronous programming directly with the sender algorithms presented in this paper. See § 4.19 User-facing sender factories, § 4.20 User-facing sender adaptors, and § 4.21 User-facing sender consumers for short explanations of the algorithms used in these code examples.

1.3.1. Hello world

using namespace std::execution;

scheduler auto sch = get_thread_pool().scheduler();                           // 1

sender auto begin = schedule(sch);                                            // 2
sender auto hi_again = then(begin, []{                                        // 3
    std::cout << "Hello world! Have an int.";                                 // 3
    return 13;                                                                // 3
});                                                                           // 3
sender auto add_42 = then(hi_again, [](int arg) { return arg + 42; });        // 4

auto [i] = this_thread::sync_wait(add_42).value();                            // 5

This example demonstrates the basics of schedulers, senders, and receivers:

1. First we need to get a scheduler from somewhere, such as a thread pool. A scheduler is a lightweight handle to an execution resource.

2. To start a chain of work on a scheduler, we call § 4.19.1 execution::schedule, which returns a sender that completes on the scheduler. sender describes asynchronous work and sends a signal (value, error, or done) to some recipient(s) when that work completes.

3. We use sender algorithms to produce senders and compose asynchronous work. § 4.20.2 execution::then is a sender adaptor that takes an input sender and a std::invocable, and calls the std::invocable on the signal sent by the input sender. The sender returned by then sends the result of that invocation. In this case, the input sender came from schedule, so its void, meaning it won’t send us a value, so our std::invocable takes no parameters. But we return an int, which will be sent to the next recipient.

4. Now, we add another operation to the chain, again using § 4.20.2 execution::then. This time, we get sent a value - the int from the previous step. We add 42 to it, and then return the result.

5. Finally, we’re ready to submit the entire asynchronous pipeline and wait for its completion. Everything up until this point has been completely asynchronous; the work may not have even started yet. To ensure the work has started and then block pending its completion, we use § 4.21.2 this_thread::sync_wait, which will either return a std::optional<std::tuple<...>> with the value sent by the last sender, or an empty std::optional if the last sender sent a done signal, or it throws an exception if the last sender sent an error.

1.3.2. Asynchronous inclusive scan

using namespace std::execution;

sender auto async_inclusive_scan(scheduler auto sch,                          // 2
                                 std::span<const double> input,               // 1
                                 std::span<double> output,                    // 1
                                 double init,                                 // 1
                                 std::size_t tile_count)                      // 3
{
  std::size_t const tile_size = (input.size() + tile_count - 1) / tile_count;

  std::vector<double> partials(tile_count + 1);                               // 4
  partials[0] = init;                                                         // 4

  return transfer_just(sch, std::move(partials))                              // 5
       | bulk(tile_count,                                                     // 6
           [=](std::size_t i, std::vector<double>& partials) {                // 7
             auto start = i * tile_size;                                      // 8
             auto end   = std::min(input.size(), (i + 1) * tile_size);        // 8
             partials[i + 1] = *--std::inclusive_scan(begin(input) + start,   // 9
                                                      begin(input) + end,     // 9
                                                      begin(output) + start); // 9
           })                                                                 // 10
       | then(                                                                // 11
           [](std::vector<double>& partials) {
             std::inclusive_scan(begin(partials), end(partials),              // 12
                                 begin(partials));                            // 12
             return std::move(partials);                                      // 13
           })
       | bulk(tile_count,                                                     // 14
           [=](std::size_t i, std::vector<double>& partials) {                // 14
             auto start = i * tile_size;                                      // 14
             auto end   = std::min(input.size(), (i + 1) * tile_size);        // 14
             std::for_each(output + start, output + end,                      // 14
               [&] (double& e) { e = partials[i] + e; }                       // 14
             );
           })
       | then(                                                                // 15
           [=](std::vector<double>& partials) {                               // 15
             return output;                                                   // 15
           });                                                                // 15
}

This example builds an asynchronous computation of an inclusive scan:

1. It scans a sequence of doubles (represented as the std::span<const double> input) and stores the result in another sequence of doubles (represented as std::span<double> output).

2. It takes a scheduler, which specifies what execution context the scan should be launched on.

3. It also takes a tile_count parameter that controls the number of execution agents that will be spawned.

4. First we need to allocate temporary storage needed for the algorithm, which we’ll do with a std::vector, partials. We need one double of temporary storage for each execution agent we create.

5. Next we’ll create our initial sender with § 4.19.3 execution::transfer_just. This sender will send the temporary storage, which we’ve moved into the sender. The sender has a completion scheduler of sch, which means the next item in the chain will use sch.

6. Senders and sender adaptors support composition via operator|, similar to C++ ranges. We’ll use operator| to attach the next piece of work, which will spawn tile_count execution agents using § 4.20.9 execution::bulk (see § 4.12 Most sender adaptors are pipeable for details).

7. Each agent will call a std::invocable, passing it two arguments. The first is the agent’s index (i) in the § 4.20.9 execution::bulk operation, in this case a unique integer in [0, tile_count). The second argument is what the input sender sent - the temporary storage.

8. We start by computing the start and end of the range of input and output elements that this agent is responsible for, based on our agent index.

9. Then we do a sequential std::inclusive_scan over our elements. We store the scan result for our last element, which is the sum of all of our elements, in our temporary storage partials.

10. After all computation in that initial § 4.20.9 execution::bulk pass has completed, every one of the spawned execution agents will have written the sum of its elements into its slot in partials.

11. Now we need to scan all of the values in partials. We’ll do that with a single execution agent which will execute after the § 4.20.9 execution::bulk completes. We create that execution agent with § 4.20.2 execution::then.

12. § 4.20.2 execution::then takes an input sender and an std::invocable and calls the std::invocable with the value sent by the input sender. Inside our std::invocable, we call std::inclusive_scan on partials, which the input senders will send to us.

13. Then we return partials, which the next phase will need.

14. Finally we do another § 4.20.9 execution::bulk of the same shape as before. In this § 4.20.9 execution::bulk, we will use the scanned values in partials to integrate the sums from other tiles into our elements, completing the inclusive scan.

15. async_inclusive_scan returns a sender that sends the output std::span<double>. A consumer of the algorithm can chain additional work that uses the scan result. At the point at which async_inclusive_scan returns, the computation may not have completed. In fact, it may not have even started.

1.3.3. Asynchronous dynamically-sized read

using namespace std::execution;

sender_of<std::size_t> auto async_read(                                       // 1
    sender_of<std::span<std::byte>> auto buffer,                              // 1
    auto handle);                                                             // 1

struct dynamic_buffer {                                                       // 3
  std::unique_ptr<std::byte[]> data;                                          // 3
  std::size_t size;                                                           // 3
};                                                                            // 3

sender_of<dynamic_buffer> auto async_read_array(auto handle) {                // 2
  return just(dynamic_buffer{})                                               // 4
       | let_value([] (dynamic_buffer& buf) {                                 // 5
           return just(std::as_writeable_bytes(std::span(&buf.size, 1))       // 6
                | async_read(handle)                                          // 7
                | then(                                                       // 8
                    [&] (std::size_t bytes_read) {                            // 9
                      assert(bytes_read == sizeof(buf.size));                 // 10
                      buf.data = std::make_unique(new std::byte[buf.size]);   // 11
                      return std::span(buf.data.get(), buf.size);             // 12
                    }
                | async_read(handle)                                          // 13
                | then(
                    [&] (std::size_t bytes_read) {
                      assert(bytes_read == buf.size);                         // 14
                      return std::move(buf);                                  // 15
                    });
       });
}

This example demonstrates a common asynchronous I/O pattern - reading a payload of a dynamic size by first reading the size, then reading the number of bytes specified by the size:

1. async_read is a pipeable sender adaptor. It’s a customization point object, but this is what it’s call signature looks like. It takes a sender parameter which must send an input buffer in the form of a std::span<std::byte>, and a handle to an I/O context. It will asynchronously read into the input buffer, up to the size of the std::span. It returns a sender which will send the number of bytes read once the read completes.

2. async_read_array takes an I/O handle and reads a size from it, and then a buffer of that many bytes. It returns a sender that sends a dynamic_buffer object that owns the data that was sent.

3. dynamic_buffer is an aggregate struct that contains a std::unique_ptr<std::byte[]> and a size.

4. The first thing we do inside of async_read_array is create a sender that will send a new, empty dynamic_array object using § 4.19.2 execution::just. We can attach more work to the pipeline using operator| composition (see § 4.12 Most sender adaptors are pipeable for details).

5. We need the lifetime of this dynamic_array object to last for the entire pipeline. So, we use let_value, which takes an input sender and a std::invocable that must return a sender itself (see § 4.20.4 execution::let_* for details). let_value sends the value from the input sender to the std::invocable. Critically, the lifetime of the sent object will last until the sender returned by the std::invocable completes.

6. Inside of the let_value std::invocable, we have the rest of our logic. First, we want to initiate an async_read of the buffer size. To do that, we need to send a std::span pointing to buf.size. We can do that with § 4.19.2 execution::just.

7. We chain the async_read onto the § 4.19.2 execution::just sender with operator|.

8. Next, we pipe a std::invocable that will be invoked after the async_read completes using § 4.20.2 execution::then.

9. That std::invocable gets sent the number of bytes read.

10. We need to check that the number of bytes read is what we expected.

11. Now that we have read the size of the data, we can allocate storage for it.

12. We return a std::span<std::byte> to the storage for the data from the std::invocable. This will be sent to the next recipient in the pipeline.

13. And that recipient will be another async_read, which will read the data.

14. Once the data has been read, in another § 4.20.2 execution::then, we confirm that we read the right number of bytes.

15. Finally, we move out of and return our dynamic_buffer object. It will get sent by the sender returned by async_read_array. We can attach more things to that sender to use the data in the buffer.

1.3.4. More end-user examples

1.3.4.1. Sudoku solver

This example comes from Kirk Shoop, who ported an example from TBB’s documentation to sender/receiver in his fork of the libunifex repo. It is a Sudoku solver that uses a configurable number of threads to explore the search space for solutions.

The sender/receiver-based Sudoku solver can be found here. Some things that are worth noting about Kirk’s solution:

1. Although it schedules asychronous work onto a thread pool, and each unit of work will schedule more work, its use of structured concurrency patterns make reference counting unnecessary. The solution does not make use of shared_ptr.

2. In addition to eliminating the need for reference counting, the use of structured concurrency makes it easy to ensure that resources are cleaned up on all code paths. In contrast, the TBB example that inspired this one leaks memory.

For comparison, the TBB-based Sudoku solver can be found here.

1.3.4.2. File copy

This example also comes from Kirk Shoop which uses sender/receiver to recursively copy the files a directory tree. It demonstrates how sender/receiver can be used to do IO, using a scheduler that schedules work on Linux’s io_uring.

As with the Sudoku example, this example obviates the need for reference counting by employing structured concurrency. It uses iteration with an upper limit to avoid having too many open file handles.

You can find the example here.

1.3.4.3. Echo server

Dietmar Kuehl has a hobby project that implements networking APIs on top of sender/receiver. He recently implemented an echo server as a demo. His echo server code can be found here.

Below, I show the part of the echo server code. This code is executed for each client that connects to the echo server. In a loop, it reads input from a socket and echos the input back to the same socket. All of this, including the loop, is implemented with generic async algorithms.

outstanding.start(
    EX::repeat_effect_until(
          EX::let_value(
              NN::async_read_some(ptr->d_socket,
                                  context.scheduler(),
                                  NN::buffer(ptr->d_buffer))
        | EX::then([ptr](::std::size_t n){
            ::std::cout << "read='" << ::std::string_view(ptr->d_buffer, n) << "'\n";
            ptr->d_done = n == 0;
            return n;
        }),
          [&context, ptr](::std::size_t n){
            return NN::async_write_some(ptr->d_socket,
                                        context.scheduler(),
                                        NN::buffer(ptr->d_buffer, n));
          })
        | EX::then([](auto&&...){})
        , [owner = ::std::move(owner)]{ return owner->d_done; }
    )
);

In this code, NN::async_read_some and NN::async_write_some are asynchronous socket-based networking APIs that return senders. EX::repeat_effect_until, EX::let_value, and EX::then and fully generic sender adaptor algorithms that accept and return senders.

This is a good example of seamless composition of async IO functions with non-IO operations. And by composing the senders in this structured way, all the state for the composite operation -- the repeat_effect_until expression and all its child operations -- is stored altogether in a single object.

1.4. Examples: Algorithms

In this section we show a few simple sender/receiver-based algorithm implementations.

1.4.1. then

// For emulating "deducing this"
template <class A, class B>
  concept __this = same_as<remove_cvref_t<A>, B>;

template<receiver R, class F>
struct _then_receiver {
  R r_;
  F f_;

  // Customize set_value by invoking the callable and passing the result to the inner receiver
  template<class... As>
    requires receiver_of<R, invoke_result_t<F, As...>>
  friend void tag_invoke(std::execution::set_value_t, _then_receiver&& self, As&&... as) {
    std::execution::set_value((R&&) self.r_, invoke((F&&) self.f_, (As&&) as...));
  }

  // Forward all other tag_invoke-based CPOs (copy_cvref_t from P1450):
  template <__this<_then_receiver> Self, class... As, invocable<copy_cvref_t<Self, R>, As...> Tag>
  friend auto tag_invoke(Tag tag, Self&& self, As&&... as)
    noexcept(is_nothrow_invocable_v<Tag, copy_cvref_t<Self, R>, As...>)
    -> invoke_result_t<Tag, copy_cvref_t<Self, R>, As...> {
    return ((Tag&&) tag)(((Self&&) self).r_, (As&&) as...);
  }
};

template<sender S, class F>
struct _then_sender : std::execution::sender_base {
  S s_;
  F f_;

  template<receiver R>
    requires sender_to<S, _then_receiver<R, F>>
  friend auto tag_invoke(std::experimental::connect_t, _then_sender&& self, R r)
    -> std::execution::connect_result_t<S, _then_receiver<R, F>> {
      return std::execution::connect((S&&) s_, _then_receiver<R, F>{(R&&) r, (F&&) f_});
  }
};

template<sender S, class F>
sender auto then(S s, F f) {
  return _then_sender{{}, (S&&) s, (F&&) f};
}

This code builds a then algorithm that transforms the value(s) from the input sender with a transformation function. The result of the transformation becomes the new value. The other receiver functions (set_error and set_done), as well as all receiver queries, are passed through unchanged.

In detail, it does the following:

1. Defines a receiver that aggregates another receiver and an invocable which:

   • Defines a constrained tag_invoke overload for transforming the value channel.

   • Defines another constrained overload of tag_invoke that passes all other customizations through unchanged.

2. Defines a sender that aggregates another sender and the invocable, which defines a tag_invoke customization for std::execution::connect that wraps the incoming receiver in the receiver from (1) and passes it and the incoming sender to std::execution::connect, returning the result.

1.4.2. retry

// _conv needed so we can emplace construct non-movable types into
// a std::optional.
template<invocable F>
  requires std::is_nothrow_move_constructible_v<F>
struct _conv {
  F f_;
  explicit _conv(F f) noexcept : f_((F&&) f) {}
  operator invoke_result_t<F>() && {
    return ((F&&) f_)();
  }
};

// pass through all customizations except set_error, which retries the operation.
template<class O, class R>
struct _retry_receiver {
  O* o_;
  explicit _retry_receiver(O* o) : o_(o) {}
  friend void tag_invoke(std::execution::set_error_t, _retry_receiver&& self, auto&&) noexcept {
    self.o_->_retry(); // This causes the op to be retried
  }
  // Forward all other tag_invoke-based CPOs (copy_cvref_t from P1450):
  template <__this<_retry_receiver> Self, class... As, invocable<copy_cvref_t<Self, R>, As...> Tag>
  friend auto tag_invoke(Tag tag, Self&& self, As&&... as)
    noexcept(is_nothrow_invocable_v<Tag, copy_cvref_t<Self, R>, As...>)
    -> invoke_result_t<Tag, copy_cvref_t<Self, R>, As...> {
    return ((Tag&&) tag)(((copy_cvref_t<Self, R>&&) self.o_->r_, (As&&) as...);
  }
};

template<sender S>
struct _retry_sender : std::execution::sender_base {
  S s_;
  explicit _retry_sender(S s) : s_((S&&) s) {}

  // Hold the nested operation state in an optional so we can
  // re-construct and re-start it if the operation fails.
  template<receiver R>
  struct _op {
    S s_;
    R r_;
    std::optional<state_t<S&, _retry_receiver<_op, R>>> o_;

    _op(S s, R r): s_((S&&)s), r_((R&&)r), o_{_connect()} {}
    _op(_op&&) = delete;

    auto _connect() noexcept {
      return _conv{[this] {
        return std::execution::connect(s_, _retry_receiver<_op, R>{this});
      }};
    }
    void _retry() noexcept try {
      o_.emplace(_connect()); // potentially throwing
      std::execution::start(*o_);
    } catch(...) {
      std::execution::set_error((R&&) r_, std::current_exception());
    }
    friend void tag_invoke(std::execution::start_t, _op& o) noexcept {
      std::execution::start(*o.o_);
    }
  };

  template<receiver R>
    requires sender_to<S&, _retry_receiver<_op<R>, R>>
  friend _op<R> tag_invoke(std::execution::connect_t, _retry_sender&& self, R r) {
    return _op<R>{(S&&) self.s_, (R&&) r};
  }
};

template<sender S>
sender auto retry(S s) {
  return _retry_sender{(S&&) s};
}

The retry algorithm takes a multi-shot sender and causes it to repeat on error, passing through values and done signals. Each time the input sender is restarted, a new receiver is connected and the resulting operation state is stored in an optional, which allows us to reinitialize it multiple times.

This example does the following:

1. Defines a _conv utility that takes advantage of C++17’s guaranteed copy elision to emplace a non-movable type in a std::optional.

2. Defines a _retry_receiver that holds a pointer back to the operation state. It passes all customizations through unmodified to the inner receiver owned by the operation state except for set_error, which causes a _retry() function to be called instead.

3. Defines an operation state that aggregates the input sender and receiver, and declares storage for the nested operation state in a std::optional. Constructing the operation state constructs a _retry_receiver with a pointer to the (under construction) operation state and uses it to connect to the aggregated sender.

4. Starting the operation state dispatches to start on the inner operation state.

5. The _retry() function reinitializes the inner operation state by connecting the sender to a new receiver, holding a pointer back to the outer operation state as before.

6. After reinitializing the inner operation state, _retry() calls start on it, causing the failed operation to be rescheduled.

7. Defines a _retry_sender that implements the connect customization point to return an operation state constructed from the passed-in sender and receiver.

1.5. Examples: Schedulers

In this section we look at some schedulers of varying complexity.

1.5.1. Inline scheduler

struct inline_scheduler {
  template <class R>
    struct _op {
      [[no_unique_address]] R rec_;
      friend void tag_invoke(std::execution::start_t, _op& op) noexcept try {
        std::execution::set_value((R&&) op.rec_);
      } catch(...) {
        std::execution::set_error((R&&) op.rec_, std::current_exception());
      }
    };

  struct _sender {
    template <template <class...> class Tuple,
              template <class...> class Variant>
      using value_types = Variant<Tuple<>>;
    template <template <class...> class Variant>
      using error_types = Variant<std::exception_ptr>;
    static constexpr bool sends_done = false;

    template <std::execution::receiver_of R>
      friend auto tag_invoke(std::execution::connect_t, _sender, R&& rec)
        noexcept(std::is_nothrow_constructible_v<std::remove_cvref_t<R>, R>)
        -> _op<std::remove_cvref_t<R>> {
        return {(R&&) rec};
      }
  };

  friend _sender tag_invoke(std::execution::schedule_t, const inline_scheduler&) noexcept {
    return {};
  }

  bool operator==(const inline_scheduler&) const noexcept = default;
};

The inline scheduler is a trivial scheduler that completes immediately and synchronously on the thread that calls std::execution::start on the operation state produced by its sender. In other words, start(connect(schedule(inline-scheduler), receiver)) is just a fancy way of saying set_value(receiver), with the exception of the fact that start wants to be passed an lvalue.

Although not a particularly useful scheduler, it serves to illustrate the basics of implementing one. The inline_scheduler:

1. Customizes execution::schedule to return an instance of the sender type _sender.

2. The _sender type models the typed_sender concept and provides the metadata needed to describe it as a sender of no values (see value_types) that can send an exception_ptr as an error (see error_types), and that never calls set_done (see sends_done).

3. The _sender type customizes execution::connect to accept a receiver of no values. It returns an instance of type _op that holds the receiver by value.

4. The operation state customizes std::execution::start to call std::execution::set_value on the receiver, passing any exceptions to std::execution::set_error as an exception_ptr.

1.5.2. Single thread scheduler

This example shows how to create a scheduler for an execution context that consists of a single thread. It is implemented in terms of a lower-level execution context called manual_event_loop.

class single_thread_context {
  manual_event_loop loop_;
  std::thread thread_;

public:
  single_thread_context()
    : loop_()
    , thread_([this] { loop_.run(); })
  {}

  ~single_thread_context() {
    loop_.finish();
    thread_.join();
  }

  auto get_scheduler() noexcept {
    return loop_.get_scheduler();
  }

  std::thread::id get_thread_id() const noexcept {
    return thread_.get_id();
  }
};

The single_thread_context owns an event loop and a thread to drive it. In the destructor, it tells the event loop to finish up what it’s doing and then joins the thread, blocking for the event loop to drain.

The interesting bits are in the manual_event_loop context implementation. It is slightly too long to include here, so we only provide a reference to it, but there is one noteworthy detail about its implementation. It uses space in its operation state to build an intrusive linked list of work items. In structured concurrency patterns, the operation states of nested operations compose statically, and in an algorithm like this_thread::sync_wait, the composite operation state lives on the stack for the duration of the operation. The end result is that work can be scheduled onto this thread with zero allocations.

1.6. What this proposal is not

This paper is not a patch on top of [P0443R14]; we are not asking to update the existing paper, we are asking to retire it in favor of this paper, which is already self-contained; any example code within this paper can be written in Standard C++, without the need to standardize any further facilities.

This paper is not an alternative design to [P0443R14]; rather, we have taken the design in the current executors paper, and applied targeted fixes to allow it to fulfill the promises of the sender/receiver model, as well as provide all the facilities we consider essential when writing user code using standard execution concepts; we have also applied the guidance of removing one-way executors from the paper entirely, and instead provided an algorithm based around senders that serves the same purpose.

1.7. Design changes from P0443

1. The executor concept has been removed and all of its proposed functionality is now based on schedulers and senders, as per SG1 direction.

2. Properties are not included in this paper. We see them as a possible future extension, if the committee gets more comfortable with them.

3. Senders now advertise what scheduler, if any, their evaluation will complete on.

4. The places of execution of user code in P0443 weren’t precisely defined, whereas they are in this paper. See § 4.5 Senders can propagate completion schedulers.

5. P0443 did not propose a suite of sender algorithms necessary for writing sender code; this paper does. See § 4.19 User-facing sender factories, § 4.20 User-facing sender adaptors, and § 4.21 User-facing sender consumers.

6. P0443 did not specify the semantics of variously qualified connect overloads; this paper does. See § 4.7 Senders can be either multi-shot or single-shot.

7. Specific type erasure facilities are omitted, as per LEWG direction. Type erasure facilities can be built on top of this proposal, as discussed in § 5.9 Ranges-style CPOs vs tag_invoke.

8. A specific thread pool implementation is omitted, as per LEWG direction.

1.8. Prior art

This proposal builds upon and learns from years of prior art with asynchronous and parallel programming frameworks in C++. In this section, we discuss async abstractions that have previously been suggested as a possible basis for asynchronous algorithms and why they fall short.

1.8.1. Futures

A future is a handle to work that has already been scheduled for execution. It is one end of a communication channel; the other end is a promise, used to receive the result from the concurrent operation and to communicate it to the future.

Futures, as traditionally realized, require the dynamic allocation and management of a shared state, synchronization, and typically type-erasure of work and continuation. Many of these costs are inherent in the nature of "future" as a handle to work that is already scheduled for execution. These expenses rule out the future abstraction for many uses and makes it a poor choice for a basis of a generic mechanism.

1.8.2. Coroutines

C++20 coroutines are frequently suggested as a basis for asynchronous algorithms. It’s fair to ask why, if we added coroutines to C++, are we suggesting the addition of a library-based abstraction for asynchrony. Certainly, coroutines come with huge syntactic and semantic advantages over the alternatives.

Although coroutines are lighter weight than futures, coroutines suffer many of the same problems. Since they typically start suspended, they can avoid synchronizing the chaining of dependent work. However in many cases, coroutine frames require an unavoidable dynamic allocation and indirect function calls. This is done to hide the layout of the coroutine frame from the C++ type system, which in turn makes possible the separate compilation of coroutines and certain compiler optimizations, such as optimization of the coroutine frame size.

Those advantages come at a cost, though. Because of the dynamic allocation of coroutine frames, coroutines in embedded or heterogeneous environments, which often lack support for dynamic allocation, require great attention to detail. And the allocations and indirections tend to complicate the job of the inliner, often resulting in sub-optimal codegen.

The coroutine language feature mitigates these shortcomings somewhat with the HALO optimization [P0981R0], which leverages existing compiler optimizations such as allocation elision and devirtualization to inline the coroutine, completely eliminating the runtime overhead. However, HALO requires a sophisiticated compiler, and a fair number of stars need to align for the optimization to kick in. In our experience, more often than not in real-world code today’s compilers are not able to inline the coroutine, resulting in allocations and indirections in the generated code.

In a suite of generic async algorithms that are expected to be callable from hot code paths, the extra allocations and indirections are a deal-breaker. It is for these reasons that we consider coroutines a poor choise for a basis of all standard async.

1.8.3. Callbacks

Callbacks are the oldest, simplest, most powerful, and most efficient mechanism for creating chains of work, but suffer problems of their own. Callbacks must propagate either errors or values. This simple requirement yields many different interface possibilities. The lack of a standard callback shape obstructs generic design.

Additionally, few of these possibilities accommodate cancellation signals when the user requests upstream work to stop and clean up.

1.9. Field experience

This proposal draws heavily from our field experience with libunifex. Libunifex has seen heavy production use at Facebook. As of October 2021, it is currently used in production within the following applications and platforms:

• Facebook Messenger on iOS, Android, Windows, and macOS

• Instagram on iOS and Android

• Facebook on iOS and Android

• Portal

• An internal Facebook product that runs on Linux

All of these applications are making direct use of the sender/receiver abstraction as presented in this paper. One product (Instagram on iOS) is making use of the sender/coroutine integration as presented. The monthly active users of these products number in the billions.

This proposal also draws heavily from our experience with Thrust and Agency. It is also inspired by the needs of countless other C++ frameworks for asynchrony, parallelism, and concurrency, including:

• HPX

• Folly

• stlab

Before this proposal is approved, we will present a new implementation of this proposal written from the specification and intended as a contribution to libc++. This implementation will demonstrate the viability of the design across the use cases and execution contexts that the committee has identified as essential.

2. Revision history

2.1. R2

The changes since R1 are as follows:

• Remove the eagerly executing sender algorithms.

• Extend the execution::connect customization point and the sender_traits<> template to recognize awaitables as typed_senders.

• Add utilities as_awaitable() and with_awaitable_senders<> so a coroutine type can trivially make senders awaitable with a coroutine.

• Add a section describing the design of the sender/awaitable interactions.

• Add a section describing the design of the cancellation support in sender/receiver.

• Add a section showing examples of simple sender adaptor algorithms.

• Add a section showing examples of simple schedulers.

• Add a few more examples: a sudoku solver, a parallel recursive file copy, and an echo server.

• Refined the forward progress guarantees on the bulk algorithm.

• Add a section describing how to use a range of senders to represent async sequences.

• Add a section showing how to use senders to represent partial success.

• Add sender factories execution::just_error and execution::just_done.

• Add sender adaptors execution::done_as_optional and execution::done_as_error.

• Document more production uses of sender/receiver at scale.

• Various fixes of typos and bugs.

2.2. R1

The changes since R0 are as follows:

• Added a new concept, sender_of.

• Added a new scheduler query, this_thread::execute_may_block_caller.

• Added a new scheduler query, get_forward_progress_guarantee.

• Removed the unschedule adaptor.

• Various fixes of typos and bugs.

2.3. R0

Initial revision.

3. Design - introduction

The following four sections describe the entirety of the proposed design.

• § 3 Design - introduction describes the conventions used through the rest of the design sections, as well as an example illustrating how we envision code will be written using this proposal.

• § 4 Design - user side describes all the functionality from the perspective we intend for users: it describes the various concepts they will interact with, and what their programming model is.

• § 5 Design - implementer side describes the machinery that allows for that programming model to function, and the information contained there is necessary for people implementing senders and sender algorithms (including the standard library ones) - but is not necessary to use senders productively.

3.1. Conventions

The following conventions are used throughout the design section:

1. The namespace proposed in this paper is the same as in [P0443R14]: std::execution; however, for brevity, the std:: part of this name is omitted. When you see execution::foo, treat that as std::execution::foo.

2. Universal references and explicit calls to std::move/std::forward are omitted in code samples and signatures for simplicity; assume universal references and perfect forwarding unless stated otherwise.

3. None of the names proposed here are names that we are particularly attached to; consider the names to be reasonable placeholders that can freely be changed, should the committee want to do so.

3.2. Queries and algorithms

A query is a std::invocable that takes some set of objects (usually one) as parameters and returns facts about those objects without modifying them. Queries are usually customization point objects, but in some cases may be functions.

An algorithm is a std::invocable that takes some set of objects as parameters and causes those objects to do something. Algorithms are usually customization point objects, but in some cases may be functions.

4. Design - user side

4.1. Execution contexts describe the place of execution

An execution context is a resource that represents the place where execution will happen. This could be a concrete resource - like a specific thread pool object, or a GPU - or a more abstract one, like the current thread of execution. Execution contexts don’t need to have a representation in code; they are simply a term describing certain properties of execution of a function.

4.2. Schedulers represent execution contexts

A scheduler is a lightweight handle that represents a strategy for scheduling work onto an execution context. Since execution contexts don’t necessarily manifest in C++ code, it’s not possible to program directly against their API. A scheduler is a solution to that problem: the scheduler concept is defined by a single sender algorithm, schedule, which returns a sender that will complete on an execution context determined by the scheduler. Logic that you want to run on that context can be placed in the receiver’s completion-signalling method.

execution::scheduler auto sch = get_thread_pool().scheduler();
execution::sender auto snd = execution::schedule(sch);
// snd is a sender (see below) describing the creation of a new execution resource
// on the execution context associated with sch

Note that a particular scheduler type may provide other kinds of scheduling operations which are supported by its associated execution context. It is not limited to scheduling purely using the execution::schedule API.

Future papers will propose additional scheduler concepts that extend scheduler to add other capabilities. For example:

• A time_scheduler concept that extends scheduler to support time-based scheduling. Such a concept might provide access to schedule_after(sched, duration), schedule_at(sched, time_point) and now(sched) APIs.

• Concepts that extend scheduler to support opening, reading and writing files asynchronously.

• Concepts that extend scheduler to support connecting, sending data and receiving data over the network asynchronously.

4.3. Senders describe work

A sender is an object that describes work. Senders are similar to futures in existing asynchrony designs, but unlike futures, the work that is being done to arrive at the values they will send is also directly described by the sender object itself. A sender is said to send some values if a receiver connected (see § 5.3 execution::connect) to that sender will eventually receive said values.

The primary defining sender algorithm is § 5.3 execution::connect; this function, however, is not a user-facing API; it is used to facilitate communication between senders and various sender algorithms, but end user code is not expected to invoke it directly.

The way user code is expected to interact with senders is by using sender algorithms. This paper proposes an initial set of such sender algorithms, which are described in § 4.4 Senders are composable through sender algorithms, § 4.19 User-facing sender factories, § 4.20 User-facing sender adaptors, and § 4.21 User-facing sender consumers. For example, here is how a user can create a new sender on a scheduler, attach a continuation to it, and then wait for execution of the continuation to complete:

execution::scheduler auto sch = get_thread_pool().scheduler();
execution::sender auto snd = execution::schedule(sch);
execution::sender auto cont = execution::then(snd, []{
    std::fstream file{ "result.txt" };
    file << compute_result;
});

this_thread::sync_wait(cont);
// at this point, cont has completed execution

4.4. Senders are composable through sender algorithms

Asynchronous programming often departs from traditional code structure and control flow that we are familiar with. A successful asynchronous framework must provide an intuitive story for composition of asynchronous work: expressing dependencies, passing objects, managing object lifetimes, etc.

The true power and utility of senders is in their composability. With senders, users can describe generic execution pipelines and graphs, and then run them on and across a variety of different schedulers. Senders are composed using sender algorithms:

• sender factories, algorithms that take no senders and return a sender.

• sender adaptors, algorithms that take (and potentially execution::connect) senders and return a sender.

• sender consumers, algorithms that take (and potentially execution::connect) senders and do not return a sender.

4.5. Senders can propagate completion schedulers

One of the goals of executors is to support a diverse set of execution contexts, including traditional thread pools, task and fiber frameworks (like HPX) and Legion), and GPUs and other accelerators (managed by runtimes such as CUDA or SYCL). On many of these systems, not all execution agents are created equal and not all functions can be run on all execution agents. Having precise control over the execution context used for any given function call being submitted is important on such systems, and the users of standard execution facilities will expect to be able to express such requirements.

[P0443R14] was not always clear about the place of execution of any given piece of code. Precise control was present in the two-way execution API present in earlier executor designs, but it has so far been missing from the senders design. There has been a proposal ([P1897R3]) to provide a number of sender algorithms that would enforce certain rules on the places of execution of the work described by a sender, but we have found those sender algorithms to be insufficient for achieving the best performance on all platforms that are of interest to us. The implementation strategies that we are aware of result in one of the following situations:

1. trying to submit work to one execution context (such as a CPU thread pool) from another execution context (such as a GPU or a task framework), which assumes that all execution agents are as capable as a std::thread (which they aren’t).

2. forcibly interleaving two adjacent execution graph nodes that are both executing on one execution context (such as a GPU) with glue code that runs on another execution context (such as a CPU), which is prohibitively expensive for some execution contexts (such as CUDA or SYCL).

3. having to customise most or all sender algorithms to support an execution context, so that you can avoid problems described in 1. and 2, which we believe is impractical and brittle based on months of field experience attempting this in Agency.

None of these implementation strategies are acceptable for many classes of parallel runtimes, such as task frameworks (like HPX) or accelerator runtimes (like CUDA or SYCL).

Therefore, in addition to the on sender algorithm from [P1897R3], we are proposing a way for senders to advertise what scheduler (and by extension what execution context) they will complete on. Any given sender may have completion schedulers for some or all of the signals (value, error, or done) it completes with (for more detail on the completion signals, see § 5.1 Receivers serve as glue between senders). When further work is attached to that sender by invoking sender algorithms, that work will also complete on an appropriate completion scheduler.

4.5.1. execution::get_completion_scheduler

get_completion_scheduler is a query that retrieves the completion scheduler for a specific completion signal from a sender. Calling get_completion_scheduler on a sender that does not have a completion scheduler for a given signal is ill-formed. If a sender advertises a completion scheduler for a signal in this way, that sender must ensure that it sends that signal on an execution agent belonging to an execution context represented by a scheduler returned from this function. See § 4.5 Senders can propagate completion schedulers for more details.

execution::scheduler auto cpu_sched = new_thread_scheduler{};
execution::scheduler auto gpu_sched = cuda::scheduler();

execution::sender auto snd0 = execution::schedule(cpu_sched);
execution::scheduler auto completion_sch0 =
  execution::get_completion_scheduler<execution::set_value_t>(snd0);
// completion_sch0 is equivalent to cpu_sched

execution::sender auto snd1 = execution::then(snd0, []{
    std::cout << "I am running on cpu_sched!\n";
});
execution::scheduler auto completion_sch1 =
  execution::get_completion_scheduler<execution::set_value_t>(snd1);
// completion_sch1 is equivalent to cpu_sched

execution::sender auto snd2 = execution::transfer(snd1, gpu_sched);
execution::sender auto snd3 = execution::then(snd2, []{
    std::cout << "I am running on gpu_sched!\n";
});
execution::scheduler auto completion_sch3 =
  execution::get_completion_scheduler<execution::set_value_t>(snd3);
// completion_sch3 is equivalent to gpu_sched

4.6. Execution context transitions are explicit

[P0443R14] does not contain any mechanisms for performing an execution context transition. The only sender algorithm that can create a sender that will move execution to a specific execution context is execution::schedule, which does not take an input sender. That means that there’s no way to construct sender chains that traverse different execution contexts. This is necessary to fulfill the promise of senders being able to replace two-way executors, which had this capability.

We propose that, for senders advertising their completion scheduler, all execution context transitions must be explicit; running user code anywhere but where they defined it to run must be considered a bug.

The execution::transfer sender adaptor performs a transition from one execution context to another:

execution::scheduler auto sch1 = ...;
execution::scheduler auto sch2 = ...;

execution::sender auto snd1 = execution::schedule(sch1);
execution::sender auto then1 = execution::then(snd1, []{
    std::cout << "I am running on sch1!\n";
});

execution::sender auto snd2 = execution::transfer(then1, sch2);
execution::sender auto then2 = execution::then(snd2, []{
    std::cout << "I am running on sch2!\n";
});

this_thread::sync_wait(then2);

4.7. Senders can be either multi-shot or single-shot

Some senders may only support launching their operation a single time, while others may be repeatable and support being launched multiple times. Executing the operation may consume resources owned by the sender.

For example, a sender may contain a std::unique_ptr that it will be transferring ownership of to the operation-state returned by a call to execution::connect so that the operation has access to this resource. In such a sender, calling execution::connect consumes the sender such that after the call the input sender is no longer valid. Such a sender will also typically be move-only so that it can maintain unique ownership of that resource.

A single-shot sender can only be connected to a receiver at most once. Its implementation of execution::connect only has overloads for an rvalue-qualified sender. Callers must pass the sender as an rvalue to the call to execution::connect, indicating that the call consumes the sender.

A multi-shot sender can be connected to multiple receivers and can be launched multiple times. Multi-shot senders customise execution::connect to accept an lvalue reference to the sender. Callers can indicate that they want the sender to remain valid after the call to execution::connect by passing an lvalue reference to the sender to call these overloads. Multi-shot senders should also define overloads of execution::connect that accept rvalue-qualified senders to allow the sender to be also used in places where only a single-shot sender is required.

If the user of a sender does not require the sender to remain valid after connecting it to a receiver then it can pass an rvalue-reference to the sender to the call to execution::connect. Such usages should be able to accept either single-shot or multi-shot senders.

If the caller does wish for the sender to remain valid after the call then it can pass an lvalue-qualified sender to the call to execution::connect. Such usages will only accept multi-shot senders.

Algorithms that accept senders will typically either decay-copy an input sender and store it somewhere for later usage (for example as a data-member of the returned sender) or will immediately call execution::connect on the input sender, such as in this_thread::sync_wait or execution::start_detached.

Some multi-use sender algorithms may require that an input sender be copy-constructible but will only call execution::connect on an rvalue of each copy, which still results in effectively executing the operation multiple times. Other multi-use sender algorithms may require that the sender is move-constructible but will invoke execution::connect on an lvalue reference to the sender.

For a sender to be usable in both multi-use scenarios, it will generally be required to be both copy-constructible and lvalue-connectable.

4.8. Senders are forkable

Any non-trivial program will eventually want to fork a chain of senders into independent streams of work, regardless of whether they are single-shot or multi-shot. For instance, an incoming event to a middleware system may be required to trigger events on more than one downstream system. This requires that we provide well defined mechanisms for making sure that connecting a sender multiple times is possible and correct.

The split sender adaptor facilitates connecting to a sender multiple times, regardless of whether it is single-shot or multi-shot:

auto some_algorithm(execution::sender auto&& input) {
    execution::sender auto multi_shot = split(input);
    // "multi_shot" is guaranteed to be multi-shot,
    // regardless of whether "input" was multi-shot or not

    return when_all(
      then(multi_shot, [] { std::cout << "First continuation\n"; }),
      then(multi_shot, [] { std::cout << "Second continuation\n"; })
    );
}

4.9. Senders are joinable

Similarly to how it’s hard to write a complex program that will eventually want to fork sender chains into independent streams, it’s also hard to write a program that does not want to eventually create join nodes, where multiple independent streams of execution are merged into a single one in an asynchronous fashion.

when_all is a sender adaptor that returns a sender that completes when the last of the input senders completes. It sends a pack of values, where the elements of said pack are the values sent by the input senders, in order. when_all returns a sender that also does not have an associated scheduler.

transfer_when_all accepts an additional scheduler argument. It returns a sender whose value completion scheduler is the scheduler provided as an argument, but otherwise behaves the same as when_all. You can think of it as a composition of transfer(when_all(inputs...), scheduler), but one that allows for better efficiency through customization.

4.10. Senders support cancellation

Senders are often used in scenarios where the application may be concurrently executing multiple strategies for achieving some program goal. When one of these strategies succeeds (or fails) it may not make sense to continue pursuing the other strategies as their results are no longer useful.

For example, we may want to try to simultaneously connect to multiple network servers and use whichever server responds first. Once the first server responds we no longer need to continue trying to connect to the other servers.

Ideally, in these scenarios, we would somehow be able to request that those other strategies stop executing promptly so that their resources (e.g. cpu, memory, I/O bandwidth) can be released and used for other work.

While the design of senders has support for cancelling an operation before it starts by simply destroying the sender or the operation-state returned from execution::connect() before calling execution::start(), there also needs to be a standard, generic mechanism to ask for an already-started operation to complete early.

The ability to be able to cancel in-flight operations is fundamental to supporting some kinds of generic concurrency algorithms.

For example:

• a when_all(ops...) algorithm should cancel other operations as soon as one operation fails

• a first_successful(ops...) algorithm should cancel the other operations as soon as one operation completes successfuly

• a generic timeout(src, duration) algorithm needs to be able to cancel the src operation after the timeout duration has elapsed.

• a stop_when(src, trigger) algorithm should cancel src if trigger completes first and cancel trigger if src completes first

The mechanism used for communcating cancellation-requests, or stop-requests, needs to have a uniform interface so that generic algorithms that compose sender-based operations, such as the ones listed above, are able to communicate these cancellation requests to senders that they don’t know anything about.

The design is intended to be composable so that cancellation of higher-level operations can propagate those cancellation requests through intermediate layers to lower-level operations that need to actually respond to the cancellation requests.

For example, we can compose the algorithms mentioned above so that child operations are cancelled when any one of the multiple cancellation conditions occurs:

sender auto composed_cancellation_example(auto query) {
  return stop_when(
    timeout(
      when_all(
        first_successful(
          query_server_a(query),
          query_server_b(query)),
        load_file("some_file.jpg")),
      5s),
    cancelButton.on_click());
}

In this example, if we take the operation returned by query_server_b(query), this operation will receive a stop-request when any of the following happens:

• first_successful algorithm will send a stop-request if query_server_a(query) completes successfully

• when_all algorithm will send a stop-request if the load_file("some_file.jpg") operation completes with an error or done result.

• timeout algorithm will send a stop-request if the operation does not complete within 5 seconds.

• stop_when algorithm will send a stop-request if the user clicks on the "Cancel" button in the user-interface.

• The parent operation consuming the composed_cancellation_example() sends a stop-request

Note that within this code there is no explicit mention of cancellation, stop-tokens, callbacks, etc. yet the example fully supports and responds to the various cancellation sources.

The intent of the design is that the common usage of cancellation in sender/receiver-based code is primarily through use of concurrency algorithms that manage the detailed plumbing of cancellation for you. Much like algorithms that compose senders relieve the user from having to write their own receiver types, algorithms that introduce concurrency and provide higher-level cancellation semantics relieve the user from having to deal with low-level details of cancellation.

4.10.1. Cancellation design summary

The design of cancellation described in this paper is built on top of and extends the std::stop_token-based cancellation facilities added in C++20, first proposed in [P2175R0].

At a high-level, the facilities proposed by this paper for supporting cancellation include:

• Add std::stoppable_token and std::stoppable_token_for concepts that generalise the interface of std::stop_token type to allow other types with different implementation strategies.

• Add std::unstoppable_token concept for detecting whether a stoppable_token can never receive a stop-request.

• Add std::in_place_stop_token, std::in_place_stop_source and std::in_place_stop_callback<CB> types that provide a more efficient implementation of a stop-token for use in structured concurrency situations.

• Add std::never_stop_token for use in places where you never want to issue a stop-request

• Add std::execution::get_stop_token() CPO for querying the stop-token to use for an operation from its receiver.

• Add std::execution::stop_token_type_t<T> for querying the type of a stop-token returned from get_stop_token()

In addition, there are requirements added to some of the algorithms to specify what their cancellation behaviour is and what the requirements of customisations of those algorithms are with respect to cancellation.

The key component that enables generic cancellation within sender-based operations is the execution::get_stop_token() CPO. This CPO takes a single parameter, which is the receiver passed to execution::connect, and returns a std::stoppable_token that the operation should use to check for stop-requests for that operation.

As the caller of execution::connect typically has control over the receiver type it passes, it is able to customise the execution::get_stop_token() CPO for that receiver type to return a stop-token that it has control over and that it can use to communicate a stop-request to the operation once it has started.

4.10.2. Support for cancellation is optional

Support for cancellation is optional, both on part of the author of the receiver and on part of the author of the sender.

If the receiver does not customise the execution::get_stop_token() CPO then invoking the CPO on that receiver will invoke the default implementation which returns std::never_stop_token. This is a special stoppable_token type that is statically known to always return false from the stop_possible() method.

Sender code that tries to use this stop-token will in general result in code that handles stop-requests being compiled out and having little to no run-time overhead.

If the sender doesn’t call execution::get_stop_token(), for example because the operation does not support cancellation, then it will simply not respond to stop-requests from the caller.

Note that stop-requests are generally racy in nature as there is often a race betwen an operation completing naturally and the stop-request being made. If the operation has already completed or past the point at which it can be cancelled when the stop-request is sent then the stop-request may just be ignored. An application will typically need to be able to cope with senders that might ignore a stop-request anyway.

4.10.3. Cancellation is inherently racy

Usually, an operation will attach a stop-callback at some point inside the call to execution::start() so that a subsequent stop-request will interrupt the logic.

A stop-request can be issued concurrently from another thread. This means the implementation of execution::start() needs to be careful to ensure that, once a stop-callback has been registered, that there are no data-races between a potentially concurrently-executing stop-callback and the rest of the execution::start() implementation.

An implementation of execution::start() that supports cancellation will generally need to perform (at least) two separate steps: launch the operation, subscribe a stop-callback to the receiver’s stop-token. Care needs to be taken depending on the order in which these two steps are performed.

If the stop-callback is subscribed first and then the operation is launched, care needs to be taken to ensure that a stop-request that invokes the stop-callback on another thread after the stop-callback is registered but before the operation finishes launching does not either result in a missed cancellation request or a data-race. e.g. by performing an atomic write after the launch has finished executing

If the operation is launched first and then the stop-callback is subscribed, care needs to be taken to ensure that if the launched operation completes concurrently on another thread that it does not destroy the operation-state until after the stop-callback has been registered. e.g. by having the execution::start implementation write to an atomic variable once it has finished registering the stop-callback and having the concurrent completion handler check that variable and either call the completion-signalling operation or store the result and defer calling the receiver’s completion-signalling operation to the execution::start() call (which is still executing).

For an example of an implementation strategy for solving these data-races see the async_recv() example in Appendix A.

4.10.4. Cancellation design status

This paper currently includes the design for cancellation as proposed in [P2175R0] - "Composable cancellation for sender-based async operations". P2175R0 contains more details on the background motivation and prior-art and design rationale of this design.

It is important to note, however, that initial review of this design in the SG1 concurrency subgroup raised some concerns related to runtime overhead of the design in single-threaded scenarios and these concerns are still being investigated.

The design of P2175R0 has been included in this paper for now, despite its potential to change, as we believe that support for cancellation is a fundamental requirement for an async model and is required in some form to be able to talk about the semantics of some of the algorithms proposed in this paper.

This paper will be updated in the future with any changes that arise from the investigations into P2175R0.

4.11. Schedulers advertise their forward progress guarantees

To decide whether a scheduler (and its associated execution context) is sufficient for a specific task, it may be necessary to know what kind of forward progress guarantees it provides for the execution agents it creates. The C++ Standard defines the following forward progress guarantees:

• concurrent, which requires that a thread makes progress eventually;

• parallel, which requires that a thread makes progress once it executes a step; and

• weakly parallel, which does not require that the thread makes progress.

This paper introduces a scheduler query function, get_forward_progress_guarantee, which returns one of the enumerators of a new enum type, forward_progress_guarantee. Each enumerator of forward_progress_guarantee corresponds to one of the aforementioned guarantees.

4.12. Most sender adaptors are pipeable

To facilitate an intuitive syntax for composition, most sender adaptors are pipeable; they can be composed (piped) together with operator|. This mechanism is similar to the operator| composition that C++ range adaptors support and draws inspiration from piping in *nix shells. Pipeable sender adaptors take a sender as their first parameter and have no other sender parameters.

a | b will pass the sender a as the first argument to the pipeable sender adaptor b. Pipeable sender adaptors support partial application of the parameters after the first. For example, all of the following are equivalent:

execution::bulk(snd, N, [] (std::size_t i, auto d) {});
execution::bulk(N, [] (std::size_t i, auto d) {})(snd);
snd | execution::bulk(N, [] (std::size_t i, auto d) {});

Piping enables you to compose together senders with a linear syntax. Without it, you’d have to use either nested function call syntax, which would cause a syntactic inversion of the direction of control flow, or you’d have to introduce a temporary variable for each stage of the pipeline. Consider the following example where we want to execute first on a CPU thread pool, then on a CUDA GPU, then back on the CPU thread pool:

auto snd = execution::then(
             execution::transfer(
               execution::then(
                 execution::transfer(
                   execution::then(
                     execution::schedule(get_thread_pool_scheduler())
                     []{ return 123; }),
                   cuda::new_stream_scheduler()),
                 [](int i){ return 123 * 5; }),
               get_thread_pool()),
             [](int i){ return i - 5; });
auto [result] = this_thread::sync_wait(snd).value();
// result == 610

auto snd0 = execution::schedule(get_thread_pool_scheduler());
auto snd1 = execution::then(snd0, []{ return 123; });
auto snd2 = execution::transfer(snd1, cuda::new_stream_scheduler());
auto snd3 = execution::then(snd2, [](int i){ return 123 * 5; })
auto snd4 = execution::transfer(snd3, get_thread_pool())
auto snd5 = execution::then(snd4, [](int i){ return i - 5; });
auto [result] = *this_thread::sync_wait(snd4);
// result == 610

auto snd = execution::schedule(get_thread_pool_scheduler())
         | execution::then([]{ return 123; })
         | execution::transfer(cuda::new_stream_scheduler())
         | execution::then([](int i){ return 123 * 5; })
         | execution::transfer(get_thread_pool())
         | execution::then([](int i){ return i - 5; });
auto [result] = this_thread::sync_wait(snd).value();
// result == 610

Certain sender adaptors are not be pipeable, because using the pipeline syntax can result in confusion of the semantics of the adaptors involved. Specifically, the following sender adaptors are not pipeable.

• execution::when_all and execution::when_all_with_variant: Since this sender adaptor takes a variadic pack of senders, a partially applied form would be ambiguous with a non partially applied form with an arity of one less.

• execution::on: This sender adaptor changes how the sender passed to it is executed, not what happens to its result, but allowing it in a pipeline makes it read as if it performed a function more similar to transfer.

Sender consumers could be made pipeable, but we have chosen to not do so. However, since these are terminal nodes in a pipeline and nothing can be piped after them, we believe a pipe syntax may be confusing as well as unnecessary, as consumers cannot be chained. We believe sender consumers read better with function call syntax.

4.13. A range of senders represents an async sequence of data

Senders represent a single unit of asynchronous work. In many cases though, what is being modelled is a sequence of data arriving asynchronously, and you want computation to happen on demand, when each element arrives. This requires nothing more than what is in this paper and the range support in C++20. A range of senders would allow you to model such input as keystrikes, mouse movements, sensor readings, or network requests.

Given some expression R that is a range of senders, consider the following in a coroutine that returns an async generator type:

for (auto snd : R) {
  if (auto opt = co_await execution::done_as_optional(std::move(snd)))
    co_yield fn(*std::move(opt));
  else
    break;
}

This transforms each element of the asynchronous sequence R with the function fn on demand, as the data arrives. The result is a new asynchronous sequence of the transformed values.

Now imagine that R is the simple expression views::iota(0) | views::transform(execution::just). This creates a lazy range of senders, each of which completes immediately with monotonically increasing integers. The above code churns through the range, generating a new infine asynchronous range of values [fn(0), fn(1), fn(2), ...].

Far more interesting would be if R were a range of senders representing, say, user actions in a UI. The above code gives a simple way to respond to user actions on demand.

4.14. Senders can represent partial success

Receivers have three ways they can complete: with success, failure, or cancellation. This begs the question of how they can be used to represent async operations that partially succeed. For example, consider an API that reads from a socket. The connection could drop after the API has filled in some of the buffer. In cases like that, it makes sense to want to report both that the connection dropped and that some data has been successfully read.

Often in the case of partial success, the error condition is not fatal nor does it mean the API has failed to satisfy its post-conditions. It is merely an extra piece of information about the nature of the completion. In those cases, "partial success" is another way of saying "success". As a result, it is sensible to pass both the error code and the result (if any) through the value channel, as shown below:

// Capture a buffer for read_socket_async to fill in
execution::just(array<byte, 1024>{})
  | execution::let_value([socket](array<byte, 1024>& buff) {
      // read_socket_async completes with two values: an error_code and
      // a count of bytes:
      return read_socket_async(socket, span{buff})
          // For success (partial and full), specify the next action:
        | execution::let_value([](error_code err, size_t bytes_read) {
            if (err != 0) {
              // OK, partial success. Decide how to deal with the partial results
            } else {
              // OK, full success here.
            }
          });
    })

In other cases, the partial success is more of a partial failure. That happens when the error condition indicates that in some way the function failed to satisfy its post-conditions. In those cases, sending the error through the value channel loses valuable contextual information. It’s possible that bundling the error and the incomplete results into an object and passing it through the error channel makes more sense. In that way, generic algorithms will not miss the fact that a post-condition has not been met and react inappropriately.

Another possibility is for an async API to return a range of senders: if the API completes with full success, full error, or cancellation, the returned range contains just one sender with the result. Otherwise, if the API partially fails (doesn’t satisfy its post-conditions, but some incomplete result is available), the returned range would have two senders: the first containing the partial result, and the second containing the error. Such an API might be used in a coroutine as follows:

// Declare a buffer for read_socket_async to fill in
array<byte, 1024> buff;

for (auto snd : read_socket_async(socket, span{buff})) {
  try {
    if (optional<size_t> bytes_read =
          co_await execution::done_as_optional(std::move(snd)))
      // OK, we read some bytes into buff. Process them here....
    } else {
      // The socket read was cancelled and returned no data. React
      // appropriately.
    }
  } catch (...) {
    // read_socket_async failed to meet its post-conditions.
    // Do some cleanup and propagate the error...
  }
}

Finally, it’s possible to combine these two approaches when the API can both partially succeed (meeting its post-conditions) and partially fail (not meeting its post-conditions).

4.15. All awaitables are senders

Since C++20 added coroutines to the standard, we expect that coroutines and awaitables will be how a great many will choose to express their asynchronous code. However, in this paper, we are proposing to add a suite of asynchronous algorithms that accept senders, not awaitables. One might wonder whether and how these algorithms will be accessible to those who choose coroutines instead of senders.

In truth there will be no problem because all generally awaitable types automatically model the typed_sender concept. The adaptation is transparent and happens in the sender customization points, which are aware of awaitables. (By "generally awaitable" we mean types that don’t require custom await_transform trickery from a promise type to make them awaitable.)

For an example, imagine a coroutine type called task<T> that knows nothing about senders. It doesn’t implement any of the sender customization points. Despite that fact, and despite the fact that the this_thread::sync_wait algorithm is constrained with the typed_sender concept, the following would compile and do what the user wants:

task<int> doSomeAsyncWork();

int main() {
  // OK, awaitable types satisfy the requirements for typed senders:
  auto o = this_thread::sync_wait(doSomeAsyncWork());
}

Since awaitables are senders, writing a sender-based asynchronous algorithm is trivial if you have a coroutine task type: implement the algorithm as a coroutine. If you are not bothered by the possibility of allocations and indirections as a result of using coroutines, then there is no need to ever write a sender, a receiver, or an operation state.

4.16. Many senders can be trivially made awaitable

If you choose to implement your sender-based algorithms as coroutines, you’ll run into the issue of how to retrieve results from a passed-in sender. This is not a problem. If the coroutine type opts in to sender support -- trivial with the execution::with_awaitable_senders utility -- then a large class of senders are transparently awaitable from within the coroutine.

For example, consider the following trivial implementation of the sender-based retry algorithm:

template <class S>
  requires single-typed-sender<S&> // See [execution.coro_utils.as_awaitable]
task<single-sender-value-type<S>> retry(S s) {
  for (;;) {
    try {
      co_return co_await s;
    } catch(...) {
    }
  }
}

Only some senders can be made awaitable directly because of the fact that callbacks are more expressive than coroutines. An awaitable expression has a single type: the result value of the async operation. In contrast, a callback can accept multiple arguments as the result of an operation. What’s more, the callback can have overloaded function call signatures that take different sets of arguments. There is no way to automatically map such senders into awaitables. The with_awaitable_senders utility recognizes as awaitables those senders that send a single value of a single type. To await another kind of sender, a user would have to first map its value channel into a single value of a single type -- say, with the into_variant sender algorithm -- before co_await-ing that sender.

4.17. Cancellation of a sender can unwind a stack of coroutines

When looking at the sender-based retry algorithm in the previous section, we can see that the value and error cases are correctly handled. But what about cancellation? What happens to a coroutine that is suspended awaiting a sender that completes by calling execution::set_done?

When your task type’s promise inherits from with_awaitable_senders, what happens is this: the coroutine behaves as if an uncatchable exception had been thrown from the co_await expression. (It is not really an exception, but it’s helpful to think of it that way.) Provided that the promise types of the calling coroutines also inherit from with_awaitable_senders, or more generally implement a member function called unhandled_done, the exception unwinds the chain of coroutines as if an exception were thrown except that it bypasses catch(...) clauses.

In order to "catch" this uncatchable done exception, one of the calling coroutines in the stack would have to await a sender that maps the done channel into either a value or an error. That is achievable with the execution::let_done, execution::upon_done, execution::done_as_optional, or execution::done_as_error sender adaptors. For instance, we can use execution::done_as_optional to "catch" the done signal and map it into an empty optional as shown below:

if (auto opt = co_await execution::done_as_optional(some_sender)) {
  // OK, some_sender completed successfully, and opt contains the result.
} else {
  // some_sender completed with a cancellation signal.
}

As described in the section "All awaitables are senders", the sender customization points recognize awaitables and adapt them transparently to model the sender concept. When connect-ing an awaitable and a receiver, the adaptation layer awaits the awaitable within a coroutine that implements unhandled_done in its promise type. The effect of this is that an "uncatchable" done exception propagates seamlessly out of awaitables, causing execution::set_done to be called on the receiver.

Obviously, unhandled_done is a library extension of the coroutine promise interface. Many promise types will not implement unhandled_done. When an uncatchable done exception tries to propagate through such a coroutine, it is treated as an unhandled exception and terminate is called. The solution, as described above, is to use a sender adaptor to handle the done exception before awaiting it. It goes without saying that any future Standard Library coroutine types ought to implement unhandled_done. The author of [P1056R1], which proposes a standard coroutine task type, is in agreement.

4.18. Composition with Parallel Algorithms

The C++ Standard Library provides a large number of algorithms that offer the potential for non-sequential execution via the use of execution policies. The set of algorithms with execution policy overloads are often referred to as "parallel algorithms", although additional policies are available.

Existing policies, such as execution::par, give the implementation permission to execute the algorithm in parallel. However, the choice of execution resources used to perform the work is left to the implementation.

We will propose a customization point for combining schedulers with policies in order to provide control over where work will execute.

template<class ExecutionPolicy>
implementation-defined executing_on(
    execution::scheduler auto scheduler,
    ExecutionPolicy && policy
);

This function would return an object of an implementation-defined type which can be used in place of an execution policy as the first argument to one of the parallel algorithms. The overload selected by that object should execute its computation as requested by policy while using scheduler to create any work to be run. The expression may be ill-formed if scheduler is not able to support the given policy.

The existing parallel algorithms are synchronous; all of the effects performed by the computation are complete before the algorithm returns to its caller. This remains unchanged with the executing_on customization point.

In the future, we expect additional papers will propose asynchronous forms of the parallel algorithms which (1) return senders rather than values or void and (2) where a customization point pairing a sender with an execution policy would similarly be used to obtain an object of implementation-defined type to be provided as the first argument to the algorithm.

4.19. User-facing sender factories

A sender factory is an algorithm that takes no senders as parameters and returns a sender.

4.19.1. execution::schedule

execution::sender auto schedule(
    execution::scheduler auto scheduler
);

Returns a sender describing the start of a task graph on the provided scheduler. See § 4.2 Schedulers represent execution contexts.

execution::scheduler auto sch1 = get_system_thread_pool().scheduler();

execution::sender auto snd1 = execution::schedule(sch1);
// snd1 describes the creation of a new task on the system thread pool

4.19.2. execution::just

execution::sender auto just(
    auto ...&& values
);

Returns a sender with no completion schedulers, which sends the provided values. If a provided value is an lvalue reference, a copy is made inside the returned sender and a non-const lvalue reference to the copy is sent. If the provided value is an rvalue reference, it is moved into the returned sender and an rvalue reference to it is sent.

execution::sender auto snd1 = execution::just(3.14);
execution::sender auto then1 = execution::then(snd1, [] (double d) {
  std::cout << d << "\n";
});

execution::sender auto snd2 = execution::just(3.14, 42);
execution::sender auto then2 = execution::then(snd1, [] (double d, int i) {
  std::cout << d << ", " << i << "\n";
});

std::vector v3{1, 2, 3, 4, 5};
execution::sender auto snd3 = execution::just(v3);
execution::sender auto then3 = execution::then(snd3, [] (std::vector<int>& v3copy) {
  for (auto&& e : v3copy) { e *= 2; } return v3copy;
}
auto&& [v3copy] = this_thread::sync_wait(then3).value();
// v3 contains {1, 2, 3, 4, 5}; v3copy will contain {2, 4, 6, 8, 10}.

execution::sender auto snd4 = execution::just(std::vector{1, 2, 3, 4, 5});
execution::sender auto then4 = execution::then(snd4, [] (std::vector<int>&& v4) {
  for (auto&& e : v4) { e *= 2; }
  return std::move(v4);
});
auto&& [v4] = this_thread::sync_wait(then4).value();
// v4 contains {2, 4, 6, 8, 10}.

4.19.3. execution::transfer_just

execution::sender auto transfer_just(
    execution::scheduler auto scheduler,
    auto ...&& values
);

Returns a sender whose value completion scheduler is the provided scheduler, which sends the provided values in the same manner as just.

execution::sender auto vals = execution::transfer_just(
    get_system_thread_pool().scheduler(),
    1, 2, 3
);
execution::sender auto snd = execution::then(vals, [](auto... args) {
    std::print(args...);
});
// when snd is executed, it will print "123"

This adaptor is included as it greatly simplifies lifting values into senders.

4.19.4. execution::just_error

execution::sender auto just_error(
    auto && error
);

Returns a sender with no completion schedulers, which completes with the specified error. If the provided error is an lvalue reference, a copy is made inside the returned sender and a non-const lvalue reference to the copy is sent to the receiver’s set_error. If the provided value is an rvalue reference, it is moved into the returned sender and an rvalue reference to it is sent to the receiver’s set_error.

4.19.5. execution::just_done

execution::sender auto just_done();

Returns a sender with no completion schedulers, which completes immediately by calling the receiver’s set_done.

4.20. User-facing sender adaptors

A sender adaptor is an algorithm that takes one or more senders, which it may execution::connect, as parameters, and returns a sender, whose completion is related to the sender arguments it has received.

Sender adaptors are lazy, that is, they are never allowed to submit any work for execution prior to the returned sender being started later on, and are also guaranteed to not start any input senders passed into them. Sender consumers such as § 4.20.13 execution::ensure_started, § 4.21.1 execution::start_detached, and § 4.21.2 this_thread::sync_wait start senders.

For more implementer-centric description of starting senders, see § 5.5 Sender adaptors are lazy.

4.20.1. execution::transfer

execution::sender auto transfer(
    execution::sender auto input,
    execution::scheduler auto scheduler
);

Returns a sender describing the transition from the execution agent of the input sender to the execution agent of the target scheduler. See § 4.6 Execution context transitions are explicit.

execution::scheduler auto cpu_sched = get_system_thread_pool().scheduler();
execution::scheduler auto gpu_sched = cuda::scheduler();

execution::sender auto cpu_task = execution::schedule(cpu_sched);
// cpu_task describes the creation of a new task on the system thread pool

execution::sender auto gpu_task = execution::transfer(cpu_task, gpu_sched);
// gpu_task describes the transition of the task graph described by cpu_task to the gpu

4.20.2. execution::then

execution::sender auto then(
    execution::sender auto input,
    std::invocable<values-sent-by(input)...> function
);

then returns a sender describing the task graph described by the input sender, with an added node of invoking the provided function with the values sent by the input sender as arguments.

then is guaranteed to not begin executing function until the returned sender is started.

execution::sender auto input = get_input();
execution::sender auto snd = execution::then(input, [](auto... args) {
    std::print(args...);
});
// snd describes the work described by pred
// followed by printing all of the values sent by pred

This adaptor is included as it is necessary for writing any sender code that actually performs a useful function.

4.20.3. execution::upon_*

execution::sender auto upon_error(
    execution::sender auto input,
    std::invocable<errors-sent-by(input)...> function
);

execution::sender auto upon_done(
    execution::sender auto input,
    std::invocable auto function
);

upon_error and upon_done are similar to then, but where then works with values sent by the input sender, upon_error works with errors, and upon_done is invoked when the "done" signal is sent.

4.20.4. execution::let_*

execution::sender auto let_value(
    execution::sender auto input,
    std::invocable<values-sent-by(input)...> function
);

execution::sender auto let_error(
    execution::sender auto input,
    std::invocable<errors-sent-by(input)...> function
);

execution::sender auto let_done(
    execution::sender auto input,
    std::invocable auto function
);

let_value is very similar to then: when it is started, it invokes the provided function with the values sent by the input sender as arguments. However, where the sender returned from then sends exactly what that function ends up returning - let_value requires that the function return a sender, and the sender returned by let_value sends the values sent by the sender returned from the callback. This is similar to the notion of "future unwrapping" in future/promise-based frameworks.

let_value is guaranteed to not begin executing function until the returned sender is started.

let_error and let_done are similar to let_value, but where let_value works with values sent by the input sender, let_error works with errors, and let_done is invoked when the "done" signal is sent.

4.20.5. execution::on

execution::sender auto on(
    execution::scheduler auto sched,
    execution::sender auto snd
);

Returns a sender which, when started, will start the provided sender on an execution agent belonging to the execution context associated with the provided scheduler. This returned sender has no completion schedulers.

4.20.6. execution::into_variant

execution::sender auto into_variant(
    execution::sender auto snd
);

Returns a sender which sends a variant of tuples of all the possible sets of types sent by the input sender. Senders can send multiple sets of values depending on runtime conditions; this is a helper function that turns them into a single variant value.

4.20.7. execution::done_as_optional

execution::sender auto done_as_optional(
    single-typed-sender auto snd
);

Returns a sender that maps the value channel from a T to an optional<decay_t<T>>, and maps the done channel to a value of an empty optional<decay_t<T>>.

4.20.8. execution::done_as_error

template<move_constructible Error>
execution::sender auto done_as_error(
    execution::sender auto snd,
    Error err = Error{}
);

Returns a sender that maps the done channel to an error of err.

4.20.9. execution::bulk

execution::sender auto bulk(
    execution::sender auto input,
    std::integral auto size,
    invocable<decltype(size), values-sent-by(input)...> function
);

Returns a sender describing the task of invoking the provided function with every index in the provided shape along with the values sent by the input sender. The returned sender completes once all invocations have completed, or an error has occurred. If it completes by sending values, they are equivalent to those sent by the input sender.

No instance of function will begin executing until the returned sender is started. Each invocation of function runs in an execution agent whose forward progress guarantees are determined by the scheduler on which they are run. All agents created by a single use of bulk execute with the same guarantee. This allows, for instance, a scheduler to execute all invocations of the function in parallel.

The bulk operation is intended to be used at the point where the number of agents to be created is known and provided to bulk via its shape parameter. For some parallel computations, the number of agents to be created may be a function of the input data or dynamic conditions of the execution environment. In such cases, bulk can be combined with additional operations such as let_value to deliver dynamic shape information to the bulk operation.

In this proposal, only integral types are used to specify the shape of the bulk section. We expect that future papers may wish to explore extensions of the interface to explore additional kinds of shapes, such as multi-dimensional grids, that are commonly used for parallel computing tasks.

4.20.10. execution::split

execution::sender auto split(execution::sender auto sender);

If the provided sender is a multi-shot sender, returns that sender. Otherwise, returns a multi-shot sender which sends values equivalent to the values sent by the provided sender. See § 4.7 Senders can be either multi-shot or single-shot.

4.20.11. execution::when_all

execution::sender auto when_all(
    execution::sender auto ...inputs
);

execution::sender auto when_all_with_variant(
    execution::sender auto ...inputs
);

when_all returns a sender which completes once all of the input senders have completed. The values send by this sender are the values sent by each of the input, in order of the arguments passed to when_all.

when_all_with_variant does the same, but it adapts all the input senders using into_variant.

The returned sender has no completion schedulers.

See § 4.9 Senders are joinable.

execution::scheduler auto sched = get_thread_pool().scheduler();

execution::sender auto sends_1 = ...;
execution::sender auto sends_abc = ...;

execution::sender auto both = execution::when_all(sched,
    sends_1,
    sends_abc
);

execution::sender auto final = execution::then(both, [](auto... args){
    std::cout << std::format("the two args: {}, {}", args...);
});
// when final executes, it will print "the two args: 1, abc"

4.20.12. execution::transfer_when_all

execution::sender auto transfer_when_all(
    execution::scheduler auto sched,
    execution::sender auto ...inputs
);

execution::sender auto transfer_when_all_with_variant(
    execution::scheduler auto sched,
    execution::sender auto ...inputs
);

Similar to § 4.20.11 execution::when_all, but returns a sender whose value completion scheduler is the provided scheduler.

See § 4.9 Senders are joinable.

4.20.13. execution::ensure_started

execution::sender auto ensure_started(
    execution::sender auto sender
);

Once ensure_started returns, it is known that the provided sender has been connected and start has been called on the resulting operation state (see § 5.2 Operation states represent work); in other words, the work described by the provided sender has been submitted for execution on the appropriate execution contexts. Returns a sender which completes when the provided sender completes and sends values equivalent to those of the provided sender.

If the returned sender is destroyed before execution::connect() is called, or if execution::connect() is called but the returned operation-state is destroyed before execution::start() is called, then a stop-request is sent to the eagerly launched operation and the operation is detached and will run to completion in the background. Its result will be discarded when it eventually completes.

Note that the application will need to make sure that resources are kept alive in the case that the operation detaches. e.g. by holding a std::shared_ptr to those resources or otherwise having some out-of-band way to signal completion of the operation so that resource release can be sequenced after the completion.

4.21. User-facing sender consumers

A sender consumer is an algorithm that takes one or more senders, which it may execution::connect, as parameters, and does not return a sender.

4.21.1. execution::start_detached

void start_detached(
    execution::sender auto sender
);

Like ensure_started, but does not return a value; if the provided sender sends an error instead of a value, std::terminate is called.

4.21.2. this_thread::sync_wait

auto sync_wait(
    execution::sender auto sender
) requires (always-sends-same-values(sender))
    -> std::optional<std::tuple<values-sent-by(sender)>>;

this_thread::sync_wait is a sender consumer that submits the work described by the provided sender for execution, similarly to ensure_started, except that it blocks the current std::thread or thread of main until the work is completed, and returns an optional tuple of values that were sent by the provided sender on its completion of work. Where § 4.19.1 execution::schedule and § 4.19.3 execution::transfer_just are meant to enter the domain of senders, sync_wait is meant to exit the domain of senders, retrieving the result of the task graph.

If the provided sender sends an error instead of values, sync_wait throws that error as an exception, or rethrows the original exception if the error is of type std::exception_ptr.

If the provided sender sends the "done" signal instead of values, sync_wait returns an empty optional.

For an explanation of the requires clause, see § 5.8 Most senders are typed. That clause also explains another sender consumer, built on top of sync_wait: sync_wait_with_variant.

Note: This function is specified inside std::this_thread, and not inside execution. This is because sync_wait has to block the current execution agent, but determining what the current execution agent is is not reliable. Since the standard does not specify any functions on the current execution agent other than those in std::this_thread, this is the flavor of this function that is being proposed. If C++ ever obtains fibers, for instance, we expect that a variant of this function called std::this_fiber::sync_wait would be provided. We also expect that runtimes with execution agents that use different synchronization mechanisms than std::thread's will provide their own flavors of sync_wait as well (assuming their execution agents have the means to block in a non-deadlock manner).

4.22. execution::execute

In addition to the three categories of functions presented above, we also propose to include a convenience function for fire-and-forget eager one-way submission of an invocable to a scheduler, to fulfil the role of one-way executors from P0443.

void execution::execute(
    execution::schedule auto sched,
    std::invocable auto fn
);

Submits the provided function for execution on the provided scheduler, as-if by:

auto snd = execution::schedule(sched);
auto work = execution::then(snd, fn);
execution::start_detached(work);

5. Design - implementer side

5.1. Receivers serve as glue between senders

A receiver is a callback that supports more than one channel. In fact, it supports three of them:

• set_value, which is the moral equivalent of an operator() or a function call, which signals successful completion of the operation its execution depends on;

• set_error, which signals that an error has happened during scheduling of the current work, executing the current work, or at some earlier point in the sender chain; and

• set_done, which signals that the operation completed without succeeding (set_value) and without failing (set_error). This result is often used to indicate that the operation stopped early, typically because it was asked to do so because the result is no longer needed.

Exactly one of these channels must be successfully (i.e. without an exception being thrown) invoked on a receiver before it is destroyed; if a call to set_value failed with an exception, either set_error or set_done must be invoked on the same receiver. These requirements are know as the receiver contract.

While the receiver interface may look novel, it is in fact very similar to the interface of std::promise, which provides the first two signals as set_value and set_error, and it’s possible to emulate the third channel with lifetime management of the promise.

Receivers are not a part of the end-user-facing API of this proposal; they are necessary to allow unrelated senders communicate with each other, but the only users who will interact with receivers directly are authors of senders.

Receivers are what is passed as the second argument to § 5.3 execution::connect.

5.2. Operation states represent work

An operation state is an object that represents work. Unlike senders, it is not a chaining mechanism; instead, it is a concrete object that packages the work described by a full sender chain, ready to be executed. An operation state is neither movable nor copyable, and its interface consists of a single algorithm: start, which serves as the submission point of the work represented by a given operation state.

Operation states are not a part of the user-facing API of this proposal; they are necessary for implementing sender consumers like execution::ensure_started and this_thread::sync_wait, and the knowledge of them is necessary to implement senders, so the only users who will interact with operation states directly are authors of senders and authors of sender algorithms.

The return value of § 5.3 execution::connect must satisfy the operation state concept.

5.3. execution::connect

execution::connect is a customization point which connects senders with receivers, resulting in an operation state that will ensure that the receiver contract of the receiver passed to connect will be fulfilled.

execution::sender auto snd = some input sender;
execution::receiver auto rcv = some receiver;
execution::operation_state auto state = execution::connect(snd, rcv);

execution::start(state);
// at this point, it is guaranteed that the work represented by state has been submitted
// to an execution context, and that execution context will eventually fulfill the
// receiver contract of rcv

// operation states are not movable, and therefore this operation state object must be
// kept alive until the operation finishes

5.4. Sender algorithms are customizable

Senders being able to advertise what their completion schedulers are fulfills one of the promises of senders: that of being able to customize an implementation of a sender algorithm based on what scheduler any work it depends on will complete on.

The simple way to provide customizations for functions like then, that is for sender adaptors and sender consumers, is to follow the customization scheme that has been adopted for C++20 ranges library; to do that, we would define the expression execution::then(sender, invocable) to be equivalent to:

1. sender.then(invocable), if that expression is well formed; otherwise

2. then(sender, invocable), performed in a context where this call always performs ADL, if that expression is well formed; otherwise

3. a default implementation of then, which returns a sender adaptor, and then define the exact semantics of said adaptor.

However, this definition is problematic. Imagine another sender adaptor, bulk, which is a structured abstraction for a loop over an index space. Its default implementation is just a for loop. However, for accelerator runtimes like CUDA, we would like sender algorithms like bulk to have specialized behavior, which invokes a kernel of more than one thread (with its size defined by the call to bulk); therefore, we would like to customize bulk for CUDA senders to achieve this. However, there’s no reason for CUDA kernels to necessarily customize the then sender adaptor, as the generic implementation is perfectly sufficient. This creates a problem, though; consider the following snippet:

execution::scheduler auto cuda_sch = cuda_scheduler{};

execution::sender auto initial = execution::schedule(cuda_sch);
// the type of initial is a type defined by the cuda_scheduler
// let’s call it cuda::schedule_sender<>

execution::sender auto next = execution::then(cuda_sch, []{ return 1; });
// the type of next is a standard-library implementation-defined sender adaptor
// that wraps the cuda sender
// let’s call it execution::then_sender_adaptor<cuda::schedule_sender<>>

execution::sender auto kernel_sender = execution::bulk(next, shape, [](int i){ ... });

How can we specialize the bulk sender adaptor for our wrapped schedule_sender? Well, here’s one possible approach, taking advantage of ADL (and the fact that the definition of "associated namespace" also recursively enumerates the associated namespaces of all template parameters of a type):

namespace cuda::for_adl_purposes {
template<typename... SentValues>
class schedule_sender {
    execution::operation_state auto connect(execution::receiver auto rcv);
    execution::scheduler auto get_completion_scheduler() const;
};

execution::sender auto bulk(
    execution::sender auto && input,
    execution::shape auto && shape,
    invocable<sender-values(input)> auto && fn)
{
    // return a cuda sender representing a bulk kernel launch
}
} // namespace cuda::for_adl_purposes

However, if the input sender is not just a then_sender_adaptor like in the example above, but another sender that overrides bulk by itself, as a member function, because its author believes they know an optimization for bulk - the specialization above will no longer be selected, because a member function of the first argument is a better match than the ADL-found overload.

This means that well-meant specialization of sender algorithms that are entirely scheduler-agnostic can have negative consequences. The scheduler-specific specialization - which is essential for good performance on platforms providing specialized ways to launch certain sender algorithms - would not be selected in such cases. But it’s really the scheduler that should control the behavior of sender algorithms when a non-default implementation exists, not the sender. Senders merely describe work; schedulers, however, are the handle to the runtime that will eventually execute said work, and should thus have the final say in how the work is going to be executed.

Therefore, we are proposing the following customization scheme (also modified to take § 5.9 Ranges-style CPOs vs tag_invoke into account): the expression execution::<sender-algorithm>(sender, args...), for any given sender algorithm that accepts a sender as its first argument, should be equivalent to:

1. tag_invoke(<sender-algorithm>, get_completion_scheduler<Signal>(sender), sender, args...), if that expression is well-formed; otherwise

2. tag_invoke(<sender-algorithm>, sender, args...), if that expression is well-formed; otherwise

3. a default implementation, if there exists a default implementation of the given sender algorithm.

where Signal is one of set_value, set_error, or set_done; for most sender algorithms, the completion scheduler for set_value would be used, but for some (like upon_error or let_done), one of the others would be used.

For sender algorithms which accept concepts other than sender as their first argument, we propose that the customization scheme remains as it has been in [P0443R14] so far, except it should also use tag_invoke.

5.5. Sender adaptors are lazy

Contrary to early revisions of this paper, we propose to make all sender adaptors perform strictly lazy submission, unless specified otherwise (the one notable exception in this paper is § 4.20.13 execution::ensure_started, whose sole purpose is to start an input sender).

Strictly lazy submission means that there is a guarantee that no work is submitted to an execution context before a receiver is connected to a sender, and execution::start is called on the resulting operation state.

5.6. Lazy senders provide optimization opportunities

Because lazy senders fundamentally describe work, instead of describing or representing the submission of said work to an execution context, and thanks to the flexibility of the customization of most sender algorithms, they provide an opportunity for fusing multiple algorithms in a sender chain together, into a single function that can later be submitted for execution by an execution context. There are two ways this can happen.

The first (and most common) way for such optimizations to happen is thanks to the structure of the implementation: because all the work is done within callbacks invoked on the completion of an earlier sender, recursively up to the original source of computation, the compiler is able to see a chain of work described using senders as a tree of tail calls, allowing for inlining and removal of most of the sender machinery. In fact, when work is not submitted to execution contexts outside of the current thread of execution, compilers are capable of removing the senders abstraction entirely, while still allowing for composition of functions across different parts of a program.

The second way for this to occur is when a sender algorithm is specialized for a specific set of arguments. For instance, we expect that, for senders which are known to have been started already, § 4.20.13 execution::ensure_started will be an identity transformation, because the sender algorithm will be specialized for such senders. Similarly, an implementation could recognize two subsequent § 4.20.9 execution::bulks of compatible shapes, and merge them together into a single submission of a GPU kernel.

5.7. Execution context transitions are two-step

Because execution::transfer takes a sender as its first argument, it is not actually directly customizable by the target scheduler. This is by design: the target scheduler may not know how to transition from a scheduler such as a CUDA scheduler; transitioning away from a GPU in an efficient manner requires making runtime calls that are specific to the GPU in question, and the same is usually true for other kinds of accelerators too (or for scheduler running on remote systems). To avoid this problem, specialized schedulers like the ones mentioned here can still hook into the transition mechanism, and inject a sender which will perform a transition to the regular CPU execution context, so that any sender can be attached to it.

This, however, is a problem: because customization of sender algorithms must be controlled by the scheduler they will run on (see § 5.4 Sender algorithms are customizable), the type of the sender returned from transfer must be controllable by the target scheduler. Besides, the target scheduler may itself represent a specialized execution context, which requires additional work to be performed to transition to it. GPUs and remote node schedulers are once again good examples of such schedulers: executing code on their execution contexts requires making runtime API calls for work submission, and quite possibly for the data movement of the values being sent by the input sender passed into transfer.

To allow for such customization from both ends, we propose the inclusion of a secondary transitioning sender adaptor, called schedule_from. This adaptor is a form of schedule, but takes an additional, second argument: the input sender. This adaptor is not meant to be invoked manually by the end users; they are always supposed to invoke transfer, to ensure that both schedulers have a say in how the transitions are made. Any scheduler that specializes transfer(snd, sch) shall ensure that the return value of their customization is equivalent to schedule_from(sch, snd2), where snd2 is a successor of snd that sends values equivalent to those sent by snd.

The default implementation of transfer(snd, sched) is schedule_from(sched, snd).

5.8. Most senders are typed

All senders should advertise the types they will send when they complete. This is necessary for a number of features, and writing code in a way that’s agnostic of whether an input sender is typed or not in common sender adaptors such as execution::then is hard.

The mechanism for this advertisement is the same as in [P0443R14]; the way to query the types is through sender_traits::value_types<tuple_like, variant_like>.

sender_traits::value_types is a template that takes two arguments: one is a tuple-like template, the other is a variant-like template. The tuple-like argument is required to represent senders sending more than one value (such as when_all). The variant-like argument is required to represent senders that choose which specific values to send at runtime.

There’s a choice made in the specification of § 4.21.2 this_thread::sync_wait: it returns a tuple of values sent by the sender passed to it, wrapped in std::optional to handle the set_done signal. However, this assumes that those values can be represented as a tuple, like here:

execution::sender auto sends_1 = ...;
execution::sender auto sends_2 = ...;
execution::sender auto sends_3 = ...;

auto [a, b, c] = this_thread::sync_wait(
    execution::transfer_when_all(
        execution::get_completion_scheduler<execution::set_value_t>(sends_1),
        sends_1,
        sends_2,
        sends_3
    )).value();
// a == 1
// b == 2
// c == 3

This works well for senders that always send the same set of arguments. If we ignore the possibility of having a sender that sends different sets of arguments into a receiver, we can specify the "canonical" (i.e. required to be followed by all senders) form of value_types of a sender which sends Types... to be as follows:

template<template<typename ...> typename TupleLike>
using value_types = TupleLike;

If senders could only ever send one specific set of values, this would probably need to be the required form of value_types for all senders; defining it otherwise would cause very weird results and should be considered a bug.

This matter is somewhat complicated by the fact that (1) set_value for receivers can be overloaded and accept different sets of arguments, and (2) senders are allowed to send multiple different sets of values, depending on runtime conditions, the data they consumed, and so on. To accomodate this, [P0443R14] also includes a second template parameter to value_types, one that represents a variant-like type. If we permit such senders, we would almost certainly need to require that the canonical form of value_types for all senders (to ensure consistency in how they are handled, and to avoid accidentally interpreting a user-provided variant as a sender-provided one) sending the different sets of arguments Types1..., Types2..., ..., TypesN... to be as follows:

template<
    template<typename ...> typename TupleLike,
    template<typename ...> typename VariantLike
>
using value_types = VariantLike<
    TupleLike<Types1...>,
    TupleLike<Types2...>,
    ...,
    TupleLike<Types3...>
>;

This, however, introduces a couple of complications:

1. A just(1) sender would also need to follow this structure, so the correct type for storing the value sent by it would be std::variant<std::tuple<int>> or some such. This introduces a lot of compile time overhead for the simplest senders, and this overhead effectively exists in all places in the code where value_types is queried, regardless of the tuple-like and variant-like templates passed to it. Such overhead does exist if only the tuple-like parameter exists, but is made much worse by adding this second wrapping layer.

2. As a consequence of (1): because sync_wait needs to store the above type, it can no longer return just a std::tuple<int> for just(1); it has to return std::variant<std::tuple<int>>. C++ currently does not have an easy way to destructure this; it may get less awkward with pattern matching, but even then it seems extremely heavyweight to involve variants in this API, and for the purpose of generic code, the kind of the return type of sync_wait must be the same across all sender types.

One possible solution to (2) above is to place a requirement on sync_wait that it can only accept senders which send only a single set of values, therefore removing the need for std::variant to appear in its API; because of this, we propose to expose both sync_wait, which is a simple, user-friendly version of the sender consumer, but requires that value_types have only one possible variant, and sync_wait_with_variant, which accepts any sender, but returns an optional whose value type is the variant of all the possible tuples sent by the input sender:

auto sync_wait_with_variant(
    execution::sender auto sender
) -> std::optional<std::variant<
        std::tuple<values0-sent-by(sender)>,
        std::tuple<values1-sent-by(sender)>,
        ...,
        std::tuple<valuesn-sent-by(sender)>
    >>;

auto sync_wait(
    execution::sender auto sender
) requires (always-sends-same-values(sender))
    -> std::optional<std::tuple<values-sent-by(sender)>>;

5.9. Ranges-style CPOs vs tag_invoke

The contemporary technique for customization in the Standard Library is customization point objects. A customization point object, will it look for member functions and then for nonmember functions with the same name as the customization point, and calls those if they match. This is the technique used by the C++20 ranges library, and previous executors proposals ([P0443R14] and [P1897R3]) intended to use it as well. However, it has several unfortunate consequences:

1. It does not allow for easy propagation of customization points unknown to the adaptor to a wrapped object, which makes writing universal adapter types much harder - and this proposal uses quite a lot of those.

2. It effectively reserves names globally. Because neither member names nor ADL-found functions can be qualified with a namespace, every customization point object that uses the ranges scheme reserves the name for all types in all namespaces. This is unfortunate due to the sheer number of customization points already in the paper, but also ones that we are envisioning in the future. It’s also a big problem for one of the operations being proposed already: sync_wait. We imagine that if, in the future, C++ was to gain fibers support, we would want to also have std::this_fiber::sync_wait, in addition to std::this_thread::sync_wait. However, because we would want the names to be the same in both cases, we would need to make the names of the customizations not match the names of the customization points. This is undesirable.

This paper proposes to instead use the mechanism described in [P1895R0]: tag_invoke; the wording for tag_invoke has been incorporated into the proposed specification in this paper.

In short, instead of using globally reserved names, tag_invoke uses the type of the customization point object itself as the mechanism to find customizations. It globally reserves only a single name - tag_invoke - which itself is used the same way that ranges-style customization points are used. All other customization points are defined in terms of tag_invoke. For example, the customization for std::this_thread::sync_wait(s) will call tag_invoke(std::this_thread::sync_wait, s), instead of attempting to invoke s.sync_wait(), and then sync_wait(s) if the member call is not valid.

Using tag_invoke has the following benefits:

1. It reserves only a single global name, instead of reserving a global name for every customization point object we define.

2. It is possible to propagate customizations to a subobject, because the information of which customization point is being resolved is in the type of an argument, and not in the name of the function:

   // forward most customizations to a subobject
   template<typename Tag, typename ...Args>
   friend auto tag_invoke(Tag && tag, wrapper & self, Args &&... args) {
       return std::forward<Tag>(tag)(self.subobject, std::forward<Args>(args)...);
   }
   
   // but override one of them with a specific value
   friend auto tag_invoke(specific_customization_point_t, wrapper & self) {
       return self.some_value;
   }
   

3. It is possible to pass those as template arguments to types, because the information of which customization point is being resolved is in the type. Similarly to how [P0443R14] defines a polymorphic executor wrapper which accepts a list of properties it supports, we can imagine scheduler and sender wrappers that accept a list of queries and operations they support. That list can contain the types of the customization point objects, and the polymorphic wrappers can then specialize those customization points on themselves using tag_invoke, dispatching to manually constructed vtables containing pointers to specialized implementations for the wrapped objects. For an example of such a polymorphic wrapper, see unifex::any_unique (example).

6. Specification

Much of this wording follows the wording of [P0443R14].

§ 7 General utilities library [utilities] is meant to be a diff relative to the wording of the [utilities] clause of [N4885]. This diff applies changes from [P1895R0].

§ 8 Thread support library [thread] is meant to be a diff relative to the wording of the [thread] clause of [N4885]. This diff applies changes from [P2175R0].

§ 9 Execution control library [execution] is meant to be added as a new library clause to the working draft of C++.

7. General utilities library [utilities]

7.1. Function objects [function.objects]

7.1.1. Header <functional> synopsis [functional.syn]

At the end of this subclause, insert the following declarations into the synopsis within namespace std:

7.1.2. execution::tag_invoke [func.tag_invoke]

Insert this section as a new subclause, between Searchers [func.search] and Class template hash [unord.hash].

8. Thread support library [thread]

Note: The specification in this section is incomplete; it does not provide an API specification for the new types added into <stop_token>. For a less formal specification of the missing pieces, see the "Proposed Changes" section of [P2175R0]. A future revision of this paper will contain a full specification for the new types.

8.1. Stop tokens [thread.stoptoken]

8.1.1. Header <stop_token> synopsis [thread.stoptoken.syn]

At the beginning of this subclause, insert the following declarations into the synopsis within namespace std:

At the end of this subclause, insert the following declarations into the synopsis of within namespace std:

8.1.2. Stop token concepts [thread.stoptoken.concepts]

Insert this section as a new subclause between Header <stop_token> synopsis [thread.stoptoken.syn] and Class stop_token [stoptoken].