P3409R0: Enabling more efficient stop-token based cancellation of senders

If we do not apply this change before releasing sender/receiver, we will not be able to apply this optimization later.

If the status quo is kept then users may write their own sender algorithm implementations that take advantage of the fact that they can register multiple stop-callbacks. e.g. by passing an inherited stop-token to multiple child operations which run concurrently and each register a stop-callback.

Adding a restriction on how stop-tokens can be used later would be a breaking change as trying to compose such user-defined algorithms with algorithms that wanted to take advantage of the stop-token restrictions would lead to undefined behaviour due to that user-code potentially trying to register multiple stop-callbacks associated with the stop-token, resulting in some pre-condition violations.

Also, trying to change an algorithm implementation from using std::inplace_stop_source to later using std::single_inplace_stop_source is going to change the layout of operation-state types and would thus be a potential ABI break for that algorithm.

There are no syntactic changes required to the stoppable_token concept in order to support this.

However, to make the intent clear, we need to add a paragraph to the description of stop-tokens that states that for a given type that models stop-callback-for, its constructor may have a pre-condition that the number of associated stop-callback objects is less than some positive, finite number.

This explicitly grants permission to implementers of the concept to add such a pre-condition to its stop-callback constructor. It also means that consumers of a generic stop-token must assume that the stop-callback constructor may have such a pre-condition, potentially with a maximum number of existing associated stop-callback objects that is zero, and therefore such generic consumers should limit themselves to constructing a single stop-callback object associated with the stop-token.

Implementations of stop-callback types are still free to define their constructor without such a precondition, and it is still valid for consumers of the corresponding stop-token type to construct multiple associated stop-callback objects.

For example, if I write a function that takes a std::inplace_stop_token then I know that this type allows an unbounded number of associated std::inplace_stop_callback objects and so, within the function I can safely construct multiple associated stop-callback objects.

void func1(std::inplace_stop_token st) {
  const auto on_stop = [] { /* do something */ };

  // Constructing multiple stop-callbacks is allowed.
  std::inplace_stop_callback cb1{st, [&] noexcept { /* do something */ }};
  std::inplace_stop_callback cb2{st, [&] noexcept { /* do something else */ }};

  // ...
}

However, if I were to write a function-template that could be instantiated with any type that satisfied std::stoppable_token, then I would need to limit myself to constructing at most one associated stop-callback object at a time.

template<std::stoppable_token StopToken>
void func2(StopToken st) {
const auto on_stop = [] { /* do something */ };
using callback_t = std::stop_callback_for_t<StopToken, decltype(on_stop)>;

// Constructing a single stop-callback is OK
callback_t cb1{st, on_stop};

// Constructing a second stop-callback would potentially be a pre-condition
// violation if StopToken happens to be e.g. std::single_inplace_stop_token.
callback_t cb2{st, on_stop};
}

Further, I would also be implying that my function also has a pre-condition that my caller provide me with a stop-token that permits me to construct at least one associated stop-callback. This would prevent them from passing, for example, a std::single_inplace_stop_token that already had an associated stop-callback object.

template<std::stoppable_token StopToken>
void func3(StopToken st) {
  const auto on_stop = [] { /* do something */ };
  using callback_t = std::stop_callback_for_t<StopToken, decltype(on_stop)>;

  callback_t cb{st, on_stop};

  // ...
}

void caller() {
  std::single_inplace_stop_source ss;

  func3(ss.get_token());  // OK. stop-token allows constructing a stop-callback

  std::single_inplace_stop_callback cb{ss.get_token(), [] { /* do something */ }};

  func3(ss.get_token());  // BUG: violates func3's pre-condition.
                          // stop-token already has an associated stop-callback.
}

One of the use-cases that needs to be carefully considered here are algorithms, like schedule_from, which are specified to connect multiple child operations ahead of time, but only actually executes one of the child operations at a time.

These are algorithms where connect() on the parent operation calls connect() on two (or more) child operations, but where start() on the parent operation calls start() on the first child but start() on the second child is not called until after the completion of the first child. i.e. where the execution of the child operations does not overlap in time.

Consider the case where such a parent operation is provided an environment with a stop-token that only permits a single stop-callback (such as the proposed std::single_inplace_stop_token).

It would be preferable to allow passing through this stop-token to both children rather than having to construct a separate std::finite_inplace_stop_source<2> and provide different stop-tokens to each child and then also attach a stop-callback to the provided stop-token which forwards stop-requests through to a call to request_stop() on the local stop-source.

However, in order to guarantee that we do not violate the pre-conditions of the stop-callback constructor, we need to ensure that the child operations do not both attempt to construct stop-callbacks with overlapping lifetimes.

The current wording for [exec.recv.concepts] p3 states:

Let rcvr be a receiver and let op_state be an operation state associated with an asynchronous operation created by connecting rcvr with a sender. Let token be a stop token equal to get_stop_token(get_env(rcvr)). token shall remain valid for the duration of the asynchronous operation's lifetime ([exec.async.ops]).

[Note: This means that, unless it knows about further guarantees provided by the type of rcvr, the implementation of op_state cannot use token after it executes a completion operation. This also implies that any stop callbacks registered on token must be destroyed before the invocation of the completion operation. — end note]

This references [exec.async.ops] p7 which defines "asynchronous operation lifetime":

The lifetime of an asynchronous operation, also known as the operation's async lifetime, begins when its start operation begins executing and ends when its completion operation begins executing. If the lifetime of an asynchronous operation's associated operation state ends before the lifetime of the asynchronous operation, the behavior is undefined. After an asynchronous operation executes a completion operation, its associated operation state is invalid. Accessing any part of an invalid operation state is undefined behavior.

The important parts of these two paragraphs are that the stop-token obtained from get_stop_token(get_env(rcvr)) is only required to be valid for the duration of the asynchronous operation's lifetime, and that an asynchronous operation's lifetime starts at the beginning of the call to start() on the operation-state and ends at the beginning of the call to a completion-function.

This implies that, unless you have additional information about the validity of a stop-token provided in the environment, you should not assume that it is valid to construct a stop-callback assocated with that stop-token (or indeed do anything else you can't do with an invalid stop-token) until the start() operation on the operation-state is called.

This constraint placed on sender algorithms and their use of stop-tokens should be sufficient to guarantee that it is safe for the class of algorithms discussed above, for example schedule_from, to pass through a single_inplace_stop_token from its environment to the environment passed to child operations.

The one change I would suggest here is to explicitly call out this restriction in the note, similarly to how the note calls out that stop-callbacks must be destroyed before the invocation of the completion operation. In particular it should call out that stop-callbacks should not be constructed until after the beginning of the invocation of the start() method on the operation-state.

A parent operation only needs to introduce a new stop-source and give separate stop-tokens to child operations if both of the following are true:

• we don't know that the stop-token can support multiple stop-callbacks at the same time; and
• the child operations have overlapping asynchronous operation lifetimes; and
• the parent operation wants to forward stop-requests to child operations

Proposes adding the following class and class-template definitions the <stop_token> header:

namespace std
{
  class single_inplace_stop_token;
  template <std::invocable CB>
  class single_inplace_stop_callback;

  class single_inplace_stop_source {
  public:
    single_inplace_stop_source() noexcept;
    ~single_inplace_stop_source();

    single_inplace_stop_source(const single_inplace_stop_source&) = delete;
    single_inplace_stop_source(single_inplace_stop_source&&) = delete;
    single_inplace_stop_source& operator=(const single_inplace_stop_source&) = delete;
    single_inplace_stop_source& operator=(single_inplace_stop_source&&) = delete;

    bool stop_possible() const noexcept;
    bool stop_requested() const noexcept;

    bool request_stop() noexcept;

    single_inplace_stop_token get_token() const noexcept;
  };

  class single_inplace_stop_token {
  public:
    template <typename CB>
    using callback_type = single_inplace_stop_callback<CB>;

    single_inplace_stop_token() noexcept;
    single_inplace_stop_token(const single_inplace_stop_token&) noexcept;
    single_inplace_stop_token(single_inplace_stop_token&&) noexcept;
    ~single_inplace_stop_token();

    single_inplace_stop_token& operator=(const single_inplace_stop_token&) noexcept;
    single_inplace_stop_token& operator=(single_inplace_stop_token&&) noexcept;

    bool stop_possible() const noexcept;
    bool stop_requested() const noexcept;

    friend bool operator==(const single_inplace_stop_token& a,
                           const single_inplace_stop_token& b) noexcept;

  private:
    single_inplace_stop_souce* source;  // exposition only
  };

  template <std::invocable CB>
  class single_inplace_stop_callback {
  public:
    template <typename Initializer>
    requires std::constructible_from<CB, Initializer>
    single_inplace_stop_callback(single_inplace_stop_token st,
                                 Initializer&& init)
      noexcept(std::is_nothrow_constructible_v<CB, Initializer>);

    ~single_inplace_stop_callback();

    single_inplace_stop_callback(const single_inplace_stop_callback&) = delete;
    single_inplace_stop_callback(single_inplace_stop_callback&&) = delete;
    single_inplace_stop_callback& operator=(const single_inplace_stop_callback&) = delete;
    single_inplace_stop_callback& operator=(single_inplace_stop_callback&&) = delete;

  private:
    single_inplace_stop_source* source;  // exposition only
    CB cb;                               // exposition only
  };

  template <typename CB>
  single_inplace_stop_callback(single_inplace_stop_token, CB)
    -> single_inplace_stop_callback<CB>;
}

The semantics of these types are identical to that of the corresponding std::inplace_stop_token, std::inplace_stop_source and std::inplace_stop_callback<CB> types, with the exception that the std::single_inplace_stop_callback constructor has a pre-condition that there are no other stop-callback objects associated with the std::single_inplace_stop_token object passed to the constructor.

In cases where a sender algorithm has multiple child operations, where the number of child operations is statically known, and where the algorithm wants to be able to communicate a stop-request to all of the child operations, using an array of std::single_inplace_stop_source objects is, in most cases, still going to be more efficient than using a std::inplace_stop_source.

However, naively storing an array of std::single_inplace_stop_source objects still has some overheads due to redundancy in the data-structures in the case where a stop-request is communicated to all of the stop-sources at the same time (sequentially on the same thread).

The std::single_inplace_stop_source data-structure contains an atomic pointer and also an atomic std::thread::id which is used to determine whether stop-callback deregistration is occurring from inside a call to request_stop().

If we store an array of N std::single_inplace_stop_source objects, then we are storing N copies of this std::thread::id value, even though in this case, they will all contain the same value. We can save some storage in this case, by instead defining a data-structure that has N atomic pointers but only one atomic std::thread::id value.

Such a data-structure would have identical performance and layout to std::single_inplace_stop_source for N=1, but would save (N-1) pointers of storage for N>=2.

This paper proposes adding an implementation of such a data-structure, named std::finite_inplace_stop_source, which is templated on the desired number of independent stop-tokens that need to be supported.

The synopsis for this class-template is as follows:

namespace std
{
  template <size_t N, size_t Idx>
  class finite_inplace_stop_token;
  template <size_t N, size_t Idx, std::invocable CB>
  class finite_inplace_stop_callback;

  template <size_t N>
  class finite_inplace_stop_source {
   public:
    finite_inplace_stop_source() noexcept;
    ~finite_inplace_stop_source();

    finite_inplace_stop_source(const finite_inplace_stop_source&) = delete;
    finite_inplace_stop_source(finite_inplace_stop_source&&) = delete;
    finite_inplace_stop_source& operator=(const finite_inplace_stop_source&) = delete;
    finite_inplace_stop_source& operator=(finite_inplace_stop_source&&) = delete;

    bool stop_possible() const noexcept;
    bool stop_requested() const noexcept;

    bool request_stop() noexcept;

    template <size_t Idx>
      requires(Idx < N)
    finite_inplace_stop_token<N, Idx> get_token() const noexcept;
  };

  template <size_t N, size_t Idx>
  class finite_inplace_stop_token {
   public:
    template <typename CB>
    using callback_type = finite_inplace_stop_callback<N, Idx, CB>;

    finite_inplace_stop_token() noexcept;

    bool stop_possible() const noexcept;
    bool stop_requested() const noexcept;

    friend bool operator==(const finite_inplace_stop_token& a,
                           const finite_inplace_stop_token& b) noexcept;

   private:
    finite_inplace_stop_source<N>* source_;  // exposition-only
  };

  template <size_t N, size_t Idx, std::invocable CB>
  class finite_inplace_stop_callback {
   public:
    template <typename Init>
      requires std::constructible_from<CB, Init>
    finite_inplace_stop_callback(
        finite_inplace_stop_token<N, Idx> st,
        Init&& init) noexcept(std::is_nothrow_constructible_v<CB, Init>);

    ~finite_inplace_stop_callback();

    finite_inplace_stop_callback(const finite_inplace_stop_callback&) = delete;
    finite_inplace_stop_callback(finite_inplace_stop_callback&&) = delete;
    finite_inplace_stop_callback& operator=(const finite_inplace_stop_callback&) = delete;
    finite_inplace_stop_callback& operator=(finite_inplace_stop_callback&&) = delete;

  private:
    CB cb;                                  // exposition-only
    finite_inplace_stop_source<N>* source_; // exposition-only
  };

  template <size_t N, size_t Idx, typename CB>
  finite_inplace_stop_callback(finite_inplace_stop_token<N, Idx>, CB)
    -> finite_inplace_stop_callback<N, Idx, CB>;
}

An instance of finite_inplace_stop_source<N> has N separate associated finite_inplace_stop_token<N, Idx> stop-tokens, where Idx is in the range 0 .. N-1.

Each finite_inplace_stop_token<N,Idx> from a given stop-source can have at most one associated finite_inplace_stop_callback<N, Idx> object at a time.

When a call to request_stop() is made on the stop-source object, the stop-request is sent to all of the associated stop-tokens. Further, any stop-callbacks associated with any of the associated stop-tokens will be invoked.

The intent here is that it would be a valid implementation of finite_inplace_stop_source<N> to just hold an array of single_inplace_stop_source objects and to have request_stop() forward to a call to request_stop() on all of the single_inplace_stop_source objects. However, a high QoI implementation may choose to use a more efficient data-structure.

There is also the question of whether we should permit this class to be instantiated with a template-parameter of 0 or not. i.e. is it valid to write finite_inplace_stop_source<0>.

Such an object would not have the ability to obtain any associated stop-tokens and therefore would not have the ability to register any stop-callbacks. Ideally, such a type would compile out to nothing.

To enable this optimization, the finite_inplace_stop_source<N>::stop_possible() method returns N >= 1. This means that it will return false for N == 0, a case when there is no possibility of obtaining an associated-stop token that could observe a stop-request. Such a stop-source object is already possible with std::stop_source(std::nostopstate) and is called a disengaged stop-source.

This allows implementations to provide a specialization for finite_inplace_stop_source<0> that is an empty class, rather than having to have an atomic_flag data-member just to make sure that request_stop() returns true on first invocation and false on subsequent invocations.

For example, a possible implementation of this specialization may be:

namespace std
{
  template<>
  class finite_inplace_stop_source<0> {
  public:
    finite_inplace_stop_source() noexcept = default;

    finite_inplace_stop_source(const finite_inplace_stop_source&) = delete;
    finite_inplace_stop_source(finite_inplace_stop_source&&) = delete;
    finite_inplace_stop_source& operator=(const finite_inplace_stop_source&) = delete;
    finite_inplace_stop_source& operator=(finite_inplace_stop_source&&) = delete;

    bool stop_possible() const noexcept { return false; }
    bool stop_requested() const noexcept { return false; }
    bool request_stop() noexcept { return false; }
  };
}

Of the initial set of sender/receiver algorithms added in P2300R10, there are two algorithms which are currently specified to construct their own std::inplace_stop_source object which could be replaced with std::single_inplace_stop_source - std::execution::when_all() and std::execution::split().

Other than the types of stop-tokens returned from queries on the environments passed to child operations, there should be no changes in user-visible behaviour of these algorithms.

The proposed std::single_inplace_stop_token type adds extra pre-conditions to the construction of std::single_inplace_stop_callback objects which are not there for the std::inplace_stop_callback type.

This means that users of this stop-token type need to be more careful about its usage to ensure that only a single associated stop-callback exists at a time as violating this pre-condition can lead to undefined behaviour.

It also means that sender-algorithm implementers need to be more careful when forwarding a get_stop_token query to multiple child operations.

It is worth noting that in most cases where an algorithm that has multiple child operations with overlapping asynchronous operation lifetimes it will often want to explicitly control the cancellation behaviour. For example, by sending a stop-request to the other child operations when it receives a particular result from one of the children.

This tends to be an inherent part of designing a concurrent algorithm, and implementations that want to control cancellation will tend to need to introduce a new stop-source and register a stop-callback with the parent environment's stop-token that forwards to this new stop-source anyway, thus avoiding the problem of passing a single-callback-stop-token to multiple, concurrent child operations.

Authors of new concurrent sender algorithms tend to need to be aware of lots of the lifetime constraints anyway and implementing them is an advanced use of the sender/receiver framework.

The constraints that this paper proposes to put on usage of stop-tokens in a sender/receiver context should not affect most users, who we expect to be largely composing existing senders. As long as sender-algorithms abide by the restrictions, composing those algorithms together should be transaprent.

The benefits to users are that when they use sender-algorithms that take advantage of the single-callback constraints, their code uses less memory (operation-state objects are smaller) and runs more efficiently (uses less CPU-time).

While there is not yet a concrete proposal for a task coroutine type that integrates with sender/receiver, something that will need to be considered in such a proposal is what stop-token type the coroutine will provide in the environment connected to awaited senders.

On the one hand, a coroutine can only await a single child operation at a time, and so if the stop-token is only ever propagated to child operations by co_await expressions then it seems reasonable to have the task coroutine provide a std::single_inplace_stop_token rather than a std::inplace_stop_token so that we can take advantage of the better performance.

However, one of the use-cases that is not uncommon in task coroutines is to use the read_env() algorithm to obtain the current stop-token from the environment, and then to construct a stop-callback as a local variable in the coroutine.

For example: A coroutine that calls a low-level, cancellable OS API, using the coroutine's stop-token natively

void os_operation_start(void(*callback)(int, void*), void* data);
void os_operation_cancel(void* data);

task<int> dequeue() {
  struct state_t {
    async_manual_reset_event event;
    std::optional<int> result;
  };

  state_t state;

  auto on_stop_request = [&] noexcept { os_operation_cancel(&state); };

  auto on_complete = [](int result, void* data) noexcept {
    auto& state = *static_cast<state_t*>(data);
    state.result = result;
    state.event.set();
  };

  std::stoppable_token auto st =
    co_await std::execution::read_env(std::execution::get_stop_token);

  os_operation_start(on_complete, &state);

  {
    // Register a stop-callback that will cancel the os_operation if a
    // stop-request comes in before the operation completes.
    std::stop_callback_for_t<decltype(st), decltype(on_stop_request)> cb{st, on_stop_request};

    co_await state.event.wait();
  }

  if (st.stop_requested()) {
    // Complete with 'stopped' result
    co_await std::execution::just_stopped{};
  }

  co_return result.value();
}

In this example, the coroutine obtains the current stop-token using the read_env algorithm and then constructs a stop-callback associated with that stop-token. Then, while this stop-callback object is still alive, the coroutine then awaits on the async_manual_reset_event to suspend the coroutine until the callback passed to os_operation_start() is invoked.

However, the coroutine will also need to pass an environment with a stop-token down to all co_await expressions so that stop-requests can be transparently propagated through coroutines to child operations. However, the child operation might then go on to register its own stop-callback to that stop-token.

If the task coroutine were to use a single_inplace_stop_token for its stop-token then this would run into potential problems with trying to attach multiple stop-callbacks.

Therefore, it's likely that we either need to ban such usage within a coroutine, or we need the coroutine's stop-token type to be chosen to allow multiple stop-callbacks to be attached. e.g. by using inplace_stop_token instead.

This will mean that when you try to compose a task object as a sender into algorithms like when_all that there will need to be an adapter that adapts between the finite_inplace_stop_token, passed by when_all in the environment to the task, and a new inplace_stop_source that can produce an inplace_stop_token that can propagate through the chain of coroutines.

This is an example of a situation where trying to use a more efficient stop-token type can actually end up hurting performance in cases where you needed an inplace_stop_token anyway.

Yes, we still need to keep the inplace_stop_token family of types.

While it is still a generally useful facility, there are two main use-cases for it in the facilities targeting C++26.

The first is the use by the yet-to-be-proposed task type mentioned above.

The second is the use by a cancellable counting_scope type proposed in P3149.

In this case, a cancellable counting_scope may spawn an unbounded number of tasks running within the async-scope. If you want to cancel all of the operations in that scope then you need some way to send a stop-request to all of those spawned operations. The easiest way to do that is to have a single inplace_stop_source and then to just pass an associated inplace_stop_token in the environment passed to the spawned operation.

This benchmark tests the performance of registering and unregistering a single callback onto a single stop-source 100k times.

This benchmark tests the performance of calling request_stop() on various stop-source configurations when there are no associated stop-callbacks. This looks at situations where you might have a when_all() of multiple child operations, none of which are actually cancellable and thus none of them register any stop-callbacks, so that we can see the relative performance of different strategies.

For the inplace_stop_source this only needs to be run once as it has the same data-structure regardless of what the maximum number of children is.

For both single_inplace_stop_source and finite_inplace_stop_source we run the test in multiple configurations, evaluating for situations where different numbers of stop-callbacks are supported.

It is worth noting here that, for the case where there are no stop-callbacks registered, the inplace_stop_source is actually more efficient for all but the case where there is at most a single stop-callback. In this case, the request_stop() method only needs to acquire and release the lock once, so the overhead is fairly low.

In the case where we have a maximum callback count of N > 1, we end up needing to perform N separate atomic compare-exchange operations to check each potential stop-callback slot, and so the cost of this operation rises linearly with the maximum callback count.

In all cases for N > 1, the finite_inplace_stop_source has a slight performance advantage over multiple single_inplace_stop_source objects, which I mainly attribute to only having to call std::this_thread::get_id() once instead of N times and the fact it can use atomic-exchange operations for the second and subsequent slots instead of compare-exchange.

With the following benchmarks, we have a stop-source configuration that has a particular maximum number of callbacks that can be registered, and then some number of stop-callbacks registered when a call to request_stop() is made.

Here we measurethe performance of registering the stop-callbacks, calling request_stop() and then deregistering the stop-callbacks 100k times, all from a single thread.

The first group looks at the case where there is only a single stop-callback registered.

In the case where there are potentially multiple children but only a single stop-callback has been registered, the

The next group of results looks at the case where we register more than one stop-callback. In this case we are registering the same lambda multiple times.

Here we can see that as the number of registered stop-callbacks goes up, the benefit of the single_ and finite_ stop-source data-structures widens.

This is largely attributed to the relatively high cost of synchronization needed for each stop-callback with the inplace_stop_source data-structure - requiring to lock/unlock the structure and maintain the next/prev pointers of the doubly-linked list.

It is worth noting that the CPU branch predictor can have a large impact on the performance of these benchmarks. For example, if, instead of registering the same lambda 10x in the last test, we instead register different lambdas such that when invoking each of the registered stop-callbacks in turn it dispatches to a different function-pointer, the performance can be up to 3x slower (e.g. \~24000us for the finite_inplace_stop_source<10> benchmark).

In this test, we spin up two threads and have each thread simultaneously try to register and unregister a single stop-callback to a stop-source 100k times.

Each run is synchronized by a spin-barrier that tries to have each thread actively running rather than blocked in an OS synchronization call so that we can better evaluate the effect of contention on the stop-source data-structure from concurrent threads. In this sense, this is trying to evaluate the worst-case scenario of multiple threads continually registering/deregistering callbacks and conflicting with each other.

The times from both threads are added to the set of run-times and then the overall results are compared. i.e. it generates two time samples for each run.

In the case of inplace_stop_source, both threads try to register stop-callbacks to the same stop-source object.

In the case of single_inplace_stop_source, each thread tries to register its callback to a separate stop-source object. I've split this out into two variants to try to highlight the impact of false-sharing in this scenario. The first result in the table shows the performance if both single_inplace_stop_source objects live in the same cache-line. The only difference in the code between the second and third rows of the table is that the third row has aligned the stop-source objects sufficiently to ensure that they are placed in different cache-lines.

In the case of finite_inplace_stop_source<2>, both threads are given a reference to the same stop-source object, but each thread registers its stop-callback using a different stop-token. get_token<0>() for the first thread, and get_token<1>() for the second thread.

The results here are a lot more variable than the single-threaded benchmarks and so I have included the minimum, maximum, 50% percentile (median) and average measurements to get a better idea of the overall distribution of times.

In all of the results except the "no false sharing" result, the mean is skewed lower than the p50 results by some outliers which happened to get lucky and run fast because the threads happened to be scheduled in such a way to contend far less. So the minimum run-times are perhaps less useful to look at.

If we look, instead, at the p50 and average times then we see that all of the runs except "no false sharing" are typically running much slower than we'd expect from the single-threaded runs.

Much of this slow-down can be attributed to the impact of multiple threads conflicting with each other, trying to atomically modify the same cache line. In the case of inplace_stop_source this is a true conflicton the shared state, and in the case of the other data-structures, the conflicts are the result of false-sharing (accessing different atomic objects that live in the same cache-line).

This is evidenced by the fact that the "no false sharing" benchmark exhibiting much less variability and times that are much closer to that of the single-threaded performance.