Document Number: N2276=07-0136
Anthony Williams <anthony@justsoftwaresolutions.co.uk>
Just Software Solutions Ltd
2007-05-07
At Oxford, the combined EWG and LWG voted to proceed with work on Thread Pools and Futures for C++0x, even though this work had previously been destined for TR2. This paper is provided to further discussions in this area. It draws heavily on N2094 and N2185.
The building blocks of the system are three class templates: future
, packaged_task
, and
promise
.
future<R>
A future<R>
encapsulates a potential value of type R
, or an exception. When the
future
is ready, you can retrieve the encapsulated value or exception either by using the explicit
get()
member function, or by making use of the implicit conversion to R
. If the future
is
not ready, then the access functions will block until the future
becomes ready. If a
ready future
encapsulates a value, then the access functions will return that value. If a ready
future
encapsulates an exception, then the access functions will throw a copy of the encapsulated
exception.
packaged_task<R>
This is a wrapper for a callable object with a return type of R
. A packaged_task
is itself a
callable type — invoking it's function call operator will invoke the encapsulated callable object. This still doesn't get
you the value returned by the encapsulated object, though — to do that you need a
future<R>
which you can obtain using the get_future()
member function. Once the
function call operator has been invoked, all future
s associated with a particular packaged_task
will
become ready, and will store either the value returned by the invocation of the encapsulated object, or the exception
thrown by said invocation. packaged_task
is intended to be used in cases where the user needs to build a thread
pool or similar structure because the standard facilities do not have the desired characteristics.
promise<R>
This is an alternative means of creating future
s without a callable object — the value of type
R
to be returned by the associated future
s is specified directly by invoking the
set_value()
member function. Alternatively, the exception to be returned can be specified by invoking the
set_exception()
member function. This allows the result to be set from a callback invoked from code that has no
knowledge of the promise
or its associated future
s.
As well as the low level building blocks, this proposal also includes high level support in the form of the
thread_pool
class, and the launch_in_thread
and launch_in_pool
function templates.
launch_in_pool
launch_in_pool
submits a task to a system-supplied thread pool to be run. It takes a single parameter: the
function or callable object to run, and returns a future
which encapsulate the result of the function call. The
intent is that the library implementor can provide a thread pool that provides what they believe to be optimal performance
characteristics for their customers, taking advantage of system-specific information. It is intended that the vast majority of
applications that need thread pool functionality will use this function to allow the system to define an appropriate thread
pool.
launch_in_thread
launch_in_thread
is similar to launch_in_pool
, but it creates a separate thread dedicated to the
task being run, which terminates when the task completes. It takes a single parameter: the function or callable object to run,
and returns a future
which encapsulate the result of the function call. This is an enhancement over
std::thread
for tasks that need a dedicated thread, but where the caller wishes to capture the returned value or
any unhandled exception.
thread_pool
A thread_pool
is just that: a pool of threads. These threads will run for the lifetime of the
thread_pool
object. The user can submit a task to the thread_pool
, which will then be added to a queue
of tasks. When one of the threads in the pool becomes idle, it will take a task from the queue and execute it. Once the task is
complete, the thread will try and obtain a new task, and so on. A task can be submitted to the thread_pool
by
passing a callable object to the submit()
member function. In return, the caller is given a future
to
encapsulate the result of the task.
<threadpool>
synopsisnamespace std { template <typename R> class future; template <typename R> class future<R&&>; template <> class future<void>; template <typename R> class packaged_task; template <typename F> packaged_task<typename result_of<F()>::type> package_task(F const& f); template <typename F> packaged_task<typename result_of<F()>::type> package_task(F&& f); template <typename R> class promise; template <typename R> class promise<R&>; template <> class promise<void>; template <typename F> future<typename result_of<F()>::type> launch_in_pool(F const& f); template <typename F> future<typename result_of<F()>::type> launch_in_pool(F&& f); template <typename F> future<typename result_of<F()>::type> launch_in_thread(F const& f); template <typename F> future<typename result_of<F()>::type> launch_in_thread(F&& f); class thread_pool; class future_canceled; class future_result_moved; class promise_result_already_set; }
future
namespace std { template<typename R> class future { public: // copying future(const future& other); future& operator=(future const& other); // functions to retrieve the stored value operator R() const; R get() const; R move(); bool result_moved() const; // functions to check ready state, and wait for ready bool ready() const; bool has_exception() const; bool has_value() const; void wait() const; bool timed_wait(target_time const& wait_until) const; // cancel the task generating the result void cancel(); }; }
future
sfuture(const future& other);
Effects: The only public
constructor exposed by future
is the copy
constructor. The copy constructor will yield a second future
which is waiting for the same result as
*this
. Both the *this
and the copy will become ready when the result is available, and both
will return the same stored value, or throw the same exception when the result is accessed.
future& operator=(future const& other);
Effects: Change *this
to wait for the same result as other
. *this
and
other
will both become ready when the result is available, and both
will return the same stored value, or throw the same exception when the result is accessed.
Returns: *this
operator R() const;
Returns: get()
R get() const;
Effects: Blocks until *this
is ready, and retrieves the stored result. This function
is a cancellation point if ready()
would return false
on entry. If result_moved
would
return true
on entry, throws an exception of type future_result_moved
.
Returns: A copy of the stored value, if any.
Throws: An exception of type future_result_moved
as described above, or if there is no stored
value, throws a copy of the stored exception. If the copy constructor of the stored value throws an exception, this is
propagated to the caller.
R move();
Effects: Blocks until *this
is ready, and retrieves the stored result. This function
is a cancellation point if ready()
would return false
on entry. If result_moved
would
return true
on entry, throws an exception of type future_result_moved
.
Returns: A copy of the stored value x
, if any, as-if by return std::move(x)
.
Throws: An exception of type future_result_moved
as described above, or if there is no stored
value, throws a copy of the stored exception.
bool result_moved() const;
Returns: true
if move
has already been called on any of the future
s
associated with the same state as *this
, false
otherwise.
bool ready() const;
Returns: true
if *this
has a stored result (whether a value or an exception),
false
otherwise.
bool has_exception() const;
Returns: true
if *this
is ready with a stored exception,
false
otherwise.
bool has_value() const;
Returns: true
if *this
is ready with a stored value,
false
otherwise.
void wait() const;
Effects: Blocks until *this
is ready. This function is a cancellation point if
ready()
would return false
on entry.
bool timed_wait(target_time const& wait_until) const;
Effects: Blocks until *this
is ready or the specified time is reached. This
function is a cancellation point if ready()
would return false
on entry.
Returns: true
if *this
is ready, false
if the operation
timed out.
void cancel();
Effects: None, if *this
is ready on entry to this function. Otherwise attempts to
cancel the task generating the result asynchronously. If the cancellation attempt is successful, then *this
shall become ready, with a stored exception of type future_canceled
. This function shall return
immediately, and not wait for *this
to become ready.
future<void>
namespace std { template<> class future<void> { public: // copying future(const future& other); future& operator=(future const& other); // functions to retrieve the stored value void get() const; bool result_moved() const; // functions to check ready state, and wait for ready bool ready() const; bool has_exception() const; bool has_value() const; void wait() const; bool timed_wait(target_time const& wait_until) const; // cancel the task generating the result void cancel(); }; }
This specialization is identical to the primary template, except that there is no implicit conversion operator or
move
function, and get
, has_value
and result_moved
behave as described
below.
void get() const;
Effects: Blocks until *this
is ready. This function is a cancellation point if
ready()
would return false
on entry.
Returns: Nothing.
Throws: A copy of the stored exception, if any.
bool has_value() const;
Returns: true
if *this
is ready with no stored exception,
false
otherwise.
future<R&&>
namespace std { template<typename R> class future<R&&> { public: // copying future(const future& other); future& operator=(future const& other); // functions to retrieve the stored value operator R(); R get(); R move(); bool result_moved() const; // functions to check ready state, and wait for ready bool ready() const; bool has_exception() const; bool has_value() const; void wait() const; bool timed_wait(target_time const& wait_until) const; // cancel the task generating the result void cancel(); }; }
This specialization is identical to the primary template, except that get
behaves as described below.
operator R(); R get();
Returns: move().
packaged_task
namespace std { template<typename R> class packaged_task { private: // packaged_task is not copyable packaged_task(packaged_task&); // for exposition only packaged_task& operator=(packaged_task&); // for exposition only public: // construction and destruction template<typename F> explicit packaged_task(F const& f); template<typename F> explicit packaged_task(F&& f); packaged_task(packaged_task&& other); ~packaged_task(); // assignment packaged_task& operator=(packaged_task&& other); void swap(packaged_task& other); // result retrieval future<R> get_future(); // execution void operator()(); // cancellation void cancel(); }; }
packaged_task
template<typename F> packaged_task(F const& f); template<typename F> packaged_task(F&& f);
Effects: Construct a new packaged_task
that encapsulates a copy of the callable object f
.
packaged_task(packaged_task&& other);
Effects: Construct a new packaged_task
that encapsulates the callable object previously
encapsulated in other
. All future
s associated with other
become associated with the
newly constructed packaged_task
. other
no longer has an associated callable object or any associated
future
s.
~packaged_task();
Effects: Destroy the packaged_task
and its encapsulated callable object, if any. If the
encapsulated callable object has not yet been invoked, all future
s associated with *this
become
ready, with a stored exception of type future_canceled
.
packaged_task& operator=(packaged_task&& other);
Effects: As if:
packaged_task temp(other); temp.swap(*this);
Returns: *this
void swap(packaged_task& other);
Effects: *this
encapsulates the callable object previously encapsulated by other
;
other
encapsulates the callable object previously encapsulated by *this
. Any future
s
previously associated with other
become associated with *this
; any future
s previously
associated with *this
become associated with other
.
packaged_task
future<R> get_future();
Returns: A new future
associated with *this
. If the encapsulated callable object
has already been invoked, then the future
is ready, with the stored result the same as-if the
future
had been constructed prior to the invocation.
packaged_task
void operator()();
Effects: Nothing, if *this
has no encapsulated callable object, or the encapsulated callable
object has already been invoked. Otherwise, invokes the encapsulated callable object. If the invocation of the encapsulated
callable object returns normally, then all future
s associated with *this
become ready with a
stored value that is a copy of the value returned. If the invocation of the encapsulated callable object throws an exception,
then all future
s associated with *this
become ready with a stored exception that is a copy of
the exception thrown. If the invocation of the encapsulated callable object is cancelled, then the future
s
associated with *this
will become ready, with a stored exception of type future_canceled
.
packaged_task
void cancel();
Effects: Nothing, if *this
has no encapsulated callable object, or the invocation of the
encapsulated callable object has already completed. If the invocation of the encapsulated callable object has started on another
thread, but not completed, then the task is cancelled as-if by calling cancel()
on the std::thread
object for the thread in which the invocation of the encapsulated callable object is running. Otherwise, destroys the
encapsulated callable object without invoking it. All future
s associated with *this
become
ready, with a stored exception of type future_canceled
.
package_task
template<typename F> packaged_task<typename result_of<F()>::type> package_task(F const& f); template<typename F> packaged_task<typename result_of<F()>::type> package_task(F&& f);
Effects: Constructs a new packaged_task
encapsulating a copy of the supplied callable object
f
. The template parameter R
for the packaged_task
is the return type of
f()
.
Returns: The newly constructed packaged_task
.
promise
namespace std { template<typename R> class promise { private: // promise is not copyable promise(promise&); // for exposition only promise& operator=(promise&); // for exposition only public: // Construction and Destruction promise(); promise(promise&& other); ~promise(); // Assignment promise& operator=(promise&& other); void swap(promise& other); // Result retrieval future<R> get_future(); // Updating the state void cancel(); void set_value(R const& r); void set_value(R&& r); void set_exception(exception_ptr e); }; }
promise();
Effects: Create a new promise
with no stored value, no stored exception, and no associated
future
s.
promise(promise&& other);
Effects: Transfer the state currently associated with other
to *this
. All
future
s previously associated with other
become associated with *this
.
~promise();
Effects: If *this
currently has neither a stored value, nor a stored exception, all
future
s associated with *this
become ready, with a stored exception of type
future_canceled
.
promise& operator=(promise&& x);
Effects: As if:
promise temp(other); temp.swap(*this);
Returns: *this
void swap(promise& other);
Effects: *this
encapsulates the stored value or exception (if any) previously encapsulated by
other
; other
encapsulates the stored value or exception (if any) previously encapsulated by
*this
. Any future
s previously associated with other
become associated with
*this
; any future
s previously associated with *this
become associated with
other
.
promise
future<R> get_future();
Returns: A new future
associated with *this
. If *this
already has a
stored value or exception, then the future
is ready, with the stored value or exception,
respectively. Otherwise, the future
is not ready.
promise
void cancel();
Effects: Nothing, if *this
already has a stored value or exception. Otherwise, all
future
s associated with *this
become ready, with a stored exception of type
future_canceled
.
void set_value(R const& r); void set_value(R&& r);
Effects: If *this
already has a stored value or exception, throws an exception of type
promise_result_already_set
. Otherwise, all future
s associated with *this
become
ready, with a stored value that is a copy of r
. If the copy constructor of r
throws an
exception, the associated future
s become ready with a stored exception that is a copy of that exception,
as if set by set_exception(current_exception())
.
void set_exception(exception_ptr e);
Effects: If *this
already has a stored value or exception, throws an exception of type
promise_result_already_set
. Otherwise, all future
s associated with *this
become
ready, with a stored exception that is a copy of the exception referred to by e
.
promise<void>
namespace std { template<> class promise<void> { private: // promise is not copyable promise(promise&); // for exposition only promise& operator=(promise&); // for exposition only public: // Construction and Destruction promise(); promise(promise&& other); ~promise(); // Assignment promise& operator=(promise&& other); void swap(promise& other); // Result retrieval future<void> get_future(); // Updating the state void cancel(); void set_value(); void set_exception(exception_ptr e); }; }
This specialization behaves identically to the primary template, except that set_value
behaves as described
below.
void set_value();
Effects: If *this
already has a stored exception, or set_value
has already been
called, throws an exception of type promise_result_already_set
. Otherwise, all future
s associated with
*this
become ready, with no stored exception.
promise<R&>
namespace std { template<typename R> class promise<R&> { private: // promise is not copyable promise(promise&); // for exposition only promise& operator=(promise&); // for exposition only public: // Construction and Destruction promise(); promise(promise&& other); ~promise(); // Assignment promise& operator=(promise&& other); void swap(promise& other); // Result retrieval future<R&> get_future(); // Updating the state void cancel(); void set_value(R& r); void set_exception(exception_ptr e); }; }
This specialization behaves identically to the primary template, except that set_value
behaves as described
below.
void set_value();
Effects: If *this
already has a stored value or exception, throws an exception of type
promise_result_already_set
. Otherwise, all future
s associated with *this
become
ready, with a stored value that is a reference to r
.
launch_in_pool
template<typename F> future<typename result_of<F()>::type> launch_in_pool(F const& f); template<typename F> future<typename result_of<F()>::type> launch_in_pool(F&& f);
Effects: Submits a copy of f
to the system-supplied thread pool for execution. Constructs a
new future
associated with the result of invoking the submitted copy of f
. When the system thread pool
invokes the stored copy of f
, the future
will become ready, with a copy of the value returned
or the execution thrown by the invocation. The template parameter R
for the future
is the return type
of f()
. A task running in the system-supplied thread pool can submit a task with launch_in_pool
and
wait on the future
returned without causing deadlock.
Returns: The newly constructed future
.
launch_in_thread
template<typename F> future<typename result_of<F()>::type> launch_in_thread(F const& f); template<typename F> future<typename result_of<F()>::type> launch_in_thread(F&& f);
Effects: Start a new thread which will execute a copy of f
. Constructs a new
future
associated with the result of invoking the copy of f
. When the newly created thread has invoked
the stored copy of f
, the future
will become ready, with a copy of the value returned or the
execution thrown by the invocation. The template parameter R
for the future
is the return type of
f()
.
Returns: The newly constructed future
.
thread_pool
namespace std { class thread_pool { private: // thread_pool is not copyable thread_pool(thread_pool&); // for exposition only thread_pool& operator=(thread_pool&); // for exposition only public: // Construction and Destruction explicit thread_pool(unsigned thread_count); ~thread_pool(); // Submitting tasks for execution template<typename F> future<typename result_of<F()>::type> submit_task(F const& f); template<typename F> future<typename result_of<F()>::type> submit_task(F&& f); }; }
thread_pool
thread_pool(unsigned thread_count);
Effects: Create a new thread_pool
with thread_count
threads, and an empty task
queue.
~thread_pool();
Effects: Cancel all threads in the thread pool. Any tasks on the queue that have not yet been scheduled for
execution on a thread in the pool are cancelled: all future
s associated with such tasks become ready, with
a stored exception of type future_canceled
. Blocks until all threads in the pool have finished execution.
thread_pool
for executiontemplate<typename F> future<typename result_of<F()>::type> submit_task(F const& f); template<typename F> future<typename result_of<F()>::type> submit_task(F&& f);
Effects: Adds a copy of f
to the task queue associated with *this
for execution.
Constructs a new future
associated with the result of invoking the submitted copy of f
. When the
stored copy of f
is invoked by one of the threads in the pool, the future
will become ready,
with a copy of the value returned or the execution thrown by the invocation. The template parameter R
for the
future
is the return type of f()
.
Returns: The newly constructed future
.
Thanks to Peter Dimov and Howard Hinnant for their earlier papers, and feedback given on the thoughts presented here.
Thanks to Clark Nelson for allowing me to submit this revised draft after the deadline.