Parameter group placeholders for bind

Introduction

The aim of this proposal is to introduce a new class of placeholder that could be used with std::bind: a group placeholder that represents a set of (zero or more) call arguments.

This paper addresses LEWG Bug 40: variadic bind.

Motivation and Scope

Before going thought this chapter the author recommends the reader to familiarize with Appendix: Why we need bind when we have a generic lambda?.

In the scope of this proposal we introduce a new type of parameter placeholder: group placeholders that are to be replaced by zero or more arguments in the invocation of the stored functor. The meaning of a single placeholder is independent from the context, so if the given placeholder is used twice, appropriate set of values will be passed twice, reproducing existing behaviour.

The following group placeholders are proposed in this paper:

To complement set of group placeholders, the variable template representation of a single-argument placeholder named _at<N> is also proposed.

Binding object with member function

The most common use case for placeholder _all is to bind an object to a member function, effectively emulating the result of expressions obj.*ptr.

For example, given the following definitions:

struct Strategy { double process(std:string, std::string, double, double); };
std::unique_ptr<Strategy> createStrategy();

We want to create a functor that will invoke method process on a given strategy. Compare the two solutions, one using lambda, the other using bind:

[s = createStrategy()] (auto&&... args) -> decltype(auto) { return s->process(std::forward<decltype(args)>(args)...); }

std::bind(&Strategy::process, createStrategy(), _1, _2, _3, _4)

The lambda approach allow us to write a functor that will be immune to the changes in the number of method arguments, but it requires an explicit specification of the return type and the use of perfect forwarding to accomplish such a simple task. The solution that uses bind is pretty straightforward, but it requires a modification every time the number of parameters of the method changes. The extension proposed in this paper allows us to avoid this problem by writing:

std::bind(&Strategy::process, createStrategy(), _all)

The same problem is also addressed by paper N3702, by the introduction of a new version of function mem_fn:

std::mem_fn(&Strategy::process, createStrategy())

Although the solution presented in N3702 provides nice and concise syntax, it address only this specific use case, and can be easily emulated with std::bind and group placeholder:

template<typename Class, typename Member, typename Object>
auto mem_fn(Member Class::* mem_ptr, Object&& obj)
{ return std::bind(mem_ptr, std::forward<Object>(obj), _all); }

Argument list manipulation

With the proposed set of the placeholders a programmer is able to perform various types of manipulation with the argument list, including, but not limited to:

• insert new argument at Nth position: bind(f{}, _to<N>, val, _from<N>)
• replace Nth argument with given value: bind(f{}, _to<N>, val, _from<N+1>)
• change order of first and second argument: bind(f{}, _2, _1, _from<3>)
• swap positions of Nth and Kth argument: bind(f{}, _to<N>, _at<K>, _between<N+1, K>, _at<N>, _from<K+1>)
• forward only first N arguments: bind(f{}, _to<N>)
• drop first N arguments: bind(f{}, _from<N+1>)

Defining custom placeholders

The existing design of std::bind allows the programmer to specify his own placeholder types via the specialization of trait is_placeholder. It is used both to check if a given type P represents placeholder (is_placeholder<P>::value > 0) and to define the index of the call argument that will be passed as a replacement of the placeholder (is_placeholder<P>::value).

The extension proposed in this paper, preserves this functionality, and also adds the ability to create user-defined group placeholders. To achieve this goal the previous responsibility of the is_placeholder is divided into two separate type functions:

• is_placeholder that is used to determine if given type P represents a placeholder. To mark type as placeholder, the specialization is_placeholder<P> should be derived from integral_constant<int, K>, where K > 0.
• parameter_indices that is used to determine indices of call arguments that should be passed as replacement of the placeholder. This type trait accepts the number of call arguments passed to functor as a second parameter. To signal that arguments with indices i1, i2, ..., iK should be used instead of placeholder, the specialization of the parameter_indices<T, N> should be derived from integer_sequence<int, i1, i2, ..., iK>.

To preserve backward compatibility with the existing single argument placeholders, the default implementation of parameter_indices<P, N> is derived from integer_sequence<int, is_placeholder<P>::value> for every placeholder type P.

An example implementation of _args placeholder that accepts indices of arguments that should be forwarded is provided bellow:

template<int... Is>
struct args_placeholder {};

template<int... Is>
constexpr args_placeholder _args{};

namespace std
{
  template<int... Is>
  struct is_placeholder<args_placeholder<Is...>>
    : integral_constant<int, 1> {};

  template<int... Is, int N>
  struct parameter_indices<args_placeholder<Is...>, N>
   : integer_sequence<int, Is...> {};
}

Design Decisions

Use of variables templates to define placeholders

The variable templates are used to define placeholders instead of set of extern variables. This approach allows the programmer to compute positions of passed parameters at compile time, which is cumbersome in case of existing placeholders. In addition the author finds single definition of _from, instead list of _1onwards, _2onwards, ..., _Nonwards, being more elegant.

Naming of placeholders

Parameter group placeholder proposed in this paper has names that begins with underscore (_all, _from) instead of the most obvious all, from. These names was chosen to reduce risk name collision in code, that uses bind in combination with using directive for std::placeholders namespace. Example:

  std::vector<std::string> erase_empty(std::vector<std::string> v)
  {
    using namespace std;
    using namespace std::placeholders;

    auto from = remove_if(begin(v), end(v), bind(&string::empty, _1));
    v.erase(from, end(v));
    return v;
  }

Furthermore the author perceive this names as more consistent with the existing numbered placeholders (_1, _2, ...).

Number of parameters required by _from<N>

The addition of the _from<N> placeholder opens the question about its behaviour when the number of parameters passed to the forwarding call wrapper produced as a result of bind(&foo, _from<N>) is equal to N-1. There are two possible approaches:

1. forward no arguments in place of _from<N> to target callable object,
2. make such invocation ill-formed and require at least N arguments if the _from<N> is used.

The first behaviour was chosen by this proposal, because it is more general and allows to easily simulate the second one by passing _N, _from<N+1> instead of _from<N> as argument.

Non-type template argument of type int

The non-type template argument of _at, _from, _to, _between and parameter_indices has an int type, although they values are required to be non-negative. This decision was made to keep them consistent with existing is_placeholder trait, that uses int to represent index of forwarded parameter.

Impact On The Standard

This proposal has no dependencies beyond a C++14 compiler and Standard Library implementation. (It depends on perfect forwarding, varidatic templates, variable templates, decltype and trailing return types.)

Nothing depends on this proposal.

Proposed wording

Change the section 20.10 [function.objects]/2.

// 20.10.9, bind:
template<class T> struct is_bind_expression;
template<class T> struct is_placeholder;
template<class T, int N> struct parameter_indices;

template<class F, class... BoundArgs>
  unspecified bind(F&&, BoundArgs&&...);
template<class R, class F, class... BoundArgs>
  unspecified bind(F&&, BoundArgs&&...);

namespace placeholders {
  // M is the implementation-defined number of placeholders
  extern unspecified _1;
  extern unspecified _2;
  .
  .
  .
  extern unspecified _M;

  template<int N>
  unspecified _at;

  template<int N>
  unspecified _from;

  template<int N>
  unspecified _to;

  template<int B, int E>
  unspecified _between;

  extern unspecified _all;
}

Change the paragraph 20.10.9.1.2 Class template is_placeholder [func.bind.isplace].

is_placeholder can be used to detect the standard placeholders _all, _between<B, E>, _to<N>, _from<N>, _at<N>, _1, _2, and so on. bind uses is_placeholder to detect placeholders.

Instantiations of the is_placeholder template shall meet the UnaryTypeTrait requirements (20.11.1). The implementation shall provide a definition that has the BaseCharacteristic of integral_constant<int, J> if T is the type of std::placeholders::_J, otherwise it shall have a BaseCharacteristic of integral_constant<int, 0>.:

• integral_constant<int, 1> if T is the type of std::placeholders::_all,
• integral_constant<int, 1> if T is the type of std::placeholders::_from<N> and N > 0,
• integral_constant<int, 1> if T is the type of std::placeholders::_to<N> and N > 0,
• integral_constant<int, 1> if T is the type of std::placeholders::_between<B, E> and B > 0 and E >= B,
• integral_constant<int, J> if T is the type of std::placeholders::_at<J> or std::placeholders::_J,
• integral_constant<int, 0> otherwise.

A program may specialize this template for a user-defined type T to have a BaseCharacteristic of integral_constant<int, N> with N > 0 to indicate that T should be treated as a placeholder type.

After paragraph 20.10.9.1.2 Class template is_placeholder, insert a new paragraph. (Paragraph 20.10.9.1.3 Function template bind [func.bind.bind] becomes 20.10.9.1.?)

20.10.9.1.3 Class template parameter_indices [func.bind.paramidx]

namespace std {
  template<class T, int N> struct parameter_indices; // see below
}

bind uses parameter_indices to determine indices of parameters of the forwarding call wrapper to be forwarded to stored callable object as replacement for placeholder.

The implementation shall provide a definition of parameter_indices<T, N> that is publicly and unambiguously derived from:

• integer_sequence<int> if T is the type of std::placeholders::_all and N == 0,
• integer_sequence<int, 1, 2, ..., N> if T is the type of std::placeholders::_all and N > 0,
• integer_sequence<int> if T is the type of std::placeholders::_between<B,B> and N >= B-1,
• integer_sequence<int, B, B+1, ..., E-1> if T is the type of std::placeholders::_between<B,E> and B < E and N >= E-1,
• integer_sequence<int> if T is the type of std::placeholders::_to<1> and N >= 0,
• integer_sequence<int, 1, 2, ..., K-1> if T is the type of std::placeholders::_to<K> and N >= K-1,
• integer_sequence<int> if T is the type of std::placeholders::_form<K> and N == K-1,
• integer_sequence<int, K, K+1, ..., N> if T is the type of std::placeholders::_form<K> and N >= K,
• integer_sequence<int, j> if T is not one of the types described in the previous items and the value j defined as is_placeholder<T>::value is positive and N >= j,

A program may specialize or partially specialize parameter_indices template for a user-defined placeholder type to be publicly and unambiguously derived from integer_sequence<int, i1, i2, ..., iN> with values i1, i2, ..., iN greater than zero, to indicate indices of parameters of the forwarding call wrapper to be forwarded to stored callable object as replacement for placeholder.

A program is ill-formed if it necessitates the instantiation of parameter_indices<T, N> that does not satisfy criteria of any of the bullets in paragraph 1 and does not match a specialization or a partial specialization of template parameter_indices defined in the program.

Change the paragraph 20.10.9.1.3 Function template bind [func.bind.bind].

template<class F, class... BoundArgs>
  unspecified bind(F&& f, BoundArgs&&... bound_args);

Requires:

is_constructible<FD, F>::value shall be true. For each Ti in BoundArgs, is_constructible<TiD, Ti>::value shall be true. INVOKE (fd, w1, w2, ..., wN) (20.10.2) shall be a valid expression for some values w1, w2, ..., wN, where N == sizeof...(bound_args). fd shall be a callable object ([func.def] 20.10.1).

Returns:

A forwarding call wrapper g with a weak result type (20.10.2). The effect of g(u1, u2, ..., uM) shall be INVOKE (fd, std::forward<V1>(v1), std::forward<V2>(v2), ..., std::forward<VN>(vN), result_of<FD cv & (V1, V2, ..., VN)>::type) INVOKE (fd, std::forward<P1>(p1)..., std::forward<P2>(p2)..., ..., std::forward<PN>(pN)...), where cv represents the cv-qualifiers of g and the values and types of the elements of each of bound arguments v1, v2, ..., vN packs p1, p2, ..., pN are determined as specified below. The copy constructor and move constructor of the forwarding call wrapper shall throw an exception if and only if the corresponding constructor of FD or of any of the types TiD throws an exception.

Throws:

Nothing unless the construction of fd or of one of the values tid throws an exception.

Remarks:

The return type shall satisfy the requirements of MoveConstructible. If all of FD and TiD satisfy the requirements of CopyConstructible, then the return type shall satisfy the requirements of CopyConstructible. [ Note: This implies that all of FD and TiD are MoveConstructible. — end note ]

template<class R, class F, class... BoundArgs>
  unspecified bind(F&& f, BoundArgs&&... bound_args);

Requires:

is_constructible<FD, F>::value shall be true. For each Ti in BoundArgs, is_constructible<TiD, Ti>::value shall be true. INVOKE (fd, w1, w2, ..., wN) (20.10.2) shall be a valid expression for some values w1, w2, ..., wN, where N == sizeof...(bound_args). fd shall be a callable object ([func.def] 20.10.1).

Returns:

A forwarding call wrapper g with a weak result type (20.10.2). The effect of g(u1, u2, ..., uM) shall be INVOKE (fd, std::forward<V1>(v1), std::forward<V2>(v2), ..., std::forward<VN>(vN), R) INVOKE (fd, std::forward<P1>(p1)..., std::forward<P2>(p2)..., ..., std::forward<PN>(pN)..., R) , where cv represents the cv-qualifiers of g and the values and types of the elements of each of bound arguments v1, v2, ..., vN packs p1, p2, ..., pN are determined as specified below. The copy constructor and move constructor of the forwarding call wrapper shall throw an exception if and only if the corresponding constructor of FD or of any of the types TiD throws an exception.

Throws:

Nothing unless the construction of fd or of one of the values tid throws an exception.

Remarks:

The return type shall satisfy the requirements of MoveConstructible. If all of FD and TiD satisfy the requirements of CopyConstructible, then the return type shall satisfy the requirements of CopyConstructible. [ Note: This implies that all of FD and TiD are MoveConstructible. — end note ]

The values of theelements of each bound arguments v1, v2, ..., vN pack pi and their corresponding types V1, V2, ..., VN depend on the types TiD derived from the call to bind, number of parameter passed to invocation forwarding call wrapper M = sizeof...(UnBoundArgs) and the cv-qualifiers cv of the call wrapper g as follows:

• if TiD is reference_wrapper<T>, the argument isthe pack contains single element with value tid.get() and its type Vi isof type T&;
• if the value of is_bind_expression<TiD>::value is true, the argument isthe pack contains single element with value tid(std::forward<Uj>(uj)...) and its type Vi isof type result_of<TiD cv & (Uj&&...)>::type&&;
• if the value j of is_placeholder<TiD>::value is not zeropositive and parameter_indices<TiD, M> is derived from integer_sequence<int, j1, j2, ..., jK>, the argument is std::forward<Uj>(uj) and its type Vi is Uj&& the pack contains K elements with values std::forward<Uj1>(uj1), std::forward<Uj2>(uj2), ..., std::forward<UjK>(ujK) of types Uj1&&, Uj2&&, ..., UjK&& respectively;
• otherwise, the value isthe pack contains single element with value tid and its type Vi isof type TiD cv &.

Change the paragraph 20.10.9.1.4 Placeholders [func.bind.place].

namespace placeholders {
  // M is the implementation-defined number of placeholders
  extern unspecified _1;
  extern unspecified _2;
  .
  .
  .
  extern unspecified _M;

  template<int N>
  unspecified _at;

  template<int N>
  unspecified _from;

  template<int N>
  unspecified _to;

  template<int B, int E>
  unspecified _between;

  extern unspecified _all;
}

All placeholder types shall be DefaultConstructible and CopyConstructible, and their default constructors and copy/move constructors shall not throw exceptions. It is implementation-defined whether placeholder types are CopyAssignable. CopyAssignable placeholders’ copy assignment operators shall not throw exceptions.

A program that necessitates the instantiation of _at<N>, _from<N> or _to<N> with N <= 0 is ill-formed.

A program that necessitates the instantiation of _between<B, E> with B <= 0 or E <= 0 or B > E is ill-formed.

Implementability

Proposed change can be implemented as pure library extension in C++14. Implementation of bind function that conforms proposed wording can be found https://github.com/tomaszkam/proposals/tree/master/bind.

Acknowledgements

Jonathan Wakely originally proposed idea of multi parameter placeholders in discussion group ISO C++ Standard - Future Proposals.

Andrzej Krzemieński and Ville Voutilainen offered many useful suggestions and corrections to the proposal.

Appendix: Why we need bind when we have a generic lambda?

After the introduction of generic lambda and extensions to lambda capture, a portion of the C++ community expresses an opinion that std::bind is no longer necessary and should no longer be recommend and even become deprecated. The author disagrees with this opinion, and in support of his position, a number of use cases is discussed here that illustrate the superiority of std::bind over lambdas.

The author wants to emphasise that the aim of this section is to demonstrate situations where std::bind leads to more readable and less error prone code than lambdas. It is not to prove that bind should be used instead of lambda in every context.

Specifying return type

The default return type deduction for a lambda will preform return by value, which is an optimal approach when a built-in type is returned; for example when we use lambda to write a predicate (function returning bool) for STL algorithms; but it introduces performance overhead if returning by reference would be preferred.

Let's assume that we want to transform a vector of Employee (ve) in to a vector of full names.

std::transform(std::begin(ve), std::end(ve), std::back_inserter(vfn),
               [](const Employee& e) { return e.full_name(); });

If the full_name function returns a const std::string&, then above code will lead to copy of the string being created for every iteration to return value form the lambda and then element of vector will get initialized from this temporary. In this case cheap move construction will be used, but for legacy classes copy constructor may be used twice. To avoid the above problem we may specify the return type for a lambda.

std::transform(std::begin(ve), std::end(ve), std::back_inserter(vfn),
               [](const Employee& e) -> const auto& { return e.full_name(); });

This approach will fix above problems, but if the function full_name is changed to return by value, then the code will cause a dangling reference problem. To avoid such problems we may use the decltype(auto) deduction:

std::transform(std::begin(ve), std::end(ve), std::back_inserter(vfn),
               [](const Employee& e) -> decltype(auto) { return e.full_name(); });

If we attempt to repeat the same exercise using standard function wrappers, none of the above tough reasoning is necessary, and additionally we will benefit from a single syntax for handling method pointers and member pointers.

std::transform(std::begin(ve), std::end(ve), std::back_inserter(vfn),
               std::mem_fn(&Employee::full_name));

Passing arguments

One of the decisions that programmer must make when writing a lambda (and any other functions) is to decide how to pass arguments to it. If we write a comparator or a predicate that only checks the state of the object, then we can use const auto&. The choice becomes less obvious if we want to write a wrapper around a function that accepts arguments by value, for example:

std::string concat_several_times(std::string val, std::size_t n);

We want to create a function that will concatenate given string 3 times. Lets begin with:

[](std::string val) { return concat_serveral_times(std::move(val), 3); }

The above solution will create a temporary std::string every time the lambda is invoked, even if the C-style string is passed to the function. Also, we will always have a second temporary being created from the rvalue reference. In case of the std::string this will end up with cheap move-construction, but it may as well introduce additional copies for legacy classes that define only custom copy-construction. To avoid these problems we will use perfect forwarding to pass a parameter.

[](auto&& val) { return concat_serveral_times(std::forward<some_type>(val), 3); }

What type should we use as some_type? If we use decltype(val) then above code would be equivalent to:

template<typename T> void foo(T&& t) { bar(std::forward<T&&>(t)); }

Instead of the usual:

template<typename T> void foo(T&& t) { bar(std::forward<T>(t)); }

Are you ok with this additional rvalue reference? We could get rid of it by using std::remove_rvalue_reference_t<decltype(val)>, but according definition of std::forward, the behaviour is same in both cases. So finally, we can safely stick to:

[](auto&& val) { return concat_serveral_times(std::forward<decltype(val)>(val), 3); }

The std::bind creates functors that perfectly forwards all non-bound parameters, so we can equivalently use:

std:bind(&concat_serveral_times, _1, 3)

Capturing variables

In most common cases, when the closure does not outlive the context in which it was created and is invoked in the same thread of the execution, like when it is passed to STL algorithm, it is safe and optimal to use "capture all be reference" ([&]) semantics. For the situation when we want to pass closure, probably wrapped into std::function, outside the current context, then it is save to use "capture all by value" ([=]) — but only if we assume that we do not use any unmanaged pointer inside. However if we want to pass our functor to other thread of execution we need to be sure, that it would not be causing any data races, and they still may occur if some handler with shallow copy semantics is captured by a lambda (e.g. std::shared_ptr).

The above reasoning leads us to conclusion that when lambda is passed outside current context (either when passing it to other thread, or when returning it from a function) it is safer to explicitly specify the variables that should be captured. For example, given the following definitions:

struct Widget { void process(std::string&) const };
struct WidgetFactory { std::unqiue_ptr<Widget> create() };

void process_in_parallel(std::vector<std::string>& vs, WidgetFactory& factory)
{
  std::vector<std::future<void>> results;
  for (std::size_t i = 0; i < vs.size(); ++i)
    results.emplace_back(std::async(some_callback));
  for (std::future<void>& fut : results)
    fut.get();
};

We want to create a callback some_callback that will process a given element with a concrete widget. Our first attempt would be:

[&vs, &factory, i] { factory.create()->process(vs[i]); }

We have accidentally postponed the creation of Widget until the point when lambda is invoked, thus causing a concurrent invocation of factory method, which could cause a data race. To fix this we might try to create the widget and capture it from local context:

for (std::size_t i = 0; i < vs.size(); ++i)
{
  auto widget = factory.create();
  results.emplace_back(std::async([&vs, widget, i] { widget->process(vs[i]); }));
}

The above code will not compile because we want to copy a move-only type std::unique_ptr<Widget>. In addition, note that we are capturing the whole vector vs, although it is sufficient to use only one element in a single thread. Both of this issues may be fixed with a C++14 extended lambda capture:

[&elem = vs[i], widget = factory.create()] { widget->process(elem); }

What it effectively does is to bind (or 'fix') two parameters to a (member) function; the Standard Library already provides a component designed exactly for such purpose, named std::bind:

std::bind(&Widget::process, factory.create(), std::ref(vs[i]))

In contrast to the problem with creating a Widget, it is worth noticing that sometimes it is desired to capture some precomputed values in lambda. Suppose we want to find an Employee with the given first and last name:

std::find_if(std::begin(ve), std::end(ve), [](const auto& e) { return e.full_name() == first + ' ' + last; });

This innocent-looking code has a performance issue inside: the string first + ' ' + last is constant for every element, but a new instance is created in every iteration. To avoid such problems we should capture the value:

std::find_if(std::begin(ve), std::end(ve), [name = first + " " + last](const auto& e) { return e.full_name() == name; });

Although the use of std::bind would also eliminate the problem, the author recommends the use of lambda is such case, because nested std::bind (which is necessary in this situation) would render a less readable code:

std::find_if(std::begin(ve), std::end(ve), std::bind(std::equal_to<>, std::bind(&Employee::full_name, _1), first + ' ' + last));

Summary

The original aim of the lambda functions was to simplify writing of ad-hoc functors for use with STL algorithms, and indeed its design makes writing such predicates simple and efficient. As a consequence of such design, lambda is not efficient when used to write function wrappers. For example, let us compare the following simple bind expression and its lambda equivalent:

std::bind(&foo, _1, expr, std::ref(a));
[e = expr, &a] (auto&& arg) -> decltype(auto) { return foo(std::forward<decltype(arg)>(arg), e, a); };

The use of std::bind to write simple function wrappers allows the programmer to avoid running into correctness and performance problems described in this appendix. The choice between using bind and a lambda can be directly compared to the between the use of STL algorithms and writing a raw loop that performs the same task: both of the solution are feasible, but using the Standard Library is simpler.

Of course, std::bind has its limitations, and should be used only for the simple task of binding constant values to a specific set of function arguments. If we want to create a function that contains composition of the two functions or more complex expression we should use a lambda or even write a separate function if the expression is large enough. Another drawback of bind is that if we want to use it with overloaded function name we need resolve the ambiguity at the point of invocation of std::bind and casting to appropriate function pointer is required.

References

1. Chris Jefferson, Ville Voutilainen, "Bug 40 - variadic bind" (LEWG Bug 40, https://issues.isocpp.org/show_bug.cgi?id=40)
2. Mikhail Semenov, "Introducing an optional parameter for mem_fn, which allows to bind an object to its member function" (N3702, http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3702.htm)
3. Tomasz Kamiński, Implementation of bind function (https://github.com/tomaszkam/proposals/tree/master/bind)