Allow pack expansion in lambda init-capture

Contents

• Motivation
• Init-capture restriction history
• Proposal
• Example
• Acknowledgements
• References

Motivation

With the introduction of generalized lambda capture [1], lambda captures can be nearly arbitrarily complex and solve nearly all problems. However, there is still an awkward hole in the capabilities of lambda capture when it comes to parameter packs: you can only capture packs by copy, by reference, or by... std::tuple?

Consider the simple example of trying to wrap a function and its arguments into a callable to be accessed later. If we copy everything, the implementation is both easy to write and read:

template<class F, class... Args>
auto delay_invoke(F f, Args... args) {
    // the capture here can also be just [=]
    return [f, args...]() -> decltype(auto) {
        return std::invoke(f, args...);
    };
}

But if we try to be more efficient about the implementation and try to move all the arguments into the lambda? It seems like you should be able to use an init-capture and write:

template<class F, class... Args>
auto delay_invoke(F f, Args... args) {
    return [f=std::move(f), args=std::move(args)...]() -> decltype(auto) {
        return std::invoke(f, args...);
    };
}

But this runs afoul of very explicit wording from [expr.prim.lambda.capture]/17, emphasis mine:

A simple-capture followed by an ellipsis is a pack expansion. An init-capture followed by an ellipsis is ill-formed.

As a result of this restriction, our only option is to put all the args... into a std::tuple. But once we do that, we don't have access to the arguments as a parameter pack, so we need to pull them back out of the tuple in the body, using something like std::apply():

template<class F, class... Args>
auto delay_invoke(F f, Args... args) {
    return [f=std::move(f), tup=std::make_tuple(std::move(args)...)]() -> decltype(auto) {
        return std::apply(f, tup);
    };
}

Which gets even worse if what we wanted to do with that captured parameter pack was invoke a named function rather than a captured object. At that point, all semblance of comprehension goes out the window:

By copy	By move
template <class... Args> auto delay_invoke_foo(Args... args) { return [args...]() -> decltype(auto) { return foo(args...); }; }	template <class... Args> auto delay_invoke_foo(Args... args) { return [tup=std::make_tuple(std::move(args)...)]() -> decltype(auto) { return std::apply([](auto const&... args) -> decltype(auto) { return foo(args...); }, tup); }; }

We can do better.

Init-capture restriction history

For every init-capture a non-static data member named by the identifier of the init-capture is declared in the closure type. This member is not a bit-field and not mutable. The type of that member corresponds to the type of a hypothetical variable declaration of the form "auto init-capture ;", except that the variable name (i.e., the identifier of the init-capture) is replaced by a unique identifier.

One problem here is that an init-capture introduces a *named* member of the closure type. A class member name that names a pack would be a new notion, and would bring with it significant additional complications (such as the inability to determine syntactically whether a construct contains an unexpanded parameter pack).

[...]

Consider this:

template <typename T> void call_f(T t) {
    f(t.x...);
}

Right now, this is ill-formed (no diagnostic required) because "t.x" does not contain an unexpanded parameter pack. But if we allow class members to be pack expansions, this code could be valid -- we'd lose any syntactic mechanism to determine whether an expression contains an unexpanded pack. This is fatal to at least one implementation strategy for variadic templates. It also admits the possibility of pack expansions occurring outside templates, which current implementations are not well-suited to handle.

Since init-captures add named members to the closure type, allowing init-captures to be pack expansions risks introducing the same problem if those names are visible in *any* context outside the body of the lambda-expression itself.

Proposal

The proposal is to simply remove the restriction on pack expansions in init-capture, which requires defining a new form of parameter pack in the language. Proposed wording is as follows.

In 6 [basic] paragraph 3:

An entity is a value, object, reference, function, enumerator, type, class member, bit-field, template, template specialization, namespace, or parameter pack.

In 8.5.2.3 [expr.sizeof] paragraph 5:

The identifier in a sizeof... expression shall name a parameter pack. If the identifier is a parameter pack, theThe sizeof... operator yields the number of arguments provided for the parameter pack identifier. Otherwise, the sizeof... operator yields the number of introduced unique identifiers in the provided init-capture pack. sizeof... expression is a pack expansion (17.6.3 [temp.variadic]).

In 8.1.5.2 [expr.prim.lambda.capture] paragraph 17:

A simple-capture capture followed by an ellipsis is a pack expansion (17.6.3 [temp.variadic]). An init-capture followed by an ellipsis is ill-formed. An init-capture followed by an ellipsis introduces an init-capture pack (17.6.3 [temp.variadic]), whose declarative region is lambda-expression's compound-statement. Each init-capture in the pack expansion introduces a new init-capture with invented, unique identifier (and otherwise behaves as described above) that can only be referred to with the init-capture pack.[ Example:

template<class... Args>
void f(Args... args) {
  auto lm = [&, args...] { return g(args...); };
  lm();
  
  auto lm2 = [xs = std::move(args)...] { return g(xs...); };
  lm2();
}

- end example ]

In 8.1.6 [expr.prim.fold]:

A fold expression performs a fold of a template function parameter pack or an init-capture pack(17.6.3 [temp.variadic]) over a binary operator.

Unary left folds and unary right folds are collectively called unary folds. In a unary fold, the cast-expression shall contain an unexpanded parameter pack (17.6.3 [temp.variadic]).

An expression of the form (e1 op1 ... op2 e2) where op1 and op2 are fold-operators is called a binary fold. In a binary fold, op1 and op2 shall be the same fold-operator, and either e1 shall contain an unexpanded parameter pack or e2 shall contain an unexpanded parameter pack, but not both. If e2 contains an unexpanded parameter pack, the expression is called a binary left fold. If e1 contains an unexpanded parameter pack, the expression is called a binary right fold.

Add a new clause to 17.6.3 [temp.variadic] after paragraph 2:

An init-capture pack is an identifier standing in for a group of unique identifiers whose size matches that of the other unexpanded packs in the pack expansion where it is introduced. [ Note: The program is ill-formed if there is no such unique size. -- end note ]. [ Example:

template <typename... Args>
void foo(Args... args) {
    [xs = args...]{
        bar(xs...); // xs is an init-capture pack
    };
}

foo();  // OK: xs contains zero identifiers
foo(1); // OK: xs contains one identifier

- end example ]

A parameter pack is either a template parameter pack or a function parameter pack.

The section describing pack expansions in 17.6.3 [temp.variadic] paragraph 4 remains unchanged:

Pack expansions can occur in the following contexts:

• In a capture-list ([expr.prim.lambda]); the pattern is a capture.

Add a new bullet to 17.6.3 [temp.variadic] paragraph 7:

Such an element, in the context of the instantiation, is interpreted as follows:

• if the pack is a template parameter pack, the element is a template parameter of the corresponding kind (type or non-type) designating the type or value from the template argument; otherwise,
• if the pack is a function parameter pack, the element is an id-expression designating the function parameter that resulted from the instantiation of the pattern where the pack is declared. ; otherwise,
• if the pack is an init-capture pack, the element is an identifier designating the unique identifier.

Example

This would simplify all the code that currently relies on std::tuple just to solve this problem, in a way that we are already used to seeing pack expansion:

C++17 today	This proposal
template <class... Args> auto delay_invoke_foo(Args... args) { return [tup=std::make_tuple(std::move(args)...)]() -> decltype(auto) { return std::apply([](auto const&... args) -> decltype(auto) { return foo(args...); }, tup); }; }	template <class... Args> auto delay_invoke_foo(Args... args) { return [args=std::move(args)...]() -> decltype(auto) { return foo(args...); }; }

Acknowledgements

Thanks to Richard Smith, John Spicer, and Daveed Vandevoorde for considering the viability of this change. Thanks to Hubert Tong and Jens Maurer for help with wording.

References

[1] N3610: "Generic lambda-capture initializers, supporting capture-by-move"

[2] N3648: "Wording Changes for Generalized Lambda-capture"

[3] A problem with generalized captures and pack expansion

[4] CWG 1760: Access of member corresponding to init-capture