Document Number

P1442R0
Date

2019-01-18
Project

Programming Language C++
Audience

Library Evolution Working Group
Summary

This paper presents a number of improvements to the Networking TS based on experience from Boost.

Overview

This paper presents a number of improvements that solve certain issues encountered when implementing high-performance, generic library components (mostly in Boost.Beast) based on top of the reference implementation of the Networking TS in the Boost.ASIO library.

The overall experience with using the Networking TS as a "base" has been positive. Unfortunately, networking at this level of abstraction is quite a complex topic and often confuses even experienced engineers. However, moving the TS "up" in terms of abstraction would come at a compromise to customizability, generality and performance. High-level, 3rd party libraries, built on top of the Networking TS, should be able to satisfy the the most common cases in a concise way.

The issues and improvements that solve them, presented in the rest of this paper, require only minor changes to the Networking TS. References to the TS are relative to N4771.

1. CompletionCondition cannot set an error code

Implementing parsing algorithms for protocols that work on top of a byte stream, like TCP/IP, often requires an online algorithm. Such an algorithm is able to process octets, read into a buffer, without having access to the entire input. This enables saving runtime resources (mostly memory) by facilitating the reuse of a single input buffer. Let us consider the following function object, which satisfies the constraints of CompletionCondition and represents a very simplified algorithm:

namespace net = std::experimental::net;

template<class DynamicBuffer>
struct condition
{
    std::size_t operator()(std::error_code const& ec, std::size_t n)
    {
        std::uint8_t message_type = 0xFF;
        net::buffer_copy(
            net::buffer(&message_type, sizeof(type)),
            db.data());

        switch (message_type)
        {
            case 0x00:
                // Not an error, the user gets a "success" error_code
                return 0;
            case 0x01:
                // Not an error, the user gets a "success" error_code
                return 0;
            default:
                // Is an error, we have no way of signalling
                // that an error occurred.
                return 0;
        }
    }

    DynamicBuffer& db_;
};

The problem encountered here is that the user of a networking algorithm (e.g. net::read()) has no way of knowing whether the algorithm completed because the condition was satisfied successfully or whether the condition encountered an irrecoverable error when parsing the input and stopped early.

Suggested actions:

  • Allow the CompletionCondition to accept an error_code& as its first argument.

  • Guarantee that changes made to the error code via the condition’s first argument are propagated to the caller (in case of synchronous algorithms) or completion handler(asynchronous algorithms).

2. CompletionCondition requires CopyConstructible

This requirement is unnecessary, because CompletionConditions are stored as a member of composed operations, which are only required to satisfy the requirements of MoveConstructible. This prevents users from efficiently reusing temporary storage, which might sometimes be necessary:

template<class DynamicBuffer>
struct condition
{
    std::size_t operator()(std::error_code const ec, std::size_t n);

    DynamicBuffer& db_;
    // Temporary storage that is used for parsing and potentially too large to be
    // allocate as a function-local array.
    std::vector<char> temporary_;
};
Suggested actions:

  • Relax the requirement of CompletionCondition from CopyConstructible to MoveConstructible.

3. Underspecified ownership of asynchronous initiation function arguments

"Caller owned" (non-const lvalue references) arguments of asynchronous initiation functions are required to remain valid until the operation’s completion handler is invoked ([async.reqmts.async.lifetime]). The following example presents a declaration of an initiation function template:

template <class AsyncReadStream, class CompletionToken>
auto async_wait_read(AsyncReadStream& s, CompletionToken&& token);

In this example, the argument s is required to remain valid until this operation’s completion handler is invoked. However, the lifetime requirements for operations that never complete are not specified. This most commonly happens when pending operations and completion handlers are "abandoned" in ExecutionContext::shutdown().

An operation can also be abandoned if the attempt to complete it (using ex2.dispatch()) results in an exception being thrown before invocation (e.g. when performing dynamic allocation of temporary storage). This has a surprising implication, that the destructor of a copy of a completion handler, that is neither moved-from, nor invoked, can be destroyed outside the context of a user’s executor.

Suggested actions:

  • Define "abandoning an operation" as "destruction of a completion handler that is not moved-from and has never been invoked".

  • Require that caller-owned initiation function arguments remain valid until operation’s completion handler is abandoned or invoked, whichever comes first.

  • Require that completion handler’s move constructor and destructor do not introduce data races if invoked outside its associated executor’s context. This allows implementations of ExecutionContexts to properly handle exceptions being thrown in threads not visible to the user.

4. Exception safety of temporary storage deallocation

Allocations made through a completion handler’s associated allocator must be deallocated before the completion handler is invoked. When performing this deallocation step, whether it happens before invocation or abandonment, the implementation has to move the handler out of the temporary storage:

Handler release_handler(temporary_storage<Handler>* storage)
{
    allocator_type alloc{net::get_associated_allocator(storage->handler_)};
    std::allocator_traits<allocator_type> traits;
    auto h = std::move(storage->handler_); // can throw
    traits.destroy(alloc, storage);
    traits.deallocate(alloc, storage, 1);
    return h;
}

CompletionHandlers are MoveConstructible, which implies their move constructors can throw. The problem this creates, is that move constructors of handlers are often invoked in "cleanup" contexts. In the above example, the exception thrown out of the move constructor would propagate out to the caller, but it results in a memory leak. The leak cannot be solved with either RAII or a catch clause, because by the time we catch the exception or execute a destructor we do not have access to a copy of the handler that is neither invoked nor moved-from, therefore we have to assume that the memory resource represented by the allocator is no longer valid:

Handler release_handler(temporary_storage<Handler>* storage)
{
    allocator_type alloc{net::get_associated_allocator(storage->handler_)};
    std::allocator_traits<allocator_type> traits;
    try {
        auto h = std::move(storage->handler_); // can throw
        traits.destroy(alloc, storage);
        traits.deallocate(alloc, storage, 1);
        return h;
    } catch(...) {
        // alloc may no longer be valid
        traits.destroy(alloc, storage);
        // even if alloc was still valid and the ownership remained in the source object,
        // we just destroyed it, so the next line uses an allocator which
        // may not refer to a valid memory resource
        traits.deallocate(alloc, storage, 1);
        throw;
    }
}
There are multiple solutions to this issue:

  • Require that the allocator copy have shared ownership of the memory resource or that the handler does not participate in ownership of the memory resource at all.

  • Disallow throwing move constructors.

  • Allow the implementation to call the move constructor in a noexcept context (effectively means that handler move constructors can can throw, but may result in termination in some contexts).

The first option severely limits the usage patterns of allocators associated with a completion handlers and misuse is impossible to detect.

The second one, requires the author of a composed operation’s state machine to manually define the move constructor, to make it possible to use some common types that have potentially throwing move constructors (e.g. std::map) as non-static data members.

The last one seems to allow most flexibility in terms of usage patterns of associated allocators. Calls to move constructors of completion handlers may occur in contexts, in which an exception being thrown already results in termination. An example of such a context would be a completion handler being moved in a call to net::dispatch() when the operation completes, which can be performed by a private thread spawned by the implementation.

5. Lack of ordering guarantees of a single-threaded io_context

A very common pattern of usage in application based on the Networking TS is using an "implicit strand". The user makes sure that no data races occur, by only allowing at most one thread to run completion handlers on a particular ExecutionContext. The problem is that io_context lacks ordering guarantees in the single-threaded case, which might be problematic to some higher-level components built on top of it, that assume FIFO ordering (which is provided by net::strand). The following snippet presents where this problem may be observable:

int main()
{
    net::io_context io;
    net::post(io, []{ std::cout << "op1 "; });
    net::post(io, []{ std::cout << "op2"; });
    io.run();
}

The user might expect the output to be op1 op2, but surprisingly there are no ordering guarantees even for this implicitly synchronized case. It is unlikely the implementation can gain anything by using LIFO ordering for executing completion handlers.

Suggested actions:

  • Provide an ordering guarantee for io_context if it only has 1 thread running it.

Acknowledgements

Many thanks to Vinnie Falco for feedback on drafts of this paper.