secure_val: a secure-clear-on-move typeDocument number: P1315R1 [latest]
Date: 2018-11-26
Author: Miguel Ojeda <miguel@ojeda.io>
Project: ISO JTC1/SC22/WG21: Programming Language C++
Audience: LWG, LEWG
Sensitive data, like passwords or keying data, should be cleared from memory as soon as they are not needed. This requires ensuring the compiler will not optimize the memory overwrite away. This proposal adds a secure_clear function to do so as well as a non-copyable secure_val class template which represents a memory area which securely clears itself on destruction and move.
When manipulating sensitive data, like passwords in memory or keying data, there is a need for library and application developers to clear the memory after the data is not needed anymore [1][2][3][4], in order to minimize the time window where it is possible to capture it (e.g. ending in a core dump or probed by a malicious actor). This requires ensuring the compiler will not optimize away the memory write. In particular, for C++, extra care is needed to consider all exceptional return paths.
For instance, the following function may be vulnerable, since the compiler may optimize the memset call away because the password buffer is not read from before it goes out of scope:
void f()
{
constexpr std::size_t size = 100;
char password[size];
getPasswordFromUser(password, size);
// ...
usePassword(password, size);
std::memset(password, size);
}
On top of that, usePassword could throw an exception (i.e. assume the stack is not overwritten and/or that the memory is held in the free store).
There are many ways that developers may use to try to ensure the memory is cleared as expected (i.e. avoiding the optimizer):
volatile pointer (e.g. decaf_bzero [5] from OpenSSL).memset through a volatile function pointer (e.g. OPENSSL_cleanse C implementation [6]).extern variable into it (e.g. CRYPTO_malloc implementation [7] from OpenSSL).OPENSSL_cleanse SPARC implementation [8]).memzero_explicit implementation [9] from the Linux Kernel).-fno-builtin-memset [10] in GCC).Or they may use a pre-existing solution whenever available:
explicit_bzero [11] from OpenBSD & FreeBSD, SecureZeroMemory [12] from Windows).memzero_explicit [13][9] from the Linux Kernel, OPENSSL_cleanse [14][6]).Regardless of how it is done, none of these ways is — at the same time — portable, easy to recognize the intent (and/or grep for it), readily available and avoiding compiler implementation details. The topic may generate discussions in major projects on which is the best way to proceed and whether an specific approach ensures that the memory is actually cleansed (e.g. [15][16][17][18][19]). Sometimes, such a way is not effective for a particular compiler (e.g. [20]). In the worst case, bugs happen (e.g. [21][22]).
C11 (and C++17 as it was based on it) added memset_s (K.3.7.4.1) to give a standard solution to this problem [4][23][24]. However, it is an optional extension (Annex K) and, at the time of writing, several major compiler vendors do not define __STDC_LIB_EXT1__ (GCC [25], Clang [26], MSVC [27], icc [28]). Therefore, in practical terms, there is no standard solution yet for C nor C++. A 2015 paper on this topic, WG14’s N1967 “Field Experience With Annex K — Bounds Checking Interfaces” [29], concludes that Annex K should be removed from the C standard.
Moreover, while ensuring that the memory is cleared as soon as possible is a good practise, there are other potential improvements when handling sensitive data in memory:
Most of these extra improvements, however, require either non-portable code or compiler support; which makes them a good target for standardization.
Other languages offer similar facilities in their standard libraries or as external projects:
SecureString class in .NET Core, .NET Framework and Xamarin [31].securemem package in Haskell [32].secstr library in Rust [33].In addition, there are also efforts on compilers to address these issues:
We can provide a simple secure_clear function that takes a pointer and a size to replace the memset but won’t be optimized away (with some extra notes explained later on):
std::secure_clear(password, size);
And also a secure_clear function template that takes any T& and clears it entirely:
std::secure_clear(password);
These solve the basic problem described above. However, we may go further: we can take advantage of move semantics to define an easy-to-use secure_val class template which securely handles sensitive values and can be passed around, if needed; but never copied. The class takes care of setting up the right storage for the sensitive data (e.g. locked memory) and clearing it on move and destruction. On top of that, it provides a single-point of access to such data (which may be used to provide extra guarantees, e.g. memory encryption or using special storage).
With this proposal, the aforementioned code could be re-written as:
void f()
{
constexpr std::size_t size = 100;
std::secure_val<char[size]> password;
password.write_access([](auto & data) {
getPasswordFromUser(data, size);
});
// ...
password.read_access([](const auto & data) {
usePassword(data, size);
});
}
With this relatively simple change the user has:
Note that making the class template as easy-to-use as possible and hard-to-misuse is critical due to the nature of the code. On this matter, move semantics help a great deal, because they enable us to disallow copying while providing the user with a way to safely move around the data. It also allows the user to write simple code like:
const auto password = getPasswordFromUser();
usePassword(std::move(password));
This proposal suggests adding a new header, <secure>, containing a secure_clear function, a secure_clear function template and a secure_val class template.
secure_clear functionnamespace std {
void secure_clear(void* data, size_t size) noexcept;
}
The secure_clear function is intended to make the contents of the memory range [data, data+size) impossible to recover. It can be considered equivalent to a call to memset(data, value, size) (with an unspecified value; there may even be different values, e.g. randomized). However, the compiler must guarantee the call is never optimized out unless the data was not in memory to begin with.
secure_clear), then the call could be elided. In a way, there was no memory to clear, so it could be considered that it was not optimized out.In addition, the compiler may provide further guarantees, like clearing other known copies of the data (e.g. in registers or cache).
secure_clear function templatenamespace std {
template <typename T>
void secure_clear(T& object) noexcept;
}
The secure_clear function template is equivalent to a call to secure_clear(addressof(object), sizeof(object)).
secure_val class templatenamespace std {
template <typename T>
class secure_val
{
T value_; // exposition only
static_assert(is_trivial_v<T>); // exposition only
public:
secure_val() noexcept;
secure_val(const secure_val<T>&) = delete;
secure_val(secure_val<T>&&) noexcept;
~secure_val();
secure_val<T>& operator=(const secure_val<T>&) = delete;
secure_val<T>& operator=(secure_val<T>&&) noexcept;
void clear() noexcept;
template <typename F>
void write_access(F f) noexcept(noexcept(f(std::declval<T&>())));
template <typename F>
void read_access(F f) const noexcept(noexcept(f(std::declval<const T&>())));
template <typename F>
void modify_access(F f) noexcept(noexcept(f(std::declval<T&>())));
void swap(secure_val<T>&) noexcept;
}
template <typename T>
void swap(secure_val<T>&, secure_val<T>&) noexcept;
}
The secure_val class template is intended to represent a value which will hold sensitive data (e.g. passwords). In particular, it will provide — at the very least — the following behaviour:
secure_clear was called on the data).secure_clear was called on the moved-from data).clear securely clears on demand (as if secure_clear was called on the data).write_access provides write access to the data during f invokation, passing a T&.
secure_clear was called on the data).f with a reference to the underlying value (which may contain encrypted data, which the user shouldn’t attempt to read) and, on return, encrypt it. It may not be deemed neccesary to securely clear the previous value before f invokation, since it will be overwritten.read_access provides read access to the data during f invokation, passing a const T&.
secure_clear was called on them).f with a const reference to it, and, on return, securely clear it.modify_access provides access to the data in a read-modify-write cycle during f invokation, passing a T&.
secure_clear was called on the data).secure_clear was called on them).f with a reference to it, and, on return, encrypt it again into the underlying data and securely clear the temporary (note: in contrast with write_access, which may not need to securely clear the data since the memory will be overwritten).The compiler/implementor may provide further guarantees, like:
read_access and modify_access invokation (and encrypting it in write_access and modify_access).
char-by-char); although it would require compiler support or a wrapper to the passed T& value.*_access invokations.Requirements:
T must be a TrivialType. This is intended to allow the compiler to treat the data as a simple memory block which can be easily managed, copied, encrypted, etc. and in order to provide any further guarantees.F (template parameter of the *_access member templates) must be Callable with a single parameter T& (for write_access and modify_access) and const T& (for read_access).A particular concern is that developers may try to create types like secure_val<string> (i.e. where the actual data is not protected). While it is hard to prevent all misusage, at least the particular case of secure_val<string> (and similar third-party string-like classes) is not allowed since string is not a TrivialType.
Removing secure_clear from this proposal (i.e. only providing secure_val).
secure_val class template itself.Instead of providing secure_clear, this proposal could make memset_s from C11 to be required in the C++ standard instead.
Removing secure_val from this proposal (i.e. only providing secure_clear).
secure_clear primitive, they can implement other solutions or write secure_val themselves (a subset of it). However, providing it with the standard library could make its use more widespread.secure_val.Other related functionality could be proposed under the same header in the future:
secure_string could be considered useful.get_password function taking a secure_val<char [N]> by reference (see, for instance, the getpass module in Python [40]).Several alternatives were considered for the prefix of secure_clear and secure_val as well as the name of the header:
explicit: follows explicit_bzero from OpenBSD & FreeBSD [11]. It collides with explicit (the keyword). It does not make for a good header name nor a good class template name. Only explicit_clear is a good choice.sensitive: follows a common word used in this context. sensitive_val is a very clear name. However, sensitive_clear does not look like an optimal choice.secure: follows SecureZeroMemory from Windows [12]. It is a good compromise and makes the purpose reasonably clear, for both the class template and the function. It may also allow the header to be more general (i.e. used to other security-related definitions).Several alternatives were considered for the second part of the name of the secure_clear function:
zero: follows explicit_bzero from OpenBSD & FreeBSD [11] and SecureZeroMemory from Windows [12]. Unambiguous, but the fact that the memory is cleared using the 0 value could be thought as an implementation detail.set: follows memset_s from C11. The name seems to suggest that a value is required (i.e. the value to overwrite memory with), which should not matter.clear: follows some C++ standard library member function (e.g. like those of the containers). It is more general: a verb commonly used with memory to refer to its deletion.Several alternatives were considered for the second part of the name of the secure_val class template. See the unique_val proposal for a discussion on it [41].
A trivial example implementation (i.e. without further compiler guarantees nor memory encryption) can be found at [42].
Thanks to Martin Sebor for pointing out the SECURE project talk at the GNU Tools Cauldron 2018.
memset function call (…) — https://www.viva64.com/en/w/v597/memset_s() to clear memory, without fear of removal — http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1381.pdfopenssl/crypto/ec/curve448/utils.c (old code) — https://github.com/openssl/openssl/blob/f8385b0fc0215b378b61891582b0579659d0b9f4/crypto/ec/curve448/utils.cOPENSSL_cleanse (implementation) — https://github.com/openssl/openssl/blob/master/crypto/mem_clr.copenssl/crypto/mem.c (old code) — https://github.com/openssl/openssl/blob/104ce8a9f02d250dd43c255eb7b8747e81b29422/crypto/mem.c#L143openssl/crypto/sparccpuid.S (example of assembly implementation) — https://github.com/openssl/openssl/blob/master/crypto/sparccpuid.S#L363memzero_explicit (implementation) — https://elixir.bootlin.com/linux/v4.18.5/source/lib/string.c#L706bzero, explicit_bzero - zero a byte string — http://man7.org/linux/man-pages/man3/bzero.3.htmlSecureZeroMemory function — https://msdn.microsoft.com/en-us/library/windows/desktop/aa366877(v=vs.85).aspxmemzero_explicit — https://www.kernel.org/doc/htmldocs/kernel-api/API-memzero-explicit.htmlOPENSSL_cleanse — https://www.openssl.org/docs/man1.1.1/man3/OPENSSL_cleanse.htmlOPENSSL_cleanse() #455 — https://github.com/openssl/openssl/pull/455SecureZeroMemory / RtlSecureZeroMemory? — https://stackoverflow.com/questions/13299420/memset() calls? — https://gcc.gnu.org/ml/gcc-help/2014-10/msg00047.htmlmemset optimized out in random.c — https://bugzilla.kernel.org/show_bug.cgi?id=82041memset, memset_s — https://en.cppreference.com/w/c/string/byte/memsetmemset_s in gcc 8.2 at Godbolt — https://godbolt.org/g/M7MyRgmemset_s in clang 6.0.0 at Godbolt — https://godbolt.org/g/ZwbkgYmemset_s in MSVC 19 2017 at Godbolt — https://godbolt.org/g/FtrVJ8memset_s in icc 18.0.0 at Godbolt — https://godbolt.org/g/vHZNrWSecureString Class — https://docs.microsoft.com/en-us/dotnet/api/system.security.securestringsecuremem: abstraction to an auto scrubbing and const time eq, memory chunk. — https://hackage.haskell.org/package/securememsecstr: Secure string library for Rust — https://github.com/myfreeweb/secstrunique_val: a default-on-move type — https://ojeda.io/cpp/unique_valsecure_val Example implementation — https://github.com/ojeda/secure_val/tree/master/proposal