“This is just a means to get a pointer to a function.”
― Ville Voutilainen
Adding a specific features to an existing software applications at runtime could be useful in many cases. Such extensions, or plugins, are usually implemented using shared library files (.dll, .so/Dynamic Shared Objects) loaded into the memory of program at runtime. Such trechique is called Dynamic Library Loading (DLL).
Current C++ Standard lacks support for Dynamic Library Loading. This proposal attempts to fix that and provide a simple to use classes and functions for DL.
A quick survey at one of the popular Russian IT sites shows that ~41% of people use dynamic loading in company or their own projects. Any wrapper around platform specific DL involves multiple macro, reinterpret_casts, work with platform specific decorations, and even UBs due to OS API designs. Examples: HPX, POCO (see SharedLibrary* files), Qt(see qlibrary_*.cpp files).
This proposal is based on the Boost.DLL library. Unlike many other libraries the Boost.DLL does not force a specific type for the imported symbols, does not include project specific stuff into the classes. It's just a abstraction around platform specific calls that attempts to unify DL behavior.
This proposal is a pure library extension. It does not propose changes to existing headers. It does not require any changes in the core language and it could be implemented in standard C++.
This proposal tries hard to beis a library only extension and avoids adding a description of shared libraries machinery as much as possible. EWG has to decide:
EWG voted on "Add DLL support to standard library?" in Albuquerque: 6 | 6 | 4 | 1 | 1, asked to embed feedback and send the proposal to LEWG.
std::filesystem::path
To simplify library usage std::filesystem::path
is used for specifying paths . All the path
related overhead is minor,
comparing to the time of loading shared library file into the memory of program or getting information about shared library file location.
Provide two overloads for some functions, one that throws an exception to report system errors, and another that sets an error_code
. This supports two common use cases:
C++ symbol mangling depend on compiler and platform. Existing attempts to get mangled symbol by unmangled name result in significant complexity and memory usage growth and overall slowdown of getting symbol from shared library file.
[ Example: // Loading a shared library file with mangled symbols mangled_library lib("libmycpp.so"); // Attempt to get function "foo::bar" with signature void(int) lib.get<void(int)>("foo::bar"); // The problem is that we do not know what `foo::bar` is: // * `foo` could be a class and `bar` could be a static member function // * `foo` could be a struct and `bar` could be a static member function // * `foo` could be a namespace and `bar` could be function in that namespace // // We also do not know the calling convention for `foo` and it's noexcept specification. // // Mangling of `foo::bar` depends on all the knowledge from above, so we are either forced to // mangle and try to obtain all the available combinations; or we need to investigate all the // symbols exported by "libmycpp.so" shared library file and find a best match. - end example ]
While no good solution was found for obtaining mangled symbol by unmangled name, this proposal concentrates on obtaining symbols by exact name match.
Keeping a shared library file loaded in memory of program while using the symbol obtained from shared library file could be
hard and error prone for some users. For the simplicity of those users, import
functions were provided. Those functions return a symbol
wrapped in a helper class along with an instance of shared_library
. In that way while
returned value in in scope, the shared
library file won't be unloaded from program memory, so the user does not need to take
care of shared library file lifetime in program memory.
import
function form Boost.DLL.While those function could simplify code development for some users, they are simple to missuse:
try {
auto f = dll::import<int()>(path_to_pugin, "function");
f();
// `f` goes out of scope along, plugin is unloaded, the exception::what() code is unreachable
} catch (const std::exception& e) {
std::cerr << e.what();
}
Searching paths for a specified shared library file may affect load times and make the proposed wording less efficient. It is assumed to be the most common case, that user exactly knows were the desired shared library file is located and provides either absolute or relative to current directory path. For that case searching system specific paths affects performance and increases the chance of finding wrong shared library file with the same name. Because some operating systems search system paths even if relative path is provided, the requirement to not do that is implicitly described in proposed wording. That OS specific logic could be avoided by implicitly converting relative paths to absolute.
There have been DLL related proposals:
The proposal you're reading attempts to avoid issues of the proposals from above, minifying the changes to the N4687 wording. The proposal you're reading:
const char*
and const string&
, not string_view
.OS specific API usually accepts symbol name as a zero terminated char array. string_view
has no zero termination
guarantee so it requires copying of the input parameter and zero termination. char*
and string&
is a more efficient API design.
'Dynamic loading' is a runtime mechanism to load shared library file into the memory of current program at any point of main
[basic.start.main] execution,
retrieve the addresses of functions and objects contained in the shared library file, execute those functions or access those objects,
unload the shared library file from memory.
'Shared library file' is a file containing set of objects and functions available for use at program startup or at runtime.
Headers <experimental/dll> and <experimental/import>
define classes and functions suitable for dynamic library load. For
those headersHeader <experimental/dll>
defines classes and functions suitable for dynamic loading. For
that header term 'symbol' relates to a
function, reference, class member or object that can be obtained from shared library file at runtime.
Term 'symbol name' relates to a character identifier of a symbol, using
which symbol can be obtained from shared library file. [ Note: For symbols declared with extern
"C"
in code of shared library file, 'symbol name' is the name of the
entity. 'symbol name' for entities without extern "C"
are unspecified. [ Note: Typically symbol names of those entities are mangled entity names — end note ].
Functions not having an argument of type error_code&
report errors as follows, unless otherwise specified:
error_code&
report errors as follows, unless otherwise specified:
error_code&
argument is set as
appropriate for the specific error. Otherwise, clear()
is called on the error_code&
argument.namespace std { namespace experimental { inline namespace dll_v1 { // shared library file load modes enum class dll_mode { default_mode, dont_resolve_dll_references, // DONT_RESOLVE_DLL_REFERENCES load_ignore_code_authz_level, // LOAD_IGNORE_CODE_AUTHZ_LEVEL rtld_lazy, // RTLD_LAZY rtld_now, // RTLD_NOW rtld_global, // RTLD_GLOBAL rtld_local, // RTLD_LOCAL rtld_deepbind, // RTLD_DEEPBIND append_decorations, // See [dll.dll_mode] search_system_directories // See [dll.dll_mode] }; constexpr dll_mode operator| (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator|=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode operator& (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator&=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode operator^ (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator^=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator~ (dll_mode& lhs) noexcept; // class to work with shared library files class shared_library; bool operator==(const shared_library& lhs, const shared_library& rhs) noexcept; bool operator!=(const shared_library& lhs, const shared_library& rhs) noexcept; bool operator< (const shared_library& lhs, const shared_library& rhs) noexcept; bool operator> (const shared_library& lhs, const shared_library& rhs) noexcept; bool operator<=(const shared_library& lhs, const shared_library& rhs) noexcept; bool operator>=(const shared_library& lhs, const shared_library& rhs) noexcept; // free functions template<class T> filesystem::path symbol_location(const T& symbol, error_code& ec); template<class T> filesystem::path symbol_location(const T& symbol); filesystem::path this_line_location(error_code& ec); filesystem::path this_line_location(); filesystem::path program_location(error_code& ec); filesystem::path program_location(); } } // support template <class T> struct hash; template <> struct hash<experimental::shared_library>; template <> struct hash<experimental::dll_mode>; }
The value of each enum dll_mode
constant shall be the same as the value
of the macro shown in the above synopsis if that macro is available on
the platform or 0 otherwise.
constexpr dll_mode operator| (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator|=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode operator& (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator&=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode operator^ (dll_mode lhs, dll_mode rhs) noexcept; constexpr dll_mode& operator^=(dll_mode& lhs, dll_mode rhs) noexcept; constexpr dll_mode operator~ (dll_mode& lhs) noexcept;
dll_mode
.template <> struct hash<experimental::dll_mode>;
The specialization is enabled (20.14.14).
The class shared_library
provides means to load shared library file into the memory of program,
check that shared library file exports symbol with specified symbol name and obtain that
symbol.
shared_library
instances share reference count to an actual shared library file loaded in memory
of current program, so it is safe and memory efficient to have multiple instances
of shared_library
referencing the same shared library file even if those instances
were loaded in memory using different paths (relative and absolute) referencing the
same shared library file.
[ Note:
It must beis safe to concurrently load shared library files into memory of program, unload and get symbols from
any shared library files using different shared_library
instances. If current
platform does
not guarantee safe concurrent work with shared library files, calls to shared_library
functions must beare serialized by shared_library
implementation.
— end note ]
[ Note:
Constructors, comparisons and reset()
functions that accept nullptr_t
are not provided because that may cause confusion:
some of the platforms provide interfaces for shared library files that accept nullptr_t
to get a handler to the current program.
— end note ]
namespace std { namespace experimental { inline namespace dll_v1 { class shared_library { public: using native_handle_type = platform-specific; // shared library file load modes using dll_mode = T1; static constexpr dll_mode default_mode; static constexpr dll_mode rtld_lazy; static constexpr dll_mode rtld_now; static constexpr dll_mode rtld_global; static constexpr dll_mode rtld_local; static constexpr dll_mode add_decorations; static constexpr dll_mode search_system_directories; // construct/copy/destruct shared_library() noexcept; shared_library(const shared_library& lib); shared_library(const shared_library& lib, error_code& ec); shared_library(shared_library&& lib) noexcept; explicit shared_library(const filesystem::path& library_path, dll_mode mode = dll_mode::default_mode); explicit shared_library(const filesystem::path& library_path); shared_library(const filesystem::path& library_path, dll_mode mode); shared_library(const filesystem::path& library_path, error_code& ec, dll_mode mode = dll_mode::default_mode); shared_library(const filesystem::path& library_path, dll_mode mode, error_code& ec); ~shared_library(); // public member functions shared_library& assign(const shared_library& lib, error_code& ec); shared_library& assign(const shared_library& lib); shared_library& operator=(const shared_library& lib); shared_library& operator=(shared_library&& lib) noexcept; void load(const filesystem::path& library_path, dll_mode mode = dll_mode::default_mode); void load(const filesystem::path& library_path, error_code& ec, dll_mode mode = dll_mode::default_mode); void load(const filesystem::path& library_path, dll_mode mode, error_code& ec); void reset() noexcept; explicit operator bool() const noexcept; bool has(const char* symbol_name) const noexcept; bool has(const string& symbol_name) const noexcept; template <typename SymbolT> SymbolT* get_if(const char* symbol_name) const noexcept; template <typename SymbolT> SymbolT* get_if(const string& symbol_name) const noexcept; template <typename SymbolT> SymbolT&auto get(const char* symbol_name) const -> conditional_t<is_member_pointer_v<SymbolT>, SymbolT, add_lvalue_reference_t<SymbolT>>; template <typename SymbolT> SymbolT&auto get(const string& symbol_name) const -> conditional_t<is_member_pointer_v<SymbolT>, SymbolT, add_lvalue_reference_t<SymbolT>>; native_handle_type native_handle() const noexcept; filesystem::path location() const; filesystem::path location(error_code& ec) const; // public static member functions static constexpr bool platform_supports_dll() noexcept; static constexpr bool platform_supports_dll_of_program() noexcept; }; } } }
shared_library
defines member bitmask type dll_mode
.
using dll_mode = T1;
EnumBitmask type dll_mode
provides modes for searching the shared library file and platform specific modes for loading the shared library file in memory of program.
[ Note: library users may extend available modes by
casting the required platform specific mode to dll_mode: [ Example: static_cast<dll_mode>(RTLD_NODELETE);
— end example ] — end note ]
Each of system family provides own modes, flags not supported by a particular platform must beare set to 0. Special modes are listened below:
appendadd_decorations
dll_mode
// Attempts to open // `./my_plugins/plugin1.dll` and `./my_plugins/libplugin1.dll` on Windows // `./my_plugins/libplugin1.so` on Linux // `./my_plugins/libplugin1.dylib` and `./my_plugins/libplugin1.so` on MacOS. // If that fails, loads `./my_plugins/plugin1` shared_library lib("./my_plugins/plugin1", dll_mode::appendadd_decorations);
rtld_lazy
0
if platform does not have RTLD_LAZY equivalent. Platform specific value of RTLD_LAZY equivalent otherwise.rtld_now
0
if platform does not have RTLD_NOW equivalent. Platform specific value of RTLD_NOW equivalent otherwise.rtld_global
0
if platform does not have RTLD_GLOBAL equivalent. Platform specific value of RTLD_GLOBAL equivalent otherwise.rtld_local
0
if platform does not have RTLD_LOCAL equivalent. Platform specific value of RTLD_LOCAL equivalent otherwise.default_mode
rtld_lazy | rtld_local
.search_system_directories
dll_mode
shared_library() noexcept;
shared_library
that does not reference any shared library file.*this
is false
.shared_library(const shared_library& lib); shared_library(const shared_library& lib, error_code& ec);
*this
reference the same shared library file that is referenced by lib
or if lib
does not reference shared library file
sets *this
to default constructed state.lib == *this
shared_library(shared_library&& lib) noexcept;
lib
to *this
and sets lib
to a default constructed state.lib
.lib
is false
, *this
is true
.explicit shared_library(const filesystem::path& library_path, dll_mode mode = dll_mode::default_mode); explicit shared_library(const filesystem::path& library_path); shared_library(const filesystem::path& library_path, dll_mode mode); shared_library(const filesystem::path& library_path, error_code& ec, dll_mode mode = dll_mode::default_mode); shared_library(const filesystem::path& library_path, dll_mode mode, error_code& ec);
library_path
is already loaded in memory of current program
makes *this
reference the previously loaded shared library file.
Otherwise loads a shared library file by specified path with a specified mode into the memory of current program,
executes all the platform specific initializations.library_path
and platform_supports_dll_of_program()
is true
.(mode &
dll_mode::appendadd_decorations)
is not 0.
Must attempt to loadLoads a shared library file from system specific shared library file
directories only if library_path
contains only shared library filename and
(mode & dll_mode::search_system_directories)
is not 0. During loads
from system
specific directories file name modification rules from above apply. If
with dll_mode::search_system_directories
more than one
shared library file has the same base name and extension, the function
loads in memory any first matching shared library file.
If (mode & dll_mode::search_system_directories)
is 0, then any
relative path or path that contains only filename must beis treated as a
path in current working directory.(mode &
~dll_mode::search_system_directories & ~dll_mode::appendadd_decorations)
converted to a platform specific
type representing shared library file load modes and adjusted to satisfy the
dll_mode::search_system_directories
requirements from above. If mode is
invalid for current platform, attempts to adjust the mode by
applying dll_mode::rtld_lazy
and reports error if resulting mode
is invalid. [ Example: If mode on POSIX was set to rtld_local
, then it will be adjusted
to rtld_lazy | rtld_local
. - end example ]shared_library::load()
with same parameters.~shared_library();
shared_library
by calling reset()
. If
the shared library file is referenced by different instances of
shared_library
,
the shared library file won't be unloaded until there is at least one
instance of shared_library
referencing it.
shared_library& operator=(const shared_library& lib);
assign(lib)
*this
shared_library& operator=(shared_library&& lib) noexcept;
*this
then calls
reset()
. Assigns the state of lib
to *this
and sets lib
to a default constructed state.lib
.lib
is false
, *this
is true
.*this
shared_library& assign(const shared_library& lib, error_code& ec); shared_library& assign(const shared_library& lib);
*this
, then calls reset()
.
Makes *this
reference the same shared library file as lib
.lib.location() == location()
, lib == *this
*this
void load(const filesystem::path& library_path, dll_mode mode = dll_mode::default_mode); void load(const filesystem::path& library_path, error_code& ec, dll_mode mode = dll_mode::default_mode); void load(const filesystem::path& library_path, dll_mode mode, error_code& ec);
void reset() noexcept;
*this
to a default constructed state. If *this
was the last instance of shared_library
referencing the shared library file,
then all the resources associated with shared library file shall be released.shared_library
referencing the shared library file.*this
is false
.explicit operator bool() const noexcept;
true
if *this
references a shared library file.bool has(const char* symbol_name) const noexcept; bool has(const string& symbol_name) const noexcept;
true
if *this
references a shared library file and symbol symbol_name could be obtained from shared library file.template <typename SymbolT> SymbolT* get_if(const char* symbol_name) const noexcept; template <typename SymbolT> SymbolT* get_if(const string& symbol_name) const noexcept;
symbol_name
, nullptr
otherwise.SymbolT
type for symbol. However implementations may
provide additional checks for
matching SymbolT
type and actual symbol type. [ Note: For
example implementations
may check that SymbolT
is a function pointer and that
symbol with symbol_name
allows execution. — end note ]template <typename SymbolT> SymbolT& get(const char* symbol_name) const; template <typename SymbolT> SymbolT& get(const string& symbol_name) const;
*get_if<SymbolT>(symbol_name)
.system_error
if symbol does not exist or if the shared library file was not loaded.native_handle_type native_handle() const noexcept;
filesystem::path location() const; filesystem::path location(error_code& ec) const;
*this
does not reference shared library file.static constexpr bool platform_supports_dll() noexcept;
false
, then any attempt to load a shared library file will fail at runtime. This function could be used to ensure platform capabilities at
compile time: static_assert(shared_library::platform_supports_dll(), "DL required for this program");
— end note ]static constexpr bool platform_supports_dll_of_program() noexcept
true
if according to platform capabilities shared_library(program_location())
may succeed.
shared_library
provides fast comparison operators that compare the referenced shared libraries file.
bool operator==(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() == rhs.native_handle()
bool operator!=(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() != rhs.native_handle()
bool operator<(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() < rhs.native_handle()
bool operator>(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() > rhs.native_handle()
bool operator<=(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() <= rhs.native_handle()
bool operator>=(const shared_library& lhs, const shared_library& rhs) noexcept;
lhs.native_handle() >= rhs.native_handle()
template <> struct hash<experimental::shared_library>;
template<class T> filesystem::path symbol_location(const T& symbol, error_code& ec); template<class T> filesystem::path symbol_location(const T& symbol);
symbol
filesystem::path this_line_location(error_code& ec); filesystem::path this_line_location();
this_line_location()
was calledfilesystem::path program_location(error_code& ec); filesystem::path program_location();
#include <experimental/dll> #include <type_traits> #include <memory> #include <functional> namespace std { namespace experimental { inline namespace dll_v1 { template <typename SymbolT> using imported_t = conditional_t<is_object_v<SymbolT>, shared_ptr<SymbolT>, function<SymbolT>>; // functions for importing a symbol template <typename SymbolT> imported_t<SymbolT> import(const filesystem::path& library_path, const char* symbol_name, dll_mode mode = dll_mode::default_mode); template <typename SymbolT> imported_t<SymbolT> import(const filesystem::path& library_path, const string& symbol_name, dll_mode mode = dll_mode::default_mode); template <typename SymbolT> imported_t<SymbolT> import(const shared_library& library, const char* symbol_name); template <typename SymbolT> imported_t<SymbolT> import(const shared_library& library, const string& symbol_name); template <typename SymbolT> imported_t<SymbolT> import(shared_library&& library, const char* symbol_name); template <typename SymbolT> imported_t<SymbolT> import(shared_library&& library, const string& symbol_name); } } }
import
functions are meant to simplify dynamic library loads of
symbols by keeping shared library file loaded in program memory while imported symbol is in scope:
[ Example: // Code of "/plugin_directory/libplugin.so" contains extern "C" void foo_function(string&&) auto foo_function = import<void(string&&)>("/plugin_directory/libplugin.so", "foo_function"); foo_function("Test"); - end example ]
template <typename SymbolT> imported_t<SymbolT> import(const filesystem::path& library_path, const char* symbol_name, dll_mode mode = dll_mode::default_mode); template <typename SymbolT> imported_t<SymbolT> import(const filesystem::path& library_path, const string& symbol_name, dll_mode mode = dll_mode::default_mode); template <typename SymbolT> imported_t<SymbolT> import(const shared_library& library, const char* symbol_name); template <typename SymbolT> imported_t<SymbolT> import(const shared_library& library, const string& symbol_name); template <typename SymbolT> imported_t<SymbolT> import(shared_library&& library, const char* symbol_name); template <typename SymbolT> imported_t<SymbolT> import(shared_library&& library, const string& symbol_name);
symbol_name
from shared library file and returns it wrapped in class that implements semantics
of shared ownership of shared library file; the last remaining owner is responsible for releasing the resources associated with the
shared library file. mode
must be passed to shared_library
constructor or shared_library::load
function.SymbolT
type for symbol.imported_t<SymbolT>
that keeps an instance an of shared_library
internallyshared_library
and symbol and provides access to the symbol only.system_error
if symbol does not exist or if failed to load the shared library file into the memory of program.For the purposes of SG10 it is sufficient to check for header <experimental/dll> or <experimental/import> using __has_include.
For the purposes of SG10, we recommend the feature-testing macro name __cpp_lib_shared_library.
Revision 3:
Revision 2:
string_view
load()
functionsimport
functionsdll_mode
description rewrittenRevision 1:
Revision 0:
[Boost.DLL] Boost DLL library. Available online at https://github.com/boostorg/dll
[N4618] Working Draft, Standard for Programming Language C++. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/n4618.pdf
[N1400] Toward standardization of dynamic libraries. Available online at http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2002/n1400.html
[N1418] Dynamic Libraries in C++. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2002/n1418.html
[N1428] Draft Proposal for Dynamic Libraries in C++. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1428.html
[N1496] Draft Proposal for Dynamic Libraries in C++ (Revision 1). Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1496.html
[N1976] Dynamic Shared Objects: Survey and Issues. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1976.html
[N2015] Plugins in C++. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2015.pdf
[N2407] C++ Dynamic Library Support. Available online at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2407.html
Klemens Morgenstern highlighted some of the missing functionality in Boost.DLL and provided implementation of mangled symbols load, that showed complexities of such approach. Renato Tegon Forti started the work on Boost.DLL and provided a lot of code, help and documentation for the Boost.DLL.
Thanks to Tom Honermann for providing comments and recommendations.