[Tooling] [isocpp-modules] Path to modules with old bad build systems

Michael Spencer bigcheesegs at gmail.com
Sat Feb 23 22:22:02 CET 2019


On Sat, Feb 23, 2019 at 8:18 AM Ben Craig <ben.craig at ni.com> wrote:

> I would like to find a way for users to decouple the upgrading of tools
> from the migration to modules.  I've got a half-baked suggestion on how to
> do so.  I think this has the potential to make the upgrade from C++17 to
> C++20 roughly the same cost to users as the upgrade from a C++14 to C++17.
> This was discussed some in the impromptu tooling session on Friday at Kona
> 2019.
>
> The no-build-system-upgrade constraint implies other constraints:
> 1. No up-front scanning of the source to find module name and dependency
> information, because a lot of current build systems don't currently have a
> scan step.
> 2. No dynamic dependencies between TUs.  Many current build systems assume
> that the .cpp -> .o[bj] transformation is trivially parallelizable.
> 3. No upgrade of build tool executables.  This has to work with versions
> of "make", "ninja", and "cmake" from 10+ years ago.
> 4. No drastically different file formats to parse (like binary module
> interfaces).
> 5. You _can_ add compiler / linker flags.
>
> The scheme I have in mind would result in no build throughput improvements
> with the old bad build systems, but I think it would still provide the
> isolation benefits of modules and be conforming.  When the user is able to
> upgrade their build system, they can start getting the build throughput
> improvements.
>
> The general idea is to treat the module interface file as a glorified
> header (Gaby has mentioned this possibility in various venues).  When the
> user passes --strawman-slow-modules to the compiler, the compiler does a
> textual inclusion of the module interface file (no BMI involved at all).
> The textual inclusion would likely involve placing a #pragma
> strawman-module begin(name-of-module) directive, with a #pragma
> strawman-module end(name-of-module) directive at the end of the module
> text.  Each TU will duplicate this work.  If the compiler can emit this
> text file, then it can be distributed using existing technologies that are
> expecting preprocessed files.  This is similar in nature to clang's
> -frewrite-modueles (I think that's the right spelling)
>
> So this requires that compilers support this textual modules approach.  It
> also requires that the compiler be able to find the module interface files
> without requiring the (dumb) build system to scan in advance.  The
> "easiest" (and slow) way to make this happen is to require that module
> names correspond to file names, and that compilers provide a search path.
> I am well aware that this isn't fast, but this general scheme is intended
> for build system compatibility.  Vendors should also provide a faster thing
> that can be used by newer build systems.  Compilers can also provide a
> command line override to say where a creatively named module can be found.
>
> Users would still need to build each module (as they have to build each
> .cpp) in order for all symbols to get defined.  This might disappoint some
> people that think that textual modules will provide behavior similar to
> "unity" / "blob" builds.  Non-inline function definitions in an imported
> module wouldn't have a strong linker definition (wrong words there, sorry)
> in importers... they would only be provided in the TU that defines that
> module.
>
> All of this is intended to allow a fully conforming modules
> implementation.  It also does not preclude additional build options
> intended for new, smart, fast, build systems.  To the contrary, this is an
> area that I encourage investigation and research.
>
> Let me know if there are holes in this plan, and if it sounds reasonable
> to implement.  Also let me know if this sounds like it won't help in
> keeping your existing tool or build system chugging along.
>

Ignoring distributed builds, this actually isn't really requiring anything
past a mapping from module names to files, as implementations can still
build the modules without ever producing that text.  I do however think
that it would be useful for other non compiler tools and distributed builds
to have this mode.

btw, the clang option is -frewrite-imports.

- Michael Spencer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.open-std.org/pipermail/tooling/attachments/20190223/caa8008d/attachment-0001.html 


More information about the Tooling mailing list