<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Thank you for this write up, Ben. This
all strongly matches what I've been thinking.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">The listed constraints suggest some
requirements:</div>
<div class="moz-cite-prefix">1. Means to determine if a header file
is a modular header unit.</div>
<div class="moz-cite-prefix">2. Means to map modules IDs to module
interface unit source file names.</div>
<div class="moz-cite-prefix">3. That pre-built packages provide:</div>
<div class="moz-cite-prefix">3.1. The above information for their
modular headers and module interface units (packaged software
can't require consumers to perform a scanning step).<br>
</div>
<div class="moz-cite-prefix">3.2. The source files for module
interface units (packaged software can't require consumers to
consume BMIs).<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Clang modules has already solved these
problems in a way that I believe 1) has proven deployable, 2) that
programmers have shown willingness to use. The following is
heavily based on how Clang modules works today.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Elaborating on #1 above. P1103R2
states in 2.3.4p2:</div>
<blockquote>
<div class="moz-cite-prefix">"When a <b>#include</b> appears
within non-modular code, if the named header file is known to
correspond to a legacy header unit, the implementation treats
the <b>#include</b> as an import of the corresponding legacy
header unit. The mechanism for discovering this correspondence
is left implementation-defined; there are multiple viable
strategies here (such as explicitly building legacy header
modules and providing them as input to downstream compilations,
or introducing accompanying files describing the legacy header
structure) and we wish to encourage exploration of this space.
An implementation is also permitted to not provide any mapping
mechanism, and process each legacy header unit independently."<br>
</div>
</blockquote>
<div class="moz-cite-prefix">For the purposes of the TR, we'll need
to define a mechanism for nominating a header file as a header
unit. Clang modules accomplishes this via a module.modulemap file
that must be co-located with the header file. When processing a
#include directive, Clang searches include paths normally for a
matching header file and, if a module.modulemap file is present,
scans it to see if the header file is associated with a defined
module. If it is, then the header is treated as a header unit,
otherwise (or if module support is disabled), it is treated as a
traditional header. I like this approach for several reasons:</div>
<div class="moz-cite-prefix">1. It doesn't require any new search
paths. Existing include paths suffice.</div>
<div class="moz-cite-prefix">2. Having the module map file
co-located with the header files it governs avoids complicated
path matching needs.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">If such a module map file is also used
to map module IDs to module interface unit source files, then we
can also avoid requiring separate search paths for module
interface units. An obvious consequence of this is that module
interface unit source files would need to be present in an include
path.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Here's a bikeshed example of a
hypothetical module map file. Please ignore concerns regarding
syntax for now.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><tt>// Definition of a header unit
module: <br>
</tt></div>
<div class="moz-cite-prefix"><tt>header module foo {</tt></div>
<div class="moz-cite-prefix"><tt> header: "foo.h";</tt></div>
<div class="moz-cite-prefix"><tt>}</tt><tt><br>
</tt></div>
<div class="moz-cite-prefix"><tt>// Definition of a module ID and
corresponding module interface unit source file:<br>
</tt></div>
<div class="moz-cite-prefix"><tt>module bar {</tt></div>
<div class="moz-cite-prefix"><tt> source_file: "bar.cppmi";</tt><tt><br>
</tt></div>
<div class="moz-cite-prefix"><tt>}</tt></div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">In principle, if we establish strong
conventions for associating module ID and module interface unit
source file names, then we can allow implicitly defined modules
without requiring them to be present in a module map file. Note
that header units can not be implicit however since an
implementation must assume that header files are just headers
unless otherwise informed.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">With this model, a compiler/tool need
only be supplied with include paths just as they are today.
Resolving a module import (whether via #include, import <>,
or import id) only requires scanning include paths for matching
header names and/or module map files.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Tom.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 2/23/19 1:17 PM, Ben Craig wrote:<br>
</div>
<blockquote type="cite"
cite="mid:DM6PR04MB462093E1D65D77111C15CD048B780@DM6PR04MB4620.namprd04.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt;
color: rgb(0, 0, 0); font-family: Calibri, Helvetica,
sans-serif, EmojiFont, "Apple Color Emoji",
"Segoe UI Emoji", NotoColorEmoji, "Segoe UI
Symbol", "Android Emoji", EmojiSymbols;">
<div>I would like to find a way for users to decouple the
upgrading of tools from the migration to modules. I've got a
half-baked suggestion on how to do so. I think this has the
potential to make the upgrade from C++17 to C++20 roughly the
same cost to users as the upgrade from a C++14 to C++17. This
was discussed some in the impromptu tooling session on Friday
at Kona 2019.</div>
<div><br>
</div>
<div>The no-build-system-upgrade constraint implies other
constraints:</div>
<div>1. No up-front scanning of the source to find module name
and dependency information, because a lot of current build
systems don't currently have a scan step.</div>
<div>2. No dynamic dependencies between TUs. Many current build
systems assume that the .cpp -> .o[bj] transformation is
trivially parallelizable.</div>
<div>3. No upgrade of build tool executables. This has to work
with versions of "make", "ninja", and "cmake" from 10+ years
ago.</div>
<div>4. No drastically different file formats to parse (like
binary module interfaces).</div>
<div>5. You _can_ add compiler / linker flags.</div>
<div><br>
</div>
<div>The scheme I have in mind would result in no build
throughput improvements with the old bad build systems, but I
think it would still provide the isolation benefits of modules
and be conforming. When the user is able to upgrade their
build system, they can start getting the build throughput
improvements.</div>
<div><br>
</div>
<div>The general idea is to treat the module interface file as a
glorified header (Gaby has mentioned this possibility in
various venues). When the user passes --strawman-slow-modules
to the compiler, the compiler does a textual inclusion of the
module interface file (no BMI involved at all). The textual
inclusion would likely involve placing a #pragma
strawman-module begin(name-of-module) directive, with a
#pragma strawman-module end(name-of-module) directive at the
end of the module text. Each TU will duplicate this work. If
the compiler can emit this text file, then it can be
distributed using existing technologies that are expecting
preprocessed files. This is similar in nature to clang's
-frewrite-modueles (I think that's the right spelling)</div>
<div><br>
</div>
<div>So this requires that compilers support this textual
modules approach. It also requires that the compiler be able
to find the module interface files without requiring the
(dumb) build system to scan in advance. The "easiest" (and
slow) way to make this happen is to require that module names
correspond to file names, and that compilers provide a search
path. I am well aware that this isn't fast, but this general
scheme is intended for build system compatibility. Vendors
should also provide a faster thing that can be used by newer
build systems. Compilers can also provide a command line
override to say where a creatively named module can be found.</div>
<div><br>
</div>
<div>Users would still need to build each module (as they have
to build each .cpp) in order for all symbols to get defined.
This might disappoint some people that think that textual
modules will provide behavior similar to "unity" / "blob"
builds. Non-inline function definitions in an imported module
wouldn't have a strong linker definition (wrong words there,
sorry) in importers... they would only be provided in the TU
that defines that module.</div>
<div><br>
</div>
<div>All of this is intended to allow a fully conforming modules
implementation. It also does not preclude additional build
options intended for new, smart, fast, build systems. To the
contrary, this is an area that I encourage investigation and
research.</div>
<div><br>
</div>
<div>Let me know if there are holes in this plan, and if it
sounds reasonable to implement. Also let me know if this
sounds like it won't help in keeping your existing tool or
build system chugging along.</div>
<div><br>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Modules mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Modules@lists.isocpp.org">Modules@lists.isocpp.org</a>
Subscription: <a class="moz-txt-link-freetext" href="http://lists.isocpp.org/mailman/listinfo.cgi/modules">http://lists.isocpp.org/mailman/listinfo.cgi/modules</a>
Link to this post: <a class="moz-txt-link-freetext" href="http://lists.isocpp.org/modules/2019/02/0089.php">http://lists.isocpp.org/modules/2019/02/0089.php</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>