<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, 11 Jan 2019 at 23:31 Tom Honermann <<a href="mailto:tom@honermann.net">tom@honermann.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<div class="m_-7578053289813035424moz-cite-prefix">On 1/11/19 9:29 AM, Ben Craig wrote:<br>
</div>
<blockquote type="cite">
<div class="m_-7578053289813035424WordSection1">
<p class="MsoNormal">> The manifest file could be generated
(based on information in source code) and<u></u><u></u></p>
<p class="MsoNormal">> usable by multiple tools, IDEs, and
build systems. It need not be a statically<u></u><u></u></p>
<p class="MsoNormal">> maintained file. I like this approach
because it appropriately separates the<u></u><u></u></p>
<p class="MsoNormal">> steps of identifying/building the
module name/file map vs using the module<u></u><u></u></p>
<p class="MsoNormal">> name/file map (most tools don't want
to be build systems).<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">This sounds like a clang compilation
database.</p>
</div>
</blockquote></div><div bgcolor="#FFFFFF" text="#000000"></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">I don't think so. It is a database, but not one that stores the
history of how a build was performed. Rather, it stores some of the
information that would be needed to construct a compiler invocation
(or for a tool/IDE to parse source files that require resolving
module names to source files).</div></blockquote><div><br></div><div><br></div><div>It's exactly what a compilation database is ( it includes informations such as include directories, defines and some flags) for every translation unit</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<div class="m_-7578053289813035424WordSection1">
<p class="MsoNormal">Such a thing is useful for non-build tools,
and can be generated automatically, as you said. I am
concerned that generating such a file early in the build
process would cause performance problems. Each source file
would need to be partially parsed before the build DAG could
be fully formed. However, I don’t have benchmarks indicating
how expensive an extra pass over all the source files is.</p>
</div>
</blockquote>
</div><div bgcolor="#FFFFFF" text="#000000"><p>Without such a file (or equivalent information statically encoded
in "the" build system), the only alternative is to examine every
source file in order to construct the build DAG in memory anyway.
How else could such a scan be avoided? </p></div></blockquote><div><br></div><div>Build systems will always need to parse every file at least once before invoking the build.<br></div><div>They can hopefully extract both the name and the dependency in a single pass.</div><div><br></div><div>The issue with lack of module mapping is that the build system can't really have a top-down</div><div>approach that would ensure some nodes of the build graph are fully resolved soon after</div><div>the build starts.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><p>(There is another
alternative; implicitly building modules on demand as is done for
Clang modules today. But to my knowledge, no implementors are
pursuing that for standard proposed modules).</p></div></blockquote><div><br></div><div>I imagine this would solve a lot of issues : Give the compiler a bunch of source files and let it figure things out. It's more or less how rust and go works unless I'm mistaking?</div><div><br></div><div>I highly recommand this article about how that problem can be solved in D</div><div><a href="https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/">https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/</a><br></div><div><br></div><div>I don't know if implementers would be willing to move in this direction ?</div><div><br></div><div>However, it's important to note that even if the compiler handle the module dependency itself</div><div>it would also benefit from a fast module -> file mapping, even more so than build systems.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><p>
</p></div><div bgcolor="#FFFFFF" text="#000000">
<p>Tom.<br>
</p>
<blockquote type="cite">
<div class="m_-7578053289813035424WordSection1">
<p class="MsoNormal"><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b>
<a class="m_-7578053289813035424moz-txt-link-abbreviated" href="mailto:tooling-bounces@open-std.org" target="_blank">tooling-bounces@open-std.org</a>
<a class="m_-7578053289813035424moz-txt-link-rfc2396E" href="mailto:tooling-bounces@open-std.org" target="_blank"><tooling-bounces@open-std.org></a>
<b>On Behalf Of </b>Tom Honermann<br>
<b>Sent:</b> Thursday, January 10, 2019 9:27 PM<br>
<b>To:</b> WG21 Tooling Study Group SG15
<a class="m_-7578053289813035424moz-txt-link-rfc2396E" href="mailto:tooling@open-std.org" target="_blank"><tooling@open-std.org></a><br>
<b>Subject:</b> Re: [Tooling] Modules naming<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 1/10/19 5:17 PM, Ben Craig wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Can you elaborate more on the kinds of
historical pains caused by tying a #include directive to a
file name? I know of issues with #pragma once, but that
feels like a distinct problem from file names.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">> This is based on decades of
experience caused by header files. <u></u><u></u></p>
<p class="MsoNormal">I think most of the participants in
wg21 have years, often decades of experience with header
files. I know of plenty of issues with the preprocessor,
but I am not yet aware of any major problems on the file
name front. (ok, getting the right slashes can be
annoying… but it’s not a huge problem for me personally).<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I’m not a fan of the MANIFEST / module
map approach in general. It requires duplicating
information that is already in the source. I get that it
has the potential to speed up builds, but I’d rather not
have to update another location when I add a new .cpp file
to my project. Many build systems allow for the user to
make the tradeoff in whether they will use a file system
glob to enumerate their source, or require the user to
list the source manually. I usually fall into the file
system glob crowd.<u></u><u></u></p>
</blockquote>
<p>The manifest file could be generated (based on information
in source code) and usable by multiple tools, IDEs, and
build systems. It need not be a statically maintained
file. I like this approach because it appropriately
separates the steps of identifying/building the module
name/file map vs using the module name/file map (most tools
don't want to be build systems).<u></u><u></u></p>
<p>Tom.<u></u><u></u></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> <a href="mailto:tooling-bounces@open-std.org" target="_blank">tooling-bounces@open-std.org</a>
<a href="mailto:tooling-bounces@open-std.org" target="_blank"><tooling-bounces@open-std.org></a>
<b>
On Behalf Of </b>Gabriel Dos Reis<br>
<b>Sent:</b> Thursday, January 10, 2019 3:15 PM<br>
<b>To:</b> WG21 Tooling Study Group SG15 <a href="mailto:tooling@open-std.org" target="_blank"><tooling@open-std.org></a><br>
<b>Subject:</b> Re: [Tooling] Modules naming<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Microsoft strongly encourages its
developers and customers to NOT tie a module name with
the containing source file of its interface. This is
based on decades of experience caused by header files.
I would rather see us move in the direction of some sort
of MANIFEST file that map modules to source files and
artifacts.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> <a href="mailto:tooling-bounces@open-std.org" target="_blank">tooling-bounces@open-std.org</a>
<<a href="mailto:tooling-bounces@open-std.org" target="_blank">tooling-bounces@open-std.org</a>>
<b>On Behalf Of </b>Corentin<br>
<b>Sent:</b> Thursday, January 10, 2019 6:53 AM<br>
<b>To:</b> WG21 Tooling Study Group SG15 <<a href="mailto:tooling@open-std.org" target="_blank">tooling@open-std.org</a>><br>
<b>Subject:</b> [Tooling] Modules naming<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Hello.<u></u><u></u></p>
<div>
<p class="MsoNormal">I would like to suggest two
modules related proposals that I think SG15 should
look at.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-<b> Compiler enforced mapping
between module names and module interface file
(resource) name. </b><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Currently, modules interfaces
can be declared in any file - which makes
dependency scanning more tedious than it needs to
be and have performance implications<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">(The build system needs to open
all files to gather a list of modules) - notably
when the build system tries to start building
while the dependency graph isn't yet complete.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Tools ( ide, code servers,
indexers, refactoring) may also greatly benefit
from an easier way to locate the source file which
declares a module.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">The specifics of the mapping
are open to bikeshedding. However, I think we
would have better luck sticking to something
simple like <module identifier> <=>
<file name>.<extension><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">(The standardese would mention
<i>resource identifier</i> rather than filename)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">- <b>A standing document
giving guidelines for modules naming.</b><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">The goal is to take everything
the community had to learn the hard way about
header naming over the past 30 years and apply it
to modules by providing a set of guidelines<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">that could be partially
enforced by build system vendors.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Encouraging consistency and
uniqueness of module identifiers across the
industry is I think a necessary step towards sane
package management.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Note that the standard requires
uniqueness of modules identifiers within (the
standard definition of) a program but says little
about a way to ensure this uniqueness.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Here is a rough draft of what I
think would be good guidelines, partially inspired
by what is done by other languages facing similar
issues.<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Prefix module
names with an entity and/or a project name
to prevent modules from different companies,
entities and projects of declaring the same
module names.</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Exported
top-level namespaces should have a name
identic to the project name used as part of
the name of the module(s) from which it is
exported.</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Do not export
multiple top-level namespaces</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Do not export
entities in the global namespace outside of
the global module fragment.</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Organize
modules hierarchically.</span></b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#42464c"> For
example, if both modules </span><code><span style="font-size:10.0pt;color:#42464c;border:none windowtext 1.0pt;padding:0in">example.foo</span></code><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#42464c"> and </span><code><span style="font-size:10.0pt;color:#42464c;border:none windowtext 1.0pt;padding:0in">example.foo.bar</span></code><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#42464c"> exist
as part of the public API of </span><code><span style="font-size:10.0pt;color:#42464c;border:none windowtext 1.0pt;padding:0in">example</span></code><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#42464c">, </span><code><span style="font-size:10.0pt;color:#42464c;border:none windowtext 1.0pt;padding:0in">example.foo</span></code><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#42464c"> should
reexport </span><code><span style="font-size:10.0pt;color:#42464c;border:none windowtext 1.0pt;padding:0in">example.foo.bar</span></code><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Avoid common
names such as </span></b><code><span style="font-size:10.0pt;color:#222324;border:none windowtext 1.0pt;padding:0in">util</span></code><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in"> and </span></b><code><span style="font-size:10.0pt;color:#222324;border:none windowtext 1.0pt;padding:0in">core</span></code><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in"> for module
name prefix and top-level namespace names.</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Use lower-case
module names</span></b><u></u><u></u></p>
<p class="MsoNormal" style="margin-right:24.0pt;margin-bottom:0in;margin-left:24.0pt;margin-bottom:.0001pt;vertical-align:baseline">
<span style="font-size:10.0pt;font-family:Symbol"><span>·<span>
</span></span></span><b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#222324;border:none windowtext 1.0pt;padding:0in">Do not use
characters outside of the basic source
character set in module name identifiers.</span></b><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal">My hope is that these 2
proposals (whose impact on the standard is
minimal) would make it easier for current tooling
to deal with modules<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">while making possible for
example to design dependency managers and build
systems able to work at the module level.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I'd love to gather feedback and
opinions before going further in that direction.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks a lot!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Corentin<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">PS: For a bit of background, I
talked about these issues there<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fcor3ntin.github.io-252Fposts-252Fmodules-5Fmapping-252F-26data-3D02-257C01-257Cgdr-2540microsoft.com-257C1139eb25a2ca43b5cb2e08d6770b6606-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636827288180838903-26sdata-3DRbCelyBe1YDW4eNJtYEgKkAeHGxvkhsYqzPk0wf3F58-253D-26reserved-3D0&d=DwMGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-UCZRX0Vl1g&m=Yv6fjy4yWnfBkW_0m604prnwiQIO5K6DRLBHMjpiaxI&s=v7Z40T9WgivvxWUJ6plSphOw4d8bdvfEz9NAqCruKwE&e=" target="_blank">https://cor3ntin.github.io/posts/modules_mapping/</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fcor3ntin.github.io-252Fposts-252Fmodules-5Fnaming-252F-26data-3D02-257C01-257Cgdr-2540microsoft.com-257C1139eb25a2ca43b5cb2e08d6770b6606-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636827288180838903-26sdata-3DtMhQa4ijeqUd2qxXV4loP47nU5NESRTKJLwZqe-252FI1fc-253D-26reserved-3D0&d=DwMGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-UCZRX0Vl1g&m=Yv6fjy4yWnfBkW_0m604prnwiQIO5K6DRLBHMjpiaxI&s=O9uUoT3QItO0vkb2QTG-EnXjsGfOiq7t93GgFz4YHx8&e=" target="_blank">https://cor3ntin.github.io/posts/modules_naming/</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Tooling mailing list<u></u><u></u></pre>
<pre><a href="mailto:Tooling@isocpp.open-std.org" target="_blank">Tooling@isocpp.open-std.org</a><u></u><u></u></pre>
<pre><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__www.open-2Dstd.org_mailman_listinfo_tooling&d=DwMDaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-UCZRX0Vl1g&m=_dusEGqwzSzglMFFwUFdPvzdZCb1dTUZ9DjjrQwHaUw&s=o4EJMe6pKxUA_1edRMmVmbN3paM7ckt_7iDjgIveiwA&e=" target="_blank">http://www.open-std.org/mailman/listinfo/tooling</a><u></u><u></u></pre>
</blockquote>
<p><u></u> <u></u></p>
</div>
</div>
<br>
<fieldset class="m_-7578053289813035424mimeAttachmentHeader"></fieldset>
<pre class="m_-7578053289813035424moz-quote-pre">_______________________________________________
Tooling mailing list
<a class="m_-7578053289813035424moz-txt-link-abbreviated" href="mailto:Tooling@isocpp.open-std.org" target="_blank">Tooling@isocpp.open-std.org</a>
<a class="m_-7578053289813035424moz-txt-link-freetext" href="http://www.open-std.org/mailman/listinfo/tooling" target="_blank">http://www.open-std.org/mailman/listinfo/tooling</a>
</pre>
</blockquote>
<p><br>
</p>
</div>
_______________________________________________<br>
Tooling mailing list<br>
<a href="mailto:Tooling@isocpp.open-std.org" target="_blank">Tooling@isocpp.open-std.org</a><br>
<a href="http://www.open-std.org/mailman/listinfo/tooling" rel="noreferrer" target="_blank">http://www.open-std.org/mailman/listinfo/tooling</a><br>
</blockquote></div></div>