First things first: this paper is a process description, and
the main audience is Working Group and Study Group chairs,
their assistants, and WG21 expert members who do proposal
reviews. This paper is not a tutorial for writing good
proposals - although keen-eyed readers may find the
description useful for producing high-quality proposals.
This paper explains the rules of engagement that have been
in use between Evolution and Core, amends the rules with regards
to Evolutionary specification review, and proposes adopting the same rules
between Library Evolution and Library.
The amendment to the rules should change nothing for Core and Library
(well, not for worse, at least), it's merely a specification-runthrough
in EWG and LEWG before the material proceeds for technical/specification
review. But as a formal addition, there
is a step after the specification review to review available
implementation/deployment experience; this review is to be done by both
the Design and the Specification group. Whether the groups want
to combine forces for that review or perform it separately
is left up to their discretion, but both groups should perform
that review, because they will be doing so from different viewpoints
with a different emphasis.
The proposal ALSO proposes a new Tentatively Ready state between
specification review and plenary poll. This is proposed in order to give
reviewers time to digest new proposals. Said state can be bypassed
in urgent situations, but by default, it's proposed in order to
allow for gathering more implementation/deployment experience,
experiment feedback, and other feedback.
To repeat and rephrase, this proposal attempts to
improve the following abilities:
the ability of CWG/LWG to even start processing an incoming paper
the ability of CWG/LWG to process an incoming paper without unnecessary overhead and waste of time
the ability to give post-EWG/LEWG-discussion feedback on proposals
the ability to review proposals after CWG/LWG review but before
the plenary
the ability to verify that a change in the standard is implementable
the ability to verify that a change in the standard is deployable
The <=> in the title is not a spaceship, it's two overlapping
arrows going into opposite directions.
Revision history
Highlight the most recent changes:
Highlight the R1->R2 changes:
Highlight the R0->R1 changes:
R3: This version, clarifications
for intended audience and how to solicit assistance in
writing wording. Also clarify the conditions for bypassing
Tentatively Ready states, and clarify that Tentatively Ready
for Plenary means the matter will be in a plenary straw poll
in the next meeting.
R2: New additions for
a pre-plenary Tentatively Ready stage both in the diagram and in the description,
the post-design Tentatively Ready status is now in the diagram, and in the description
Also changed from "wording" to "specification".
R1: Added clarifications for
what the qualifications of a wordsmith are,
how to find wordsmiths, and
how a WG can choose to do a small-group specification pre-review instead of relying on a single individual wordsmith to do it
R0: Initial version
Further elaboration on the problems this proposal aims to tackle
Here's an alternative way to put it, in three parts:
Avoid wasting time in the various stage of the pipeline. Especially avoid wasting time in the stages that are our bottlenecks, aka the specification review.
Define what it requires to advance from one pipeline stage
to the next.
Define checkpoints for going back from one pipeline stage to
an earlier one, if necessary.
Here's why we propose to introduce new checkpoints for possibly
going back to earlier pipeline stages, and the problem we're trying
to tackle by doing so, as phrased by Peter Dimov:
Roughly and approximately, the problem that the committee can potentially
inflict billions of dollars of damage on the C++ community by breaking code
in a sufficiently problematic way. This requires a certain amount of due
diligence.
Some remarks of the alleged downsides of this proposal
The following concerns might be raised about this proposal:
it introduces very stringent gatekeeping
it makes the process harder for new committee members
For the first, it should be pointed out that this proposal doesn't
actually introduce anything new; we have done all of the things
in it in an inconsistent fashion, sometimes doing it, sometimes not.
This proposal changes what our defaults are, and allows bypassing
those defaults if we make an Educated Decision to do so.
For the second, this proposal makes our pipeline stages officially-specified
for everybody, both old and new committee members. And it gives
new and old committee members a process specification that they can read,
as opposed to relying on hearsay and never really knowing what to
expect.
Finally, while stringent gatekeeping might seem like a nuisance
to some, and a multi-stage process that requires multiple championing
steps might be inconvenient for proposal authors who can't attend
multiple meetings, all that must be secondary to the ultimate
goal of all of this, which is a higher-quality International
Standard. This is simply not negotiable.
Some historical background
The original problem
Way back when, not really important when, but a couple of years
ago, we had three problematic issues with material that flowed
from Evolution to Core:
some material didn't have specification when it arrived on Core's
plate.
some material had specification that hadn't been pre-reviewed by
a Core expert (in some cases this was because the specification
had been written by people who over-appreciated their level
of specification expertise).
some material needed to be sent back to Evolution, and
Evolution wasn't always prepared nor welcoming of such an
event of material being sent back.
similar issues occurred between LWG and LEWG, for all of those
cases.
The solution
To alleviate these problems, the Chair of Evolution specified
the following rules, although they weren't written down (a snag
that this document aims to rectify):
a proposal forwarded to Core must have wording; this is not
negotiable, and Core shouldn't and doesn't entertain suggestions
for fishing expeditions, they are busy.
such wording must be either written by a recognized wordsmith
or reviewed by one.
if, for any reason, Core thinks they need a design clarification
or have any other question or a challenge about the design,
the rule is to send the material back to Evolution accompanied
by a member of Core that can explain the problem.
Notably, regardless of whether it's
a "please clarify", or
a "please reconsider", or
a "we think this stinks",
the rule is the same - send it back and worry about
possible hurt egos later. Core is not a design group, but they
have enough experience that they are allowed and invited
to challenge designs that make no sense to them. And in any
case, Core shouldn't try to resolve design issues in Core,
that's Evolution's job and Evolutions responsibility, and at the
very least Evolution should understand the Core-technical
consequences of their decisions.
The rationale for these points shouldn't be hard to understand:
Core (or Library) is supposed to review a technical specification and make
sure it's worded suitably, accurately, and consistently, and
wording-wise and otherwise fits into the language. Doing so
requires having the actual wording that is proposed to be incorporated
into the Working Draft.
Core (or Library) doesn't write wording.
Some members of Core (or Library) might be so kind to help write wording.
Core's (or Library's) workload is hefty, and the time of the experts in that
room is not used well if they are looking at a specification
that is nowhere near ready to have a chance to survive the first
5 minutes of review before there is a request to go away and
come back with proper wording. This is both about consistency
and completeness - the wording should be roughly similar to
the wording we already have terminology-wise, style-wise and
otherwise, and it must have all the things that are part of the
design.
Core (or Library) doesn't write wording.
Some members of Core (or Library) might be so kind to help write wording.
The process
Once these issues were recognized and the alleviation for them
agreed on, we established the following process for Design->Specification:
Evolution reviews a proposal, and decides they are happy with
it and want to forward it to Core.
The EWGChair queries/verifies the existence of the wording.
In case it doesn't exist, the instructions for the paper
author are "find a wordsmith, produce wording, and report
back to me".
In case it exists, the question is "has it been reviewed
by a reputable wordsmith? If not, have it so reviewed
and report back to me."
Once EWGChair is satisfied with the reputable wordsmith's
report that the wording is ready for Core consumption, Core
is made aware of incoming material and the proposal author
is greenlighted to proceed to Core review at Core's scheduling
pleasure.
And, as mentioned earlier, the following process for Design<-Specification:
Core reviews a proposal, and discovers a design issue.
The CWGChair notifies EWGChair that something needs to come
back, and designates a Core champion to explain the issue in Evolution.
EWGChair schedules such a discussion with elevated priority,
and we iterate again, with the Design<->Specification rules as before.
The New Rules
These rules apply to both EWG->CWG and LEWG->LWG. Furthermore, there
is now a new Tentatively Ready step between specification review and
a plenary poll.
Rather than the chair of a design group making sure that wording
exists and is ready for the technical/specification review group's consumption,
that responsibility now lies with the whole design group. That is,
once the proposal author has made sure their wording is either written
by a wordsmith or reviewed by one, they approach the design group
(with the wordsmith) and give a presentation about the wording, illustrating
how it implements the design, and report whether any oddities or questions
surfaced.
If such oddities or questions did surface, the design group needs
to discuss them and possibly clarify their design.
This rule change is made for a couple of reasons:
the design groups must understand the technical consequences
of their designs
the design groups must double-check that the design is accurately
implemented by the specification, instead of saying very late in the process
that "that's not what we meant" or "that's against the design intent".
"What's a 'wordsmith'? Who qualifies?"
A wordsmith is, by the definition for the purposes of this process,
"a person who can write or help write CWG/LWG-consumable wording".
It's ultimately the
decision of the chairs of EWG/LEWG to decide
what sort of wording they're willing to accept as wording incoming
into their specification review (which might be less
strict than the specification-group review), and those groups should have confidence
that what they forward is written and championed
by a person that won't be wasting Core's or Library's time.
CWG and LWG can also, if they so choose, arrange for a small group
of their members to do this specification-work. Note, however, that such arranging
must not consume group time, and is not the responsibility of the chair
of the specification review group.
The solicitation for wording assistance is highly recommended
to have roughly the following priority order:
The members of EWG/LEWG should be queried for volunteers to help with
wording when the initial design review is done.
Well-known wordsmiths can be asked for help by more seasoned
EWG/LEWG members, which is a slight variation of the above.
Wording assistance requests can be made on the reflector.
It is not appropriate to spend CWG's or LWG's time in the quest for a
wordsmith, as a group.
Further explanation of the new process
So, to sum it up, let's bulletize the Design->Specification process:
The design group, be it LEWG or EWG, decides after reviewing
a proposal that it should go forward to LWG or CWG.
In all the subsequent steps, both the wordsmith
and the proposal author/champion should be available and present.
That includes both the Design group's wording review and the Specification
group's wording review.
If the proposal has wording, the design group must run through
it and verify that the design is wholly and accurately implemented
by the wording, and that there are neither omissions nor additions
that the design doesn't cover.
If the proposal does not have wording, the design group must
not greenlight the proposal to go forward; they must instead
instruct the proposal to come back with the wording.
When the proposal has new wording not yet reviewed in the second
bullet, go back to the third bullet.
Repeat as many times as necessary.
Once the run-through is complete, and the design group is confident
that the wording implements the design wholly and accurately, the
design group sends the material onwards, forwards.
By default, that means
first parking it for further design feedback,
possibly reiterating the design based on such feedback, if any,
and then making it Tentatively Ready
Let's also bulletize the Design<-Specification process, and also the
steps between Specification->Plenary:
The specification/technical review group reviews a proposal and discovers
a design-level addition/omission/bug/issue/snag, or a technical
problem (an implementability problem, a consistency problem, or
some other problem).
The chair of the specification/technical review group makes sure
the problem is minuted, sends a heads-up to the chair of the design
group, and designates a champion that can explain the problem to
the design group.
The design group reviews the matter, and resolves it according
to the Design->Specification process.
The specification/technical review group continues progress on the proposal.
Once the specification review is complete, both the Specification group and the Design group look at the available implementation/deployment experience
of the proposal.
If that experience is deemed sufficient, the proposal
is made Tentatively Ready for a plenary poll.
If there's feedback that necessitates looking
again at either the design or the specification, the proposal is sent
back to those stages.
Otherwise, proceed to the plenary poll
*in the next meeting*.
Bypassing the Tentatively Ready status after design review or the
Tentatively Ready for Plenary status
after specification review is allowed by the process, but
such
exceptional bypasses shall happen only after an explicit decision made
by the design group (in the case of bypassing Tentatively Ready) or by
both the design group
and the specification group (in the case of bypassing Tentatively
Ready for Plenary)
such decisions
shall be made after a minuted discussion.