"Mistakes have been made, as all can see and I admit it."
― Ulysses S. Grant
1. The problem
In C++20 "Extending
to Calendars and Time Zones" ([P0355]) and
"Text Formatting" ([P0645]) proposals were integrated ([P1361]).
Unfortunately during this integration a design issue was missed:
is locale-independent by default and provides control over locale via format
specifiers but the new formatter specializations for chrono types are localized
by default and don’t provide such control.
For example:
std :: locale :: global ( std :: locale ( "ru_RU" )); std :: string s1 = std :: format ( "{}" , 4.2 ); // s1 == "4.2" (not localized) std :: string s2 = std :: format ( "{:L}" , 4.2 ); // s2 == "4,2" (localized) using sec = std :: chrono :: duration < double > ; std :: string s3 = std :: format ( "{:%S}" , sec ( 4.2 )); // s3 == "04,200" (localized)
In addition to being inconsistent with the design of
, there is no
way to avoid locale other than doing formatting of date and time components
manually.
Confusingly, some chrono format specifiers such as
may give an impression
that they are locale-independent by having a locale’s alternative representation
like
while in fact they are not.
The implementation of [P1361] in [FMT] actually did the right thing and made most chrono specifiers locale-independent by default, for example:
using sec = std :: chrono :: duration < double > ; std :: string s = fmt :: format ( "{:%S}" , sec ( 4.2 )); // s == "04.200" (not localized)
This implementation has been available and actively used in this form for 2+ years. The bug in the specification of chrono formatters in the standard and the mismatch with the actual implementation have only been discovered recently and reported in [LWG3547].
2. The solution
We propose fixing this issue by making chrono formatters locale-independent by default
and providing the
specifier to opt into localized formatting in the same way
as it is done for all other standard formatters
(format.string.std).
Before | After |
|
|
|
|
3. Changes from R1
-
Replaced two occurrences of "If the L option is used, the locale is the locale passed to the formatting function, or otherwise the global locale. If the L option is not used, the "C” locale is used." with a paragraph specifying the choice of locale in wording.
-
Added the
option inL
.[ time . zone . zonedtime . nonmembers ]
4. Changes from R0
-
Kept ostream insertion operators (
) locale-dependent for consistency with the rest of the ostream library per LEWG feedback.operator << -
Drive-by fix: made ostream insertion operators of
,utc_time
,tai_time
andgps_time
use the stream’s locale instead of the global locale. They are locale-dependent because the decimal point is localized.file_time -
Drive-by fix: Used the correct locale if the chrono-specs is omitted.
-
Added LEWG poll results.
5. Locale alternative forms
Some specifiers (
) produce digits which are
not localized (aka they use the Arabic numerals
) although as we
demonstrated earlier
is still using a localized decimal separator.
They have an equivalent form (
)
where the numerals can be localized. For example, Japanese numerals 〇 一 二 三 四 五
can be used as the "alternative representation" by a
locale.
But because the
option applies to all specifiers, we do not propose to
modify the specifiers.
For example,
and
should be valid specifiers producing 午後
and 午後一
respectively.
Appropriate use of numeral systems for localized numbers and dates requires more work, this paper focuses on a consistent default behavior.
6. The "C" locale
The "C" locale is used in the wording as a way to express locale-independent
behavior. The C standard specifies the "C" locale behavior for
as
follows
In the "C" locale, the E and O modifiers are ignored and the replacement strings for the following specifiers are:% a the first three characters of % A . % A one of Sunday , Monday , ... , Saturday . % b the first three characters of % B . % B one of January , February , ... , December . % c equivalent to % a % b % e % T % Y . % p one of AM or PM . % r equivalent to % I : % M : % S % p . % x equivalent to % y . % X equivalent to % T . % Z implementation - defined .
This makes it possible, as long as the
option is not specified, to format
dates in environment without locale support (embedded platforms,
if
someone proposes it, etc).
7. SG16 polls
Poll: LWG3547 raises a valid design defect in [time.format] in C++20.
SF | F | N | A | SA |
---|---|---|---|---|
7 | 2 | 2 | 0 | 0 |
Outcome: Strong consensus that this is a design defect.
Poll: The proposed LWG3547 resolution as written should be applied to C++23.
SF | F | N | A | SA |
---|---|---|---|---|
0 | 4 | 2 | 4 | 1 |
Outcome: No consensus for the resolution
SA motivation: Migrating things embedded in a string literal is very difficult. There are options to deal with this in an additive way. Needless break in backwards with compatibility.
SG16 recognized that this is a design defect but was concerned about this being a breaking change. However, the following facts were not known at the time of the discussion:
-
The implementation of [P1361] in [FMT] is locale-independent. This was the only implementation available for 2+ years and was cited as the only source of implementation experience in the paper.
-
Both
and% S
depend on locale and there is no locale-independent equivalent.% OS -
The chrono formatting in the Microsoft’s implementation has only been merged into the main branch on 22 April and has bugs that will require breaking changes.
-
Some chrono types are partially localized, e.g.
may be formatted asmonth_day_last { May }
in a German locale with only month localized.Mai / last
8. LEWG polls
Poll: Revise D2372 to keep the ostream operators for chrono formatting dependent on the stream locale
SF | F | N | A | SA |
---|---|---|---|---|
10 | 8 | 2 | 0 | 0 |
Outcome: Strong Consensus in Favor
Poll: LEWG approves of the direction of this work and encourages further work as directed above with the intent that D2372 (Fixing locale handling in chrono formatters) will land in C++23 and be applied retroactively to C++20
SF | F | N | A | SA |
---|---|---|---|---|
14 | 8 | 0 | 0 | 0 |
Outcome: Unanimous approval
9. Implementation experience
The
specifier has been implemented for durations in the fmt library ([FMT]).
Additionally, some format specifiers like
have never used a locale by default
so this was a novel behavior accidentally introduced in C++20:
std :: locale :: global ( std :: locale ( "ru_RU" )); using sec = std :: chrono :: duration < double > ; std :: string s = fmt :: format ( "{:%S}" , sec ( 4.2 )); // s == "04.200" (not localized)
This proposed fix has also been implemented and submitted to the Microsoft standard library.
10. Impact on existing code
Changing the semantics of chrono formatters to be consistent with standard format specifiers (format.string.std) is a breaking change. At the time of writing the Microsoft’s implementation recenly merged the chrono formatting into the main branch and is known to be not fully conformant. For example:
using sec = std :: chrono :: duration < double > ; std :: string s = std :: format ( "{:%S}" , sec ( 4.2 )); // s == "04" (incorrect)
11. Wording
All wording is relative to the C++ working draft [N4885].
Update the value of the feature-testing macro
to the date of
adoption in [version.syn]:
Change in [time.format]:
A locale used by a formatting function is determined as follows:chrono - format - spec : fill - and - align opt width opt precision opt L opt chrono - specs opt
-
the "C" locale if the
option is not present in chrono-format-spec, otherwiseL - the locale passed to the formatting function if any, otherwise
- the global locale.
2 Each conversion specifier conversion-spec is replaced by appropriate
characters as described in Table [tab:time.format.spec]; the formats specified
in ISO 8601:2004 shall be used where so described. Some of the conversion
specifiers depend on
the locale that is passed to the formatting function
if the latter takes one, or the global locale otherwise.
a locale.
If the formatted object does not contain the information the conversion
specifier refers to, an exception of type
is thrown.
...
6 If the chrono-specs is omitted, the chrono object is formatted as if by
streaming it to
with a locale imbued
and
copying
through the output iterator of the context with additional
padding and adjustments as specified by the format specifiers.
Change in [time.clock.system.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const sys_time < Duration >& tp );
...
2 Effects: Equivalent to:
auto const dp = floor ( tp ); return os << format ( os . getloc (), STATICALLY - WIDEN ( "{} {}" "{:L} {:L}" ), year_month_day { dp }, hh_mm_ss { tp - dp });
Change in [time.clock.utc.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const utc_time < Duration >& t );
1 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN ( "{:%F %T}" "{:L%F %T}" ), t );
(Adding
is a drive-by fix.)
Change in [time.clock.tai.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const tai_time < Duration >& t );
1 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN ( "{:%F %T}" "{:L%F %T}" ), t );
Change in [time.clock.gps.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const gps_time < Duration >& t );
1 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN ( "{:%F %T}" "{:L%F %T}" ), t );
Change in [time.clock.file.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const file_time < Duration >& t );
1 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN ( "{:%F %T}" "{:L%F %T}" ), t );
[time.cal.day.nonmembers] is intentionally left unchanged because
is
locale-independent.
Change in [time.cal.month.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const month & m );
7 Effects: Equivalent to:
return os << ( m . ok () ? format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{:%b}" "{:L%b}" ), m ) : format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{} is not a valid month" ), static_cast < unsigned > ( m )));
[time.cal.year.nonmembers] is intentionally left unchanged because
is
locale-independent.
Change in [time.cal.wd.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const weekday & wd );
6 Effects: Equivalent to:
return os << ( wd . ok () ? format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{:%a}" "{:L%a}" ), wd ) : format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{} is not a valid weekday" ), static_cast < unsigned > ( wd . wd_ )));
Change in [time.cal.wdidx.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const weekday_indexed & wdi );
2 Effects: Equivalent to:
auto i = wdi . index (); return os << ( i >= 1 && i <= 5 ? format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}[{}]" "{:L}[{}]" ), wdi . weekday (), i ) : format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{ :L }[{} is not a valid index]" ), wdi . weekday (), i ));
Change in [time.cal.wdlast.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const weekday_last & wdl );
2 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{ :L }[last]" ), wdl . weekday ());
Change in [time.cal.md.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const month_day & md );
3 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}" "{:L}/{:L}" ), md . month (), md . day ());
Change in [time.cal.mdlast]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const month_day_last & mdl );
9 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/last" "{:L}/last" ), mdl . month ());
Change in [time.cal.mwd.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const month_weekday & mwd );
2 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}" "{:L}/{:L}" ), mwd . month (), mwd . weekday_indexed ());
Change in [time.cal.mwdlast.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const month_weekday_last & mwdl );
2 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}" "{:L}/{:L}" ), mwdl . month (), mwdl . weekday_last ());
Change in [time.cal.ym.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const year_month & ym );
14 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}" "{:L}/{:L}" ), ym . year (), ym . month ());
[time.cal.ymd.nonmembers] is intentionally left unchanged because
is
locale-independent.
Change in [time.cal.ymdlast.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const year_month_day_last & ymdl );
12 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}" "{:L}/{:L}" ), ymdl . year (), ymdl . month_day_last ());
Change in [time.cal.ymwd.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const year_month_weekday & ymwd );
11 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}/{}" "{:L}/{:L}/{:L}" ), ymwd . year (), ymwd . month (), ymwd . weekday_indexed ());
Change in [time.cal.ymwdlast.nonmembers]:
template < class charT , class traits > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const year_month_weekday_last & ymwdl );
11 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{}/{}/{}" "{:L}/{:L}/{:L}" ), ymwdl . year (), ymwdl . month (), ymwdl . weekday_last ());
Change in [time.hms.nonmembers]:
template < class charT , class traits , class Duration > basic_ostream < charT , traits >& operator << ( basic_ostream < charT , traits >& os , const hh_mm_ss < Duration >& hms );
1 Effects: Equivalent to:
return os << format ( os . getloc (), STATICALLY - WIDEN < charT > ( "{:%T}" "{:L%T}" ), hms );
Change in [time.zone.zonedtime.nonmembers]:
2 Effects: Streams the value returned from
to
using
the format
.