Text Formatting

Revision History

Changes since R2

Add section Argument visitation with an example of how to use the visitation API to implement dynamic format specifiers.
Rename visit to visit_format_arg to distinguish from variant's visit.
Merge [format.syntax] into [format.functions].
Remove the restriction that '\0' cannot be used as a fill character.
Replace Postconditions with Ensures.

Changes since R1

Add the format_to_n function taking an output iterator and a size.
Add a note that compile-time processing of format strings applies to user-defined types with custom parsers to section Compile-time processing of format strings.
Rename count to formatted_size.
Drop nested namespace fmt.
Add format prefix or infix to class and function names to avoid potential name collision after removing the nested namespace.
Improve wording.
Replace the requirement of implementing a formatter via ostream insertion operator if the latter is provided with a note that it can be implemented in such way.
Expand the Acknowledgements section and remove "Reply to".

Changes since R0

Add section Compile-time processing of format strings.
Separate parsing and formatting in the extension API replacing format_value function template with class template formatter to allow compile-time processing of format strings.
Change return type of format_to and vformat_to to OutputIterator in synopsis.
Remove sections Null-terminated string view and Format string, and replace basic_cstring_view with basic_string_view.
Add a link to the implementation in Introduction.
Add a note regarding time formatting and compatibility with D0355 "Extending <chrono> to Calendars and Time Zones" [16] to section Extensibility.
Rename basic_args to basic_format_args.
Rename is_numeric to is_arithmetic.
Add the count function that counts the number of characters and use it to define output ranges.
Remove basic_buffer and section Formatting buffer and replace buffers with output iterators.
Add Appendix A: Benchmarks.
Explain the purpose of the type-erased API in more details in the Binary footprint section.
Add Appendix B: Binary code comparison.
Add formatting function overloads for wchar_t strings.

Even with proliferation of graphical and voice user interfaces, text remains one of the main ways for humans to interact with computer programs and programming languages provide a variety of methods to perform text formatting. The first thing we do when learning a new programming language is often write a "Hello, World!" program that performs simple formatted output.

C++ has not one but two standard APIs for producing formatted output, the printf family of functions inherited from C and the I/O streams library (iostreams). While iostreams are usually the recommended way of producing formatted output in C++ for safety and extensibility reasons, printf offers some advantages, such as an arguably more natural function call API, the separation of formatted message and arguments, possibly with argument reordering as a POSIX extension, and often more compact source and binary code.

This paper proposes a new text formatting library that can be used as a safe and extensible alternative to the printf family of functions. It is intended to complement the existing C++ I/O streams library and reuse some of its infrastructure such as overloaded insertion operators for user-defined types.

Example:

string message = format("The answer is {}.", 42);

A full implementation of this proposal is available at https://github.com/fmtlib/fmt/tree/std.

Design

Format string syntax

Variations of the printf format string syntax are arguably the most popular among the programming languages and C++ itself inherits printf from C [1]. The advantage of the printf syntax is that many programmers are familiar with it. However, in its current form it has a number of issues:

Many format specifiers like hh, h, l, j, etc. are used only to convey type information. They are redundant in type-safe formatting and would unnecessarily complicate specification and parsing.
There is no standard way to extend the syntax for user-defined types.
There are subtle differences between different implementations. For example, POSIX positional arguments [2] are not supported on some systems [6].
Using '%' in a custom format specifier poses difficulties, e.g. for put_time-like time formatting.

Although it is possible to address these issues while maintaining resemblance to the original printf format, this will still break compatibility and can potentially be more confusing to users than introducing a different syntax.

Therefore we propose a new syntax based on the ones used in Python [3], the .NET family of languages [4], and Rust [5]. This syntax employs '{' and '}' as replacement field delimiters instead of '%' and it is described in detail in [format.functions]. Some advantages of the proposal are:

A consistent and easy to parse mini-language focused on formatting rather than conveying type information
Extensibility and support for custom format strings for user-defined types
Positional arguments
Support for both locale-specific and locale-independent formatting (see Locale support)
Formatting improvements such as better alignment control, fill character, and binary representation

The syntax is expressive enough to enable translation, possibly automated, of most printf format strings. The correspondence between printf and the new syntax is given in the following table:

printf	new
-	<
+	+
space	space
#	#
0	0
hh	unused
h	unused
l	unused
ll	unused
j	unused
z	unused
t	unused
L	unused
c	c (optional)
s	s (optional)
d	d (optional)
i	d (optional)
o	o
x	x
X	X
u	d (optional)
f	f
F	F
e	e
E	E
a	a
A	A
g	g (optional)
G	G
n	unused
p	p (optional)

Width and precision are represented similarly in printf and the proposed syntax with the only difference that runtime value is specified by '*' in the former and '{}' in the latter, possibly with the index of the argument inside the braces:

printf("%*s", 10, "foo");
format("{:{}}", "foo", 10);

As can be seen from the table above, most of the specifiers remain the same which simplifies migration from printf. A notable difference is in the alignment specification. The proposed syntax allows left, center, and right alignment represented by '<', '^', and '>' respectively which is more expressive than the corresponding printf syntax. The latter only supports left and right alignment.

The following example uses center alignment and '*' as a fill character:

format("{:*^30}", "centered");

resulting in "***********centered***********". The same formatting cannot be easily achieved with printf.

In addition to positional arguments, the grammar can be easily extended to support named arguments.

Extensibility

Both the format string syntax and the API are designed with extensibility in mind. The mini-language can be extended for user-defined types and users can provide functions that implement parsing, possibly at compile time, and formatting for such types.

The general syntax of a replacement field in a format string is

replacement-field ::= '{' [arg-id] [':' format-spec] '}'

where format-spec is predefined for built-in types, but can be customized for user-defined types. For example, the syntax can be extended for put_time-like date and time formatting

time_t t = time(nullptr);
string date = format("The date is {0:%Y-%m-%d}.", *localtime(&t));

by providing a specialization of formatter for tm:

template<>
  struct formatter<tm> {
    constexpr parse_context::iterator parse(parse_context& ctx);

    template<class FormatContext>
      typename FormatContext::iterator format(const tm& tm, FormatContext& ctx);
  };

The formatter<tm>::parse function parses the format-spec portion of the format string corresponding to the current argument and formatter<tm>::format formats the value and writes the output via the iterator ctx.begin().

Note that date and time formatting is not covered by this proposal but formatting facilities provided by D0355 "Extending <chrono> to Calendars and Time Zones" [16] can be easily implemented using this extension API.

An implementation of formatter<T>::format can use ostream insertion operator<< for user-defined type T if available.

The extension API is based on specialization instead of the argument-dependent lookup, because the parse function doesn't take the object to be formatted as an argument and therefore some other way of parameterizing it on the argument type T such as introducing a dummy argument has to be used, e.g.

constexpr auto parse(type<T>, parse_context& ctx);

Safety

Formatting functions rely on variadic templates instead of the mechanism provided by <cstdarg>. The type information is captured automatically and passed to formatters guaranteeing type safety and making many of the printf specifiers redundant (see Format String Syntax). Memory management is automatic to prevent buffer overflow errors common to printf.

Locale support

As pointed out in P0067 "Elementary string conversions"[17] there is a number of use cases that do not require internationalization support, but do require high throughput when produced by a server. These include various text-based interchange formats such as JSON or XML. The need for locale-independent functions for conversions between integers and strings and between floating-point numbers and strings has also been highlighted in N4412: Shortcomings of iostreams. Therefore a user should be able to easily control whether to use locales or not during formatting.

We follow Python's approach [3] and designate a separate format specifier 'n' for locale-aware numeric formatting. It applies to all integral and floating-point types. All other specifiers produce output unaffected by locale settings. This can also have positive effect on performance because locale-independent formatting can be implemented more efficiently.

Positional arguments

An important feature for localization is the ability to rearrange formatting arguments as the word order may vary in different languages [7]. For example:

printf("String `%s' has %d characters\n", string, length(string)));

A possible German translation of the format string might be:

"%2$d Zeichen lang ist die Zeichenkette `%1$s'\n"

using POSIX positional arguments [2]. Unfortunately these positional specifiers are not portable [6]. The C++ I/O streams don't support such rearranging of arguments by design because they are interleaved with the portions of the literal string:

cout << "String `" << string << "' has " << length(string) << " characters\n";

The current proposal allows both positional and automatically numbered arguments, for example:

format("String `{}' has {} characters\n", string, length(string)));

with the German translation of the format string being:

"{1} Zeichen lang ist die Zeichenkette `{0}'\n"

Performance

The formatting library has been designed with performance in mind. It tries to minimize the number of virtual function calls and dynamic memory allocations done per formatting operation. In particular, if formatting output can fit into a fixed-size array allocated on stack, it should be possible to avoid both of them altogether by using a suitable API.

The format_to function takes an arbitrary output iterator and, for performance reasons, can be specialized for random-access and contiguous iterators as shown in the reference implementation [14].

The locale-independent formatting can also be implemented more efficiently than the locale-aware one. However, the main goal for the former is to support specific use cases (see Locale support) rather than to improve performance.

See Appendix A: Benchmarks for a small performance comparison of the reference implementation of this proposal versus the standard formatting facilities.

Binary footprint

In order to minimize binary code size each formatting function that uses variadic templates can be implemented as a small inline wrapper around its non-variadic counterpart. This wrapper creates a basic_format_args object, representing an array of type-erased argument references, with make_format_args and calls the non-variadic function to do the actual work. For example, the format variadic function calls vformat:

string vformat(string_view fmt, format_args args);

template<class... Args>
  inline string format(string_view fmt, const Args&... args) {
    return vformat(fmt, make_format_args(args...));
  }

basic_format_args can be implemented as an array of tagged unions. If the number of arguments is small then the tags that indicate the arguments types can be combined and passed into a formatting function as a single integer. This single integer representing all argument types is computed at compile time and only needs to be stored, resulting in smaller binary code.

Given a reasonable optimizing compiler, this will result in a compact per-call binary code, effectively consisting of placing argument pointers (or, possibly, copies for primitive types) and packed tags on the stack and calling a formatting function. See Appendix B: Binary code comparison for a specific example.

Exposing the type-erased API rather than making it an implementation detail and only providing variadic functions allows applying the same technique to the user code. For example, consider a simple logging function that writes a formatted error message to clog:

template<class... Args>
  void log_error(int code, string_view fmt, const Args&... args) {
    clog << "Error " << code << ": " << format(fmt, args...);
  }

The code for this function will be generated for every combination of argument types which can be undesirable. However, if we use the type-erased API, there will be only one instance of the logging function while the wrapper function can be trivially inlined:

void vlog_error(int code, string_view fmt, format_args args) {
  clog << "Error " << code << ": " << vformat(fmt, args);
}

template<class... Args>
  inline void log_error(int code, string_view fmt, const Args&... args) {
    vlog_error(code, fmt, make_format_args(args...));
  }

The current design allows users to easily switch between the two approaches.

Compile-time processing of format strings

It is possible to parse format strings at compile time with constexpr functions which has been demonstrated in the reference implementation [14] and in [18]. Unfortunately a satisfactory API cannot be provided using existing C++17 features. Ideally we would like to use a function call API similar to the one proposed in this paper:

template<class String, class... Args>
  string format(String fmt, const Args&... args);

where String is some type representing a compile-time format string, for example

struct MyString {
  static constexpr string_view value() { return string_view("{}", 2); }
};

However, requiring a user to create a new type either manually or via a macro for every format string is not acceptable. P0424R1 "Reconsidering literal operator templates for strings" [15] provides a solution to this problem based on user-defined literal operators. If this or other similar proposal for compile-time strings is accepted into the standard, it should be easy to provide additional formatting APIs that make use of this feature. Obviously runtime checks will still be needed in cases where the format string is not known at compile time, but as shown in Appendix A: Benchmarks even with runtime parsing performance can be on par with or better than that of existing methods.

Compile-time processing of format strings can work with a user-defined type T if the formatter<T>::parse function is constexpr.

Argument visitation

The argument visitation API can be used to implement dynamic format specifiers, for example, width and precision passed as additional formatting arguments as opposed to being encoded in the format string itself:

int width = 10;
int precision = 3;
auto s = format("{0:{1}.{2}f}", 12.345678, width, precision);
// s == "    12.346"

An example of a user-defined formatter with dynamic width:

struct Answer {};

template<>
  struct formatter<Answer> {
    int width_arg_index = 0;

    // Parses dynamic width in the format "{<digit>}".
    auto parse(parse_context& parse_ctx) {
      auto iter = parse_ctx.begin();
      auto get_char = [&]() { return iter != parse_ctx.end() ? *iter : 0; };

      if (get_char() != '{')
        return iter;
      ++iter;
      char c = get_char();
      if (!std::isdigit(c) || (++iter, get_char()) != '}')
        throw format_error("invalid format");
      width_arg_index = c - '0';
      return ++iter;
    }

    auto format(Answer, format_context& format_ctx) {
      auto arg = format_ctx.args().get(width_arg_index);
      int width = visit_format_arg([](auto value) -> int {
        if constexpr (!std::is_integral_v<decltype(value)>)
          throw format_error("width is not integral");
        else if (value < 0 || value > std::numeric_limits<int>::max())
          throw format_error("invalid width");
        else
          return value;
        }, arg);
      return format_to(format_ctx.out(), "{:{}}", 42, width);
    }
  };

std::string s = format("{0:{1}}", Answer(), 10);
// s == "        42"

In many cases formatting can be delegated to standard formatters which makes manual handling of dynamic specifiers unneccessary, but the latter is still important for more complex cases.

This API can also be used to implement a custom formatting engine, such as the one compatible with the printf syntax, to provide some of the benefits of the current proposal to the legacy code.

Impact on existing code

The proposed formatting API is defined in the new header <format> and should have no impact on existing code.

Proposed wording

Add a new section in 19 [utilities].

Formatting utilities [format]

Header `<format>` synopsis [format.syn]

namespace std {
  // [format.error], class format_error
  class format_error;

  // [format.formatter], formatter
  template<class charT> class basic_parse_context;
  using parse_context = basic_parse_context<char>;
  using wparse_context = basic_parse_context<wchar_t>;
  
  template<class OutputIterator, class charT> class basic_format_context;
  using format_context = basic_format_context<unspecified, char>;
  using wformat_context = basic_format_context<unspecified, wchar_t>;

  template<class T, class charT = char> struct formatter;
  
  // [format.arguments], arguments
  template<class Context> class basic_format_arg;

  template<class Visitor, class Context>
    see below visit_format_arg(Visitor&& vis, basic_format_arg<Context> arg);

  template<class Context, class... Args> using format_arg_store = unspecified;

  template<class Context> class basic_format_args;
  using format_args = basic_format_args<format_context>;
  using wformat_args = basic_format_args<wformat_context>;

  template<class OutputIterator, class charT>
    using format_args_t = basic_format_args<basic_format_context<OutputIterator, charT>>;

  template<class Context, class... Args>
    format_arg_store<Context, Args...> make_format_args(const Args&... args);
  template<class... Args>
    format_arg_store<format_context, Args...> make_format_args(const Args&... args);
  template<class... Args>
    format_arg_store<wformat_context, Args...> make_wformat_args(const Args&... args);

  // [format.functions], formatting functions
  template<class... Args>
    string format(string_view fmt, const Args&... args);
  template<class... Args>
    wstring format(wstring_view fmt, const Args&... args);

  string vformat(string_view fmt, format_args args);
  wstring vformat(wstring_view fmt, wformat_args args);

  template<class OutputIterator, class... Args>
    OutputIterator format_to(OutputIterator out, string_view fmt,
                             const Args&... args);
  template<class OutputIterator, class... Args>
    OutputIterator format_to(OutputIterator out, wstring_view fmt,
                             const Args&... args);

  template<class OutputIterator>
    OutputIterator vformat_to(OutputIterator out, string_view fmt,
                              format_args_t<OutputIterator, char> args);
  template<class OutputIterator>
    OutputIterator vformat_to(OutputIterator out, wstring_view fmt,
                              format_args_t<OutputIterator, wchar_t> args);

  template<class OutputIterator, class Size>
    struct format_to_n_result {
      OutputIterator out;
      Size size;
    };
  
  template<class OutputIterator, class Size, class... Args>
    format_to_n_result<OutputIterator, Size>
      format_to_n(OutputIterator out, Size n, string_view fmt,
                  const Args&... args);
  template<class OutputIterator, class Size, class... Args>
    format_to_n_result<OutputIterator, Size>
      format_to_n(OutputIterator out, Size n, wstring_view fmt,
                  const Args&... args);

  template<class... Args>
    size_t formatted_size(string_view fmt, const Args&... args);
  template<class... Args>
    size_t formatted_size(wstring_view fmt, const Args&... args);
}

Class `format_error` [format.error]

namespace std {
  class format_error : public runtime_error {
  public:
    explicit format_error(const string& what_arg);
    explicit format_error(const char* what_arg);
  };
}

The class format_error defines the type of objects thrown as exceptions to report errors from the formatting library.

format_error(const string& what_arg);

Effects: Constructs an object of class format_error.

Ensures: strcmp(what(), what_arg.c_str()) == 0.

format_error(const char* what_arg);

Effects: Constructs an object of class format_error.

Ensures: strcmp(what(), what_arg) == 0.

Arguments [format.arguments]

Class template `basic_format_arg`

namespace std {
  template<class Context>
  class basic_format_arg {
  public:
    class handle;

    basic_format_arg();

    explicit operator bool() const noexcept;

    bool is_arithmetic() const;
    bool is_integral() const;
  };
}

The class template basic_format_arg describes an object that can refer to a formatting function argument. It is parameterized on a formatting context type (see [format.formatter]) and can hold a value of one of the following types:

bool
charT
any integral type other than bool and charT
any floating-point type
const charT*
basic_string_view<charT>
const void*
basic_format_arg::handle referring to an argument of a user-defined type T with an enabled specialization of formatter<T>
monostate representing an empty state, i.e. when the object does not refer to an argument

where charT is typename Context::char_type. The value can be accessed via the visitation interface defined in the next section. basic_format_arg must preserve argument values, but does not have to distinguish between integral types other than charT and bool, e.g. an argument of type short can be stored as int and observed as int via the visitation interface. Similarly, basic_format_arg does not have to distinguish between floating-point types.

basic_format_arg();

Effects: Constructs a basic_format_arg object that doesn't refer to an argument.

Ensures: !(*this).

explicit operator bool() const noexcept;

Returns: true if and only if *this refers to an argument.

bool is_arithmetic() const;

Returns: true if and only if *this refers to an argument of an arithmetic type.

bool is_integral() const;

Returns: true if and only if *this refers to an argument of an integral type.

namespace std {
  template<class Context>
  class basic_format_arg<Context>::handle {
  public:
    void format(Context& ctx) const;
  };
}

The class handle describes an object that can refer to a formatting function argument of a user-defined type.

void format(Context& ctx) const;: Effects: Constructs a local variable of type typename Context::template formatter_type<T> (named f for exposition purposes), where T is the type of the object obj referred to by this handle. Executes ctx.parse_context().advance_to(f.parse(ctx.parse_context())). Then calls ctx.advance_to(f.format(obj, ctx)).

Option	Meaning
`'<'`	Forces the field to be left-aligned within the available space. This is the default for non-arithmetic types.
`'>'`	Forces the field to be right-aligned within the available space. This is the default for arithmetic types.
`'='`	Forces the padding to be placed after the sign (if any) but before the digits. This is used for printing fields in the form `+000000120`. This alignment option is only valid for arithmetic types.
`'^'`	Forces the field to be centered within the available space.

Option	Meaning
`'+'`	Indicates that a sign should be used for both positive as well as negative numbers.
`'-'`	Indicates that a sign should be used only for negative numbers (this is the default behavior).
space	Indicates that a leading space should be used for positive numbers, and a minus sign for negative numbers.

Type	Meaning
`'s'`	String format. This is the default type for strings and may be omitted.
none	The same as `'s'`.

Type	Meaning
`'c'`	Character format. This is the default type for characters and may be omitted.
none	The same as `'c'`.

Type	Meaning
`'b'`	Binary format. Outputs the number in base 2. Using the `'#'` option with this type adds the prefix `"0b"` to the output value.
`'B'`	Binary format. Outputs the number in base 2. Using the `'#'` option with this type adds the prefix `"0B"` to the output value.
`'d'`	Decimal integer. Outputs the number in base 10.
`'o'`	Octal format. Outputs the number in base 8. Using the '#' option with this type adds the prefix "0" to the output value.
`'x'`	Hex format. Outputs the number in base 16, using lower-case letters for the digits above 9. Using the `'#'` option with this type adds the prefix `"0x"` to the output value.
`'X'`	Hex format. Outputs the number in base 16, using upper-case letters for the digits above 9. Using the `'#'` option with this type adds the prefix `"0X"` to the output value.
`'n'`	Number. This is the same as `'d'` except that it uses the current locale to insert the appropriate number separator characters.
none	The same as `'d'`.

Text Formatting

Revision History

Changes since R2

Changes since R1

Changes since R0

Introduction

Design

Format string syntax

Extensibility

Safety

Locale support

Positional arguments

Performance

Binary footprint

Compile-time processing of format strings

Argument visitation

Impact on existing code

Proposed wording

Formatting utilities [format]

Header `<format>` synopsis [format.syn]

Class `format_error` [format.error]

Arguments [format.arguments]

Class template `basic_format_arg`

Argument visitation

Class template `basic_format_args`

Function template `make_format_args`

Function template `make_wformat_args`

Formatting functions [format.functions]

Formatter [format.formatter]

Class template `basic_parse_context`

Class template `basic_format_context`

Class template `formatter`

Formatter requirements

Related work

Implementation

Acknowledgements

References

Appendix A: Benchmarks

Appendix B: Binary code comparison

Type	Meaning
`'a'`	Hexadecimal floating point format. Prints the number in base 16 with prefix `"0x"` and lower-case letters for digits above 9. Uses `'p'` to indicate the exponent.
`'A'`	Same as `'a'` except it uses upper-case letters for the prefix, digits above 9 and to indicate the exponent.
`'e'`	Exponent notation. Prints the number in scientific notation using the letter `'e'` to indicate the exponent.
`'E'`	Exponent notation. Same as `'e'` except it uses an upper-case `'E'` as the separator character.
`'f'`	Fixed point. Displays the number as a fixed-point number.
`'F'`	Fixed point. Same as `'f'`, but converts `nan` to `NAN` and `inf` to `INF`.
`'g'`	General format. For a given precision `p >= 1`, this rounds the number to `p` significant digits and then formats the result in either fixed-point format or in scientific notation, depending on its magnitude. A precision of `0` is treated as equivalent to a precision of `1`.
`'G'`	General format. Same as `'g'` except switches to `'E'` if the number gets too large. The representations of infinity and NaN are uppercased, too.
`'n'`	Number. This is the same as `'g'`, except that it uses the locale to insert the appropriate number separator characters.
none	The same as `'g'`.

Type	Meaning
`'p'`	Pointer format. This is the default type for pointers and may be omitted.
none	The same as `'p'`.

Expression	Return type	Requirement
`f.parse(pc)`	`PC::iterator`	Shall parse `format-spec` for type `T`, store the parsed specifiers in `*this`, and return an iterator past the end of the parsed range.
`f.format(t, fc)`	`FC::iterator`	Shall format `t` according to the specifiers stored in `*this`, write the output to `fc.out()` and return an iterator past the end of the output range.
`f.format(u, fc)`	`FC::iterator`	Shall not modify u.

Text Formatting

Changes since R2

Changes since R1

Changes since R0

Formatting utilities [format]

Header <format> synopsis [format.syn]

Class template basic_format_arg

Argument visitation

Class template basic_format_args

Function template make_format_args

Function template make_wformat_args

Class template basic_parse_context

Class template basic_format_context

Class template formatter

Formatter requirements

Header `<format>` synopsis [format.syn]

Class template `basic_format_arg`

Class template `basic_format_args`

Function template `make_format_args`

Function template `make_wformat_args`

Class template `basic_parse_context`

Class template `basic_format_context`

Class template `formatter`