Doc No: X3J16/94-0082, WG21/N0469 Date: June 1, 1994 Reply-to: Norihiro KUMAGAI, Sharp Corporation, kuma@slab.tnr.sharp.co.jp Title: Overview of the Proposal for the Template IOStream Classes [pre-Waterloo] Overview of the Proposal for the template IOStream classes Norihiro KUMAGAI SHARP Corporation kuma@slab.tnr.sharp.co.jp 1. Introduction According to the decision in the San Diego Meeting, I prepare a proposal of the template IOStream class. This document will tell us what is the problems, how to approach to solve them, and the solution I would like to propose. During finalizing this document, some of the C++ IOStream experts have eagerly helped me. Without these help I could not finished this document. Especially, Nathan Myers and Takanori Adachi gave me a lot of ideas and reviewed my draft to give me a lot of useful comments. Jerry Schwarz, Per Bothner, and Bill Plauger give me a lot of advices. I much thank those peoples. 2. Requirements R1: Wide-oriented streams: we should provide wide-oriented streambuf classes as well as the traditional narrow-oriented streambuf classes. R2: Parity: the C++ Standard should provide for the same or more functionality for wide-oriented streams as about narrow-oriented streams. R3: Multibyte character streams: wide-oriented filebuf whose external source/sink byte stream (file) represents a sequence of multibyte characters. R4: It needs not, or it should be impossible for an application to access a streambuf with both wide-oriented member functions and narrow-oriented member functions. R5: Raw byte I/O operations: In some cases, we would like to dump out/ input binary data directly to/from a stream. Although the above Requirement R3, we should provide raw byte I/O by any means. R6: Locale support: The behavior of IOStream objects is affected from the Nathan Myers's locale object(X3J11/94-0064R1). R7: streamsize: as in a Per Bothner's c++std-lib-2179 message, A Vote in San Diego requires me to include 'streamsize'. 3. Approach 3.1 Overview of the Approach In this proposal, we introduce a collection of the new "template IOStream classes" and reconstruct the traditional "narrow-oriented(or skinny-oriented) IOStream classes" and the new "wide-oriented IOStream classes." The template IOStream classes accept a "character container type" (char, wchar_t, or some user-defined class) as one of the template parameters, called 'charT' in the following. As well as the template IOStream class, this proposal provides two collections of IOStream classes, narrow-oriented and wide-oriented as the versions generated from the template IOStream classes by specifying the template parameter as 'char' or 'wchar_t' respectively. Here is a templatized version of narrow-oriented streams that can also be implemented as wide-oriented streams. An implementation is obliged to provide instantiations for char and wchar_t stream elements. These can be specializations. So even if it takes a while before implementations can provide the fully templatized version, they can still deliver the two most important specializations, char and wchar_t version. [Character Container Type and Baggage Class] A character container type is a class or a type which represents single 'character'(a element of text data). Two builtin-character types, char, wchar_t, or other user-defined class can be used as a character container type. The template IOStream classes accept a character container type as one of the template parameters to generate a version of the IOStream class handling this container type. A baggage class is a collection of typedefs and member functions with which the template IOStream classes are implemented. The template IOStream classes accept a baggage class as one of the template parameters. Programmers can customize the behavior of the template IOStream classes by providing a new baggage class including their own typedefs and/or function definitions. As well as the IOStream class, the template string classes are now proposed to include the Standard. In the template string draft, there is a baggage class (called 'string_char_baggage') similar to that in this template IOStream class. So Takanori Adachi, the author of the template string class draft, and I decide to integrate both baggages. There are two types, char and wchar_t, which we usually use as character container types. All of the implementation should support both of the types on the IOStream classes specialized to them. The character container type and the baggage class as the template parameters are important from the standardization point of view. These features lead the IOStream class library to augment the extensibility of the character type. [Template IOStream Classes] The new template classes (basic_ios, basic_istream, basic_ostream, basic_streambuf,...) corresponding to all of the traditional IOStream classes are introduced. These classes are the origin of the narrow-oriented and the wide-oriented IOStream classes. The template classes take two parameters, a character container type and a baggage type as follows; template > class basic_streambuf { ... }; The first template parameter, 'charT' represents the character container type. The second template parameter, 'BAG' stands for the baggage class which defines all of the necessary functions and types for implementations of the template IOStream classes. The second parameter, 'BAG' takes the default parameter, ios_baggage corresponding to each character container types. [Member Functions] All member functions in the traditional IOStream classes which handle 'char' type as 'character' are supported in the corresponding template IOStream classes so as to handle the character container type 'charT'. For example, a member function of one of the traditional IOStream classes; istream& istream::getline (char* s, int len, char delim = '\n'); corresponds to a member function of the template IOStream class, 'basic_istream'; basic_istream& basic_istream::getline (charT* s, int len, charT delim = baggage::char_bag::newline()); In the above example, a function 'baggage::char_bag::newline()' is defined in the baggage class. [Raw byte I/O] Programmers sometimes use the traditional IOStream classes to input/output binary data (not a sequence of characters). If all that we would provide about the template IOStream class were to modify all member functions all of whose 'char' type arguments change to 'charT', there would be no means to handle such binary data. So there are several new member functions which handle binary data in the proposal(Requirement 5: Raw byte I/O operations). [Locale Support] The template IOStream classes have a 'locale' object as one of their private members. In all of the locale-dependent operations, the template IOStream classes refer its locale member to use the member function which the locale object provides. We can 'imbue' a locale object into a template IOStream class object to change its corresponding private member. template > class basic_ios { .... public: locale imbue(const locale& loc) { // imbueing the new locale object into the ios class. locale saveloc = local; local = loc; return saveloc; } private: locale local; }; [New IOStream Classes and Header Files] We appends as many new IOStream classes as we can in the traditional header files. A new header file is introduced in order to include the 'basic_convbuf' class. The names of new template classes are as follows; basic_ios basic_streambuf basic_istream basic_ostream basic_smanip basic_imanip basic_omanip basic_strstreambuf basic_istrstream basic_ostrstream basic_stringbuf basic_istringstream basic_ostringstream basic_convbuf basic_filebuf basic_ifstream basic_ofstream 3.2 The Baggage Class: There are several types and functions needed for implementing the template IOStream classes. Some of these types and functions depend on the definition of the character container type. The collection of these functions describes the behavior which the implementation of the template IOStream class expects to the character container class. Those who provide a character container class as the template parameter have to provide all of these functions as well as the container class itself. From the point of view, the collection of these functions can be regarded as the collection of the common definitions for the implementation of the character container class. The baggage class 'ios_baggage' is a template class which takes 'charT' as a parameter. It consists of two parts (1) common definitions for the character container type, charT. (2) repositioning information class, pos_type and off_type There are some requirements about them; * Some of the part (1) member functions are locale-dependent, it means that they have to refer the locale object imbued in each IOStream classes. * About the part (1), users who want to use a new character container type may use other constraints than the default ones. * About the part (2), there can be no common definitions for the implementation of the base class 'basic_streambuf'. On the contrary, some common definitions appear for the implementation of the class 'basic_convbuf'. In order to support these parts; the following approach is adopted. (a) The definition of the baggage class, 'ios_baggage' is as follows; template struct ios_baggage {}; For each character container class, we provide the following version of the template 'ios_baggage'. struct ios_baggage { typedef ios_char_baggage char_bag; // part (1) typedef ios_pos_baggage pos_bag; // part (2) }; In the above definition, 'ios_char_baggage' is a type name of a struct holding common definitions for a character container type as specified in (b), 'ios_pos_baggage' is a type name of a struct holding common definitions for repositioning as specified in (c). It is useful to encapsulate these two types of common definitions separately because they have quite different nature. Library users sometimes specify a new set of common definitions for character if they want to provide a new character container type. On the other hand, common definitions for repositioning much depend on the implementation of the external source/sink stream. To introduce a new character container type causes to specify a new set of common definitions for it but allows us to use the definitions for repositioning unchanged. (b) The template class 'ios_char_baggage' which provides the part (1). There is no declaration(empty body) in the template definition. Every member functions are provided in the each specialized version where the template parameter 'charT' is fixed. The base class 'string_char_baggage' provides a part of the common definition necessary to implement the IOStream classes. // In the following declaration, the base class, 'string_baggage' is // the baggage struct for the template string classes. Some of // the definitions in it are available for implementing the // template IOStream classes. In order to unify these two // baggages, 'ios_char_baggage' is the derived struct from // 'string_char_baggage'. template struct ios_char_baggage : public string_char_baggage {}; // The following struct specialized for a certain character // container type, 'CHAR_T' provides the common definition // necessary for CHAR_T. struct ios_char_baggage { typedef CHAR_T char_type; // 'charT' typedef INT_T int_type; // 'intT' // 'baggage' for a character container type, 'CHAR_T' static char_type to_char_type (int_type c); static int_type to_int_type (char_type c); // type-conversion between 'charT' and 'intT' static bool eq_char_type (char_type c1, char_type c2); static bool eq_int_type (int_type c1, int_type c2); // equal function about 'charT'/'intT' type objects. static int_type eof(); // end-of-file value. static int_type not_eof(); // non-eof value, used in streambuf::overflow(); static bool is_eof(int_type c); // eof predicate function. static char_type newline(); // newline character, as the default value of // one of arguments in istream::getline(). typedef locale::ctype ctype_type; // convenience use for the following declaration static bool is_whitespace(ctype_type& ctype, char_type c) { return ctype.isspace(c); } // locale-dependent whitespace classification // function. Its first argument points the // locale::ctype facet object imbued in the // template IOStream classes. }; // You can expect the following definitions to the base class // 'string_char_baggage'. // // static char_type eos (); // // null character, string termination. // static size_t length (const char_type* s); // // measures the length of a 'character' string. // static char_type* copy (char_type* s1, const char_type* s2, size_t n); // // copies a sequence of 'character's. // The following two struct declarations provides common // definitions for the fundamental character container type, char // and wchar_t. struct ios_char_baggage { typedef char char_type; typedef int int_type; // the type 'intT' // 'baggage' for the character container type, 'CHAR' static char_type to_char_type (int_type c) { return c; } static int_type to_int_type (char_type c) { return c; } static bool eq_char_type (char_type c1, char_type c2) { return c1 == c2; } static bool eq_int_type (int_type c1, int_type c2) { return c1 == c2; } static int_type eof() { return EOF; } static int_type not_eof() { return ~EOF; } // non-eof value, used in streambuf::overflow(); static bool is_eof(int_type c) { return c == EOF; } static char_type newline() { return '\n'; } typedef locale::ctype ctype_type; static bool is_whitespace(char_type c, const ctype_type& loc) { return ctype.isspace (c); } // locale-dependent member functions, // non-static function, refers the 'local' // member function. }; struct ios_char_baggage { typedef wchar_t char_type; typedef wint_t int_type; ... (same as above) }; The following is a brief description of the common definitions; intT(int_type): another character container type which can also hold end-of-file value. It is specified as the return type of some of the IOStream class member function(in case that the character container type is 'char', intT is 'int', for 'wchar_t' it is 'wint_t'). type-conversion functions(to_int_type(),to_char_pos()): conversion between 'charT' and 'intT'. All valid 'character' value in 'charT' shall have the corresponding value on 'intT'. end-of-file value(eof()): an intT type value which is different from any valid character container value. It may be represented in 'charT'. end-of-file predicate function(is_eof()): determines whether the argument is end-of-file or not. newline value(newline()): the 'charT' newline character. whitespace character predicate function(is_whitespace()): determines whether the argument is whitespace or not. This function is locale-dependent so its first argument is a pointer pointing to the locale::ctype facet object. As well as the above definitions, there are some more definitions necessary for the template IOStream classes. The following are provided in the base class, 'string_char_baggage'. end-of-string value(eos): string terminating value, null character. The default constructor for the character container type provides the value. From the IOStream's view, the extractor for 'character' arrays need to terminate the result value by the end-of-string value and the inserter for the 'charT' arrays detects the end-of-string value to determine the length of the string data to be inserted. length function(length()): measures the length of a 'character' string. copy function(copy()): duplicates a sequence of 'characters'. The special definitions of these two functions, length and copy, may be prepared in order to providing higher performance. (c) The template class 'ios_pos_baggage' provides the part (2). They correspond to the types, pos_type and off_type, respectively. The baggage about pos_type and off_type, is provided in the each version of the template 'ios_pos_baggage' The two parameter types, pos_type and off_type are the stream repositioning information container type and corresponding the traditional types, streampos and streamoff. They do not depend on the nature of the character container type but depend on the implementation of the external source/sink stream. An example of 'ios_pos_baggage' is showed as follows; template struct ios_pos_baggage {}; struct ios_pos_baggage { typedef streampos char_pos; typedef streamoff char_off; }; (d) The baggage 'conv_baggage' for the 'basic_convbuf' differs to the above 'ios_baggage'. It takes another baggage struct, 'ios_conv_baggage', which provides other definitions for 'basic_convbuf' implementation. The template class, 'basic_convbuf' is, as defined in 3.6 in this document, a derived class from the 'streambuf' class and provides the functionality for conversion between a sequence of 'character's and a underlaid sequence of other type of 'character's. For example, if you want to provide wide-oriented IOStream classes for multibyte characters in a shift encoding scheme, you have to convert in a wide-oriented streambuf (or its derived) class object between a wide character sequence (or its derived) class object and a underlaid multibyte character sequence. These conversion occurs in the two protected member functions, underflow() and overflow(). The overflow function converts wide characters in the streambuf buffer to the multibyte character sequence and the underflow function converts multibyte characters to the wide characters to fill the streambuf buffer. In order to perform this kind of conversion in the overflow/underflow member functions, we have to maintain (or keep) the shift state at a certain position on the multibyte character sequence. The template class 'basic_convbuf' has a member object to maintain the 'conversion state'. In the above example, the shift state is a kind of the conversion state. The conversion state represents the state at the current position on the underlaid sequence of characters. The state is used to parse the underlaid sequence to extract the 'character's or to generate the underlaid sequence from the 'character's in the streambuf buffer. The nature of the conversion state depends on the underlaid sequence and its encoding rule. In order to support variety of implementations, the definition of the conversion state is provided in the baggage struct as a template parameter. The following code fragment describes the specification about the conversion state; // the template struct, ios_conv_baggage, provides the // additional definitions for the implementation of // 'basic_convbuf'. It takes the type of the conversion state // as the template parameter. template struct ios_conv_baggage { typedef stateT state_type; .... }; // The wchar_t specialization. The class 'state_t' represent // the conversion state for this specialization. template struct conv_baggage {} struct conv_baggage { typedef ios_char_baggage char_bag; typedef ios_pos_baggage pos_bag; typedef ios_conv_baggage conv_bag; }; // The template class, basic_convbuf, has a private member of // the 'state_type' type for maintaining the current conversion // state. template > class basic_convbuf : public basic_streambuf { public: .... typedef cbaggage::conv_bag::state_type state_type; .... private: // state_type state; exposition only }; We will see other definitions appearing in the 'ios_conv_baggage' struct on the following section 3.6 in this document. The fact that the ios_baggage class takes 'charT' type as a template parameter means the principle that the baggage class should contain only the feature depend on the nature of the character container class. Although the principle, we introduce other implementation- dependent or locale-dependent features in the baggage class so as to minimize the number of the parameter types of the template IOStream classes. [Locale Support] All of the constraint functions whose behavior is locale-dependent have to take an pointer argument pointing to a corresponding locale feature set (facet) object. This approach means that the Standard should determine locale-dependency for each constraint member function. All locale-dependent functionalities but the numeric inserter/extractor are provided in the baggage class. Some of them are the whitespace character predicate function, and the code-conversion function. 3.3 Supporting the traditional narrow-oriented IOStream classes If we would like to use some of the traditional narrow-oriented IOStream classes(ifstream, ostrstream, ...), we can get a 'char' version of the template IOStream class. For example, when we need an input string stream 'istrstream' class object, we can declare; basic_istrstream narrowb("parse.me"); It is natural that we define all of the traditional narrow-oriented classes in the appropriate standard header file as follows; class istrstream : public basic_istrstream {}; The IOStream Standard will specify which header files include such definitions. See the section 4.1 of this document. A template baggage class, ios_baggage, is the default template parameter for the template IOStream classes if the character container class is 'char'. It is available to all of the narrow-oriented IOStream classes specified in the Standard. 3.4 Supporting the wide-oriented IOStream classes We can get a 'wchar_t' version of the template IOStream class by specifying 'wchar_t' as a template parameter. For example, the following code fragment; basic_istrstream wideb("parse.me2"); defines a wide-oriented string stream. Thus one of the above requirement(R1: wide-oriented streams) is supported. A template baggage class, ios_baggage, is the default template parameter for the template IOStream classes if the character container class is 'wchar_t'. It is available to all of the wide-oriented IOStream classes specified in the Standard. 3.5 Parity The fact that both wide-oriented and narrow-oriented IOStream classes are the versions generated from the common template IOStream classes ensures the parity between wide-oriented and narrow-oriented IOStream classes (Requirement 2: Parity). In either type of classes, all of the member functions defined in the template IOStream classes are available in quite the same signature(function name, number of arguments, and type of each arguments), and their behaviors are quite same, too. 3.6 Code-Conversion Stream: basic_convbuf For large character set environments, the contents of any text file is a sequence of multibyte characters. So the IOStream classes for file I/O have to input from/output to files containing multibyte characters. In such the classes, it is necessary to convert between multibyte characters and 'character's represented on the character container type. This conversion occurs in streambuf class objects and be characterized by the 'encoding rule'. An 'encoding rule' is a specification how to convert a sequence of multibyte characters to a sequence of 'character's and how to generate a sequence of multibyte characters from a sequence of 'character's. It is implementation-defined and locale-dependent. It consist of the two part of meanings; 1) How each character (a member of the extended character set) is represented on the byte sequence, or how we parse a certain byte sequence to extract a character. If a certain byte sequence may represent two or more kind of characters depending on the parsing state, the encoding rule is called 'state-dependent'. On the contrary, in case a certain byte sequence holds the same meaning, the encoding rule is called 'state-independent'. 2) Which value each character takes as a 'character' type. It is important to provide extensibility of conversion mechanism. In every locale, we may have different encoding rules. In order to support several different encoding rules in a single implementation of streambuf class, it is useful for IOStream implementors to provide some way to customize the supporting encoding rule. Extensibility of conversion mechanism becomes more important(or essential) if we want to utilize the user-definable character container type. If a library user defines a new character container type, he or she have to specify a new encoding rule to convert between byte representation(multibyte character) and the new character container type representation. To support external source/sink stream, the 'basic_convbuf' provides a mechanism which can endure not only multibyte character sequences but also a sequence of (more generalized) 'character's. Not only conversions between multibyte characters and wide characters but also conversions between other kinds of character type streams will become for future. [basic_convbuf] In order to provide extensibility of conversion mechanism, the following new template IOStream class, 'basic_convbuf' is introduced; * It is derived from the class, 'basic_streambuf'. It takes the same template parameters, 'charT' and 'baggage'. template< class charT, class baggage = conv_baggage > class basic_convbuf : public basic_streambuf { public: typedef charT char_type; // The following five definitions, int_type, pos_type, // off_type, upos_type and state_type are provided in the // baggage, 'ios_conv_baggage' in the 'conv_baggage'. typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::char_pos pos_type; typedef baggage::pos_bag::char_off off_type; private: typedef baggage::conv_bag::uchar_type uchar_type; // the 'character' type for the // underlaid 'character' streams. // The following 'locale::codecvt' facet // provides conversion functions between // 'uchar_type' type streams and charT // type streams. typedef baggage::conv_bag::conv_upos upos_type; // repositional information for the // underlaid source/sink stream. typedef baggage::conv_bag::conv_state state_type; // conversion state. It can be applied // to the conversion functions provided // in the locale object. // The following two definitions are for convenience. typedef locale::codecvt codecvt_in; typedef locale::codecvt codecvt_out; ... (declaration/definition of member functions) ... private: locale* local; // imbued locale object holder. state_type state; // the current conversion state codecvt_in* ccvt_in; // locale facet for input conversion codecvt_out* ccvt_out;// locale facet for output conversion }; Note that the default baggage value is 'conv_baggage', not 'ios_baggage'. And there are more typedefs in top of the 'basic_convbuf'. * Its external source/sink stream is a stream of 'character's which is a byte stream representing a sequence of 'character's according to a certain encoding rules. Multibyte character sequence is a typical case of this external source/sink stream. * The expected behavior(protocol) of code-conversion performed in the several protected member functions, overflow, underflow, and uflow, is specified in the Standard as well as the buffering behavior. In these protected member functions, the following converting functions, convin and convout, provided in the baggage object are used to fill/dump the character buffer, g-buf/p-buf from/to the external source/sink multibyte character stream. * We can specify a code-conversion function as a member function in the baggage object. locale::result convin(codecvt_in* ccvt, conv_state& state, const uchar_type* from, const uchar_type* from_end, const char*& from_next, charT* to, charT* to_limit, charT*& to_next); locale::result convout(codecvt_out* ccvt, conv_state& state, const charT* from, const charT* from_end, const charT*& from_next, uchar_type* to, uchar_type* to_limit, uchar_type*& to_next); The function 'convin' converts a sequence of 'uchar_type' characters pointed to by 'from' and 'from_end' to the corresponding sequence of 'character's into the buffer pointed to by 'to'. The function 'convout' converts a sequence of 'character's pointed to by 'from' and 'from_end' to the corresponding sequence of 'uchar_type' characters into the buffer pointed to by 'to'. In the default behavior of these functions, they refer the locale facet object pointed to by 'ccvt' to use their member function 'locale::codecvt::convert' as the conversion functions. Of course library users can provide their own definition of conversion function in their baggage class. These new definition may ignore the 'ccvt' parameter and perform conversion independent from the locale value imbued in the 'basic_streambuf' object. * In order to support state-dependent encoding rules(or shift encoding rules), it is necessary for a 'basic_convbuf' object to maintain the parsing state on the external source/sink 'uchar_type' character stream. We introduce a 'state_type' object which holds the parsing state for the conversion function. The 'basic_convbuf' object has a 'state_type' private member which contains the current parsing state for the external source/sink 'uchar_type' character sequences. In order to provide extensibility for the encoding rule, users can specify the 'state_type' type member in the baggage class as well as the conversion functions. * Repositioning: Performing the repositioning function(seekpos, seekoff) in the basic_convbuf object consists of two processes; (1) Repositioning the external source/sink 'uchar_type' sequence. (2) Restore the parsing state held in a private member of the 'basic_convbuf' object. The Standard specifies two types of information to perform the above processes; (1) repositioning information for the external source/sink 'uchar_type' sequence(file position or else) and (2) the parsing state 'state_type'. A new type 'upos_type' in the baggage object can be specified for maintaining the former information. [Baggage for the basic_convbuf] In order to specify the baggage for the 'basic_convbuf', the third struct, 'conv_baggage' is introduced in the default template class 'conv_baggage'. It provides the following types and functions; uchar_type: the 'character' type which is a type of the element in the external source/sink 'character' sequences. convin and convout: The conversion functions which performs conversion between external source/sink 'uchar_type' character sequences and 'character' sequences according to a certain encoding rule(possibly locale-dependent). state_type: the parsing state for the conversion function. upos_type: positional information for repositioning the external source/sink byte sequence. get_pos, get_off: the constructors from stateT and upos_type . get_posstate, get_offstate: extract stateT from pos_type/off_type. get_posupos, get_offupos: extract upos_type from pos_type /off_type. See the following code fragment; In the example, we want to support multibyte character sequences with a state-dependent encoding as the underlaid streams. class state_t { ... }; // Impl-defined parsing state for // wide-oriented stream. class upos_t { ... }; // Impl-defined file-position for the // external source/sink. class wstreampos { ... }; class wstreamoff { ... }; // Impl-defined repositional object common // to all of the wide-oriented IOStream // classes. template struct ios_conv_baggage {}; // specialized type definitions for wchar_t and state_t struct ios_conv_baggage { typedef wchar_t char_type; typedef state_t conv_state; typedef upos_t conv_upos; typedef char uchar_type; typedef wstreampos pos_type; typedef wstreamoff off_type; typedef locale::codecnv codecvt_in; typedef locale::codecnv codecvt_out; locale::result convin(codecvt_in*, conv_state&, const char*, const char*, const char*&, charT*, charT*, charT*&); locale::result convout(codecvt_out*, conv_state&, const charT*, const charT*, const charT*&, char*, char*, char*&); pos_type& get_pos (conv_state&, conv_upos&); off_type& get_off (conv_state&, conv_upos&); conv_state& get_posstate (pos_type&); conv_state& get_offstate (off_type&); conv_upos& get_posupos (pos_type&); conv_upos& get_offupos (off_type&); }; template struct conv_baggage {} struct conv_baggage { typedef ios_char_baggage char_bag; typedef ios_pos_baggage pos_bag; typedef conv_baggage conv_bag; }; // An implementation of 'overflow' function. template > basic_convbuf& basic_convbuf::overflow () { .... uchar_type *bnext, buf[MAXBUF]; locale::result res; charT *psend // converting the pending characters to the corresponding // multibyte characters. res = baggage::conv_bag::convout(codecnv_out, state, pbeg, pnext, psend, &buf[0], &buf[MAXBUF], bnext); // flush out the multibyte character sequence on 'buf' to the // external sink. .... // adjust the p-pointers so that the rest of the pending // characters comes to the whole of the pending characters. // Note that the copy function for 'charT' type is provided in // the baggage class. baggage::char_bag::copy (psend, pbeg, (pnext - psend)); pnext = pbeg + (pnext - psend); ... } 3.7 pos_type, off_type: the repositioning information container types The types, pos_type, off_type are the stream repositioning information container types. They correspond to streampos, streamoff in the traditional IOStream classes, respectively. We can customize these two types in the baggage class. Its definition relates to the implementation of the streambuf (or its derived) classes. Exactly speaking, it depends on the nature of the external source/sink stream. In order to support a new source/sink, we need to prepare new streampos/streamoff classes so as to represent enough information to restore the source/sink. [pos_type/off_type for the ordinal template IOStream classes] We introduce a template class 'ios_pos_baggage' to provide the definition and their baggage. Because they do not depend on the nature of the character container type but depend on the implementation of the external source/sink stream, no special member functions(baggage) are provided as follows; class astrpos { ... }; // Impl-defined streampos/streamoff class; class astroff { ... }; template struct ios_pos_baggage {}; struct ios_pos_baggage { typedef astrpos char_pos; typedef astroff char_off; // There are no more special member functions }; [pos_type/off_type for the 'basic_convbuf' class] They can hold two types of information; the parsing state and the current position of the external source/sink. In order to reposition the basic_convbuf class, we have to restore the parsing state held in the object as well as the current position of the external source/sink. [streampos/streamoff] All of the narrow-oriented IOStream classes share the same definitions of pos_type , off_type, namely, streampos and streamoff as in the traditional IOStream classes. [wstreampos/wstreamoff] All of the wide-oriented IOStream classes share the same definition of pos_type , off_type, that is, wstreampos and wstreamoff. Because wfilebuf is derived from basic_convbuf, wstreampos and wstreamoff obeys to the baggage in 'conv_baggage'. An implementor provides the definition of 'conv_baggage' for 'wstreampos/wstreamoff. 3.8 Locale Support There are several locale-dependent features in the IOStream classes as follows; (1) inserters, (2) extractors, (2) istream::ipfx, (3) the encoding rule for a basic_convbuf class, (4) widen/narrow. (1) Inserters The numeric conversion behaviors of the following inserters are locale-dependent. basic_ostream::operator<<(short val); basic_ostream::operator<<(unsigned short val); basic_ostream::operator<<(int val); basic_ostream::operator<<(unsigned int val); basic_ostream::operator<<(long val); basic_ostream::operator<<(unsigned long val); basic_ostream::operator<<(float val); basic_ostream::operator<<(double val); basic_ostream::operator<<(long double val); According to the Nathan Myers's locale object draft [X3J16/94-0064R1,WG21/N0451R1], the class locale::num_get<> and locale::num_put<> handle locale-dependent numeric formatting and parsing. The above inserter functions refers the imbued locale value to utilize these numeric formatting functionality. The formatting conversion occurs as if it performed the following code fragment; locale::num_put& fmt = loc.use< locale::num_put >(); fmt.put (ostreambuf_iterator(*this), *this, loc, val); In the above fragment, 'loc' stands for the private member of the 'basic_ios' class which maintains the imbued locale object. The first argument provides an object of the 'ostreambuf_iterator' class which is an iterator for ostream class. It bypasses ostreams and uses streambufs directly. Class locale relies on these types as its interface to iostream, since for flexibility it has been abstracted away from direct dependence on ostream. Because the class, locale::num_put<> depend on the class, 'ostreambuf_iterator' as the fundamental access way to the output stream with some efficiency, the class, 'ostreambuf_iterator' is defined in the header. The template class, 'ostreambuf_iterator' writes successive 'character's onto the output stream from which it was constructed. It is not possible to get a value out of the output iterator. Two output iterators are equal if they are constructed with the same output streambuf. The following is the definition of the class, 'ostreambuf_iterator'; template > class ostreambuf_iterator { public: typedef charT char_type; typedef baggage baggage_type; typedef basic_streambuf streambuf; typedef basic_ostream ostream; private: streambuf* sbuf_; public: ostreambuf_iterator () : sbuf_(0) {} ostreambuf_iterator (ostream& s) : sbuf_(s.rdbuf()) {} ostreambuf_iterator (streambuf* s) : sbuf_(s) {} ostreambuf_iterator& operator*() { return *this; } ostreambuf_iterator& operator++() { return *this; } ostreambuf_iterator& operator++(int) { return *this; } ostreambuf_iterator& operator= (charT c) { sbuf_->sputc(baggage::to_int_type(c)); } bool equal (ostreambuf_iterator& b) { return sbuf_ == b.sbuf_; } }; output_iterator iterator_category (const ostreambuf_iterator&) { return output_iterator(); } template > bool operator==(ostreambuf_iterator& a, ostreambuf_iterator& b) { return a.equal (b); } template > bool operator!=(ostreambuf_iterator& a, ostreambuf_iterator& b) { return !a.equal (b); } (2) Extractors The numeric conversion behaviors of the following extractors are locale-dependent. basic_istream::operator>>(short& val); basic_istream::operator>>(unsigned short& val); basic_istream::operator>>(int& val); basic_istream::operator>>(unsigned int& val); basic_istream::operator>>(long& val); basic_istream::operator>>(unsigned long& val); basic_istream::operator>>(float& val); basic_istream::operator>>(double& val); basic_istream::operator>>(long double& val); As in the case of the inserters, these extractors depend on the Nathan Myers's locale::num_get<> object to perform parsing the input stream data. The conversion occurs as if it performed the following code fragment; HOLDERTYPE tmp; locale::num_get& fmt = loc.use< locale::num_get >(); fmt.get (iter, *this, loc, tmp); if ((val = (TYPE)tmp) != tmp) // set fail bit... In the above fragment, 'loc' stands for the private member of the 'basic_ios' class, TYPE stands for the type of the argument of the extractor, and HOLDTYPE is as follows; * for 'short', 'int' and 'long', the HOLDTYPE is 'long'; * for 'unsigned short', 'unsigned int' and 'unsigned long', the HOLDTYPE is 'unsigned long'. * for 'float', 'double', the HOLDTYPE is 'double'. * for 'long double', the HOLDTYPE is 'long double'. The first argument provides an object of the 'istream_iterator' class which is an iterator pointed to an input stream. It bypasses istreams and uses streambufs directly. Class locale relies on this type as its interface to istream, since the flexibility it has been abstracted away from direct dependence on istream. The definition of the class, 'istreambuf_iterator' is provided in the header, . The template class, 'istreambuf_iterator', reads successive 'character's from the streambuf for which it was constructed. After it is constructed, and every time ++ is used, the iterator reads and stores a value of 'character'. If the endo of stream is reached (streambuf::sgetc() returns baggage::eof()), the iterator becomes equal to the 'end of stream' iterator value. The default constructor 'istreambuf_iterator()' and the constructor 'istreambuf_iterator(0)' always construct an end of stream iterator object, which is the only legitimate iterator to be used for the end condition. The result of operator*() on an end of stream is undefined. For any other iterator value a 'const charT&' is returned. It is impossible to store things into input iterators. Note that in the input iterators, ++ operators are not equality preserving, that is, i == j does not guarantee at all that ++i == ++j. Every time ++ is used a new value is used. The practical consequence of this fact is that an 'istreambuf_iterator' object can be used only for one-pass algorithms, which actually makes perfect sense, since for multi-pass algorithms it is always more appropriate to use in-memory data structures. Two end of stream iterators are always equal. An end of stream iterator is not equal to a non-end of stream iterator. Two non-end of stream iterators are equal when they are constructed from the same stream. The following is the definition of the class, 'istream_iterator'; template > class istreambuf_iterator { public: typedef charT char_type; typedef baggage baggage_type; typedef baggage::int_type int_type; typedef basic_streambuf streambuf; typedef basic_istream istream; class proxy { charT keep_; streambuf* sbuf_; proxy (charT c, streambuf* sbuf) : keep_(c), sbuf_(sbuf) {} public: charT operator*() { return keep_; } friend class istreambuf_iterator; }; private: streambuf* sbuf_; public: istreambuf_iterator () : sbuf_(0) {} istreambuf_iterator (istream& s) : sbuf_(s.rdbuf()) {} istreambuf_iterator (streambuf* s) : sbuf_(s) {} istreambuf_iterator (const proxy& p) : sbuf_(p.sbuf_) {} charT operator* { return baggage::to_char_type(sbuf_->sgetc()); } istreambuf_iterator& operator++() { sbuf_->sbumpc(); return *this; } proxy operator++(int) { return proxy (baggage::to_char_type(sbuf_->sbumpc()), sbuf_); } bool equal (istreambuf_iterator& b) { // If either of the two iterator is an end-of-stream // iterator, the other iterator is examined whether it is an // end-of-stream by invoking sbuf_->sgetc(). return !sbuf_ ? (!b.sbuf_ || baggage::is_eof (b.sbuf_->sgetc())) : !b.sbuf_ ? baggage::is_eof (sbuf_->sgetc()) : (sbuf_ == b.sbuf_); } }; input_iterator iterator_category (const istreambuf_iterator&) { return input_iterator(); } template > bool operator==(istreambuf_iterator& a, istreambuf_iterator& b) { return a.equal (b); } template > bool operator!=(istreambuf_iterator& a, istreambuf_iterator& b) { return !a.equal (b); } (3) istream::ipfx: whitespace characters to be skipped The proposal will say as follows; The function 'basic_istream::ipfx()' uses the function, 'bool baggage::char_bag::is_whitespace(const locale*, charT)' in the baggage structure to determine whether the next input character is whitespace or not; A typical implementation of the ipfx function may be as follows; template > int basic_istream::ipfx() { ... // skipping whitespace according to a constraint function, // is_whitespace intT c; typedef locale::ctype ctype_type; ctype_type& ctype = getloc().use(); while ((c = rdbuf()->snextc()) != eof()) { if (!baggage::char_bag::is_whitespace (ctype, c)==0) { rdbuf()->sputbackc (c); break; } } ... } In case we use 'ios_baggage' or 'ios_baggage', the behavior of the constraint function, baggage::char_bag::is_whitespace() is as if it invokes, locale::ctype& ctype = getloc().use >(); ctype.is(locale::ctype::SPACE, c); otherwise, the behavior of the function, baggage::char_bag::is_whitespace() is unspecified. Those who want to use locale-independent whitespace predicate can specify their definition of is_whitespace in their new ios_char_baggage as follows; struct my_baggage : public ios_baggage { typedef my_char_baggage char_bag; }; struct my_char_baggage : public ios_char_baggage { static bool is_whitespace (const locale::ctype& ctype, char c) { ....(my own implementation)... } }; (4) code-conversion functions: In the description of the 'basic_convbuf', in case we use 'conv_baggage' or 'conv_baggage', the behavior of the constraint function; locale::result convin(codecvt_in ccvt_in, conv_state& state, const uchar_type* from, const uchar_type* from_end, const char*& from_next, charT* to, charT* to_limit, charT*& to_next); locale::result convout(codecvt_out ccvt_out, conv_state& state, const charT* from, const charT* from_end, const charT*& from_next, uchar_type* to, uchar_type* to_limit, uchar_type*& to_next); is the same as that of the following code fragment; ccvt_in.convert(state, from, from_end, from_next, to, to_limit, to_next); and; ccvt_out.convert(state, from, from_end, from_next, to, to_limit, to_next); respectively. Otherwise, the behaviors of the functions convin and convout are unspecified. (5) widen/narrow: The terms, 'widen' and 'narrow' mean the conversion between a character represented by the container character and a single-byte character. The term, 'widen' represents the conversion from a single-byte character to the corresponding 'character' representation. On the contrary, the term, 'narrow' represent the opposite conversion. The locale object provides the 'widen/narrow' conversion as well as the 'codecnv' conversion. It differs to the 'codecnv' conversion between 'character' and narrow character. The most notable difference between these two types of conversion is the region of the character set covered by them. Although the 'codecnv' conversion covers all of the extended character set(or, all of the large character set supported in a certain implementation), the 'widen/narrow' functions return a valid result only if the character can be represented a single-byte character. So we can safely use these functions only if we really lives in some single-byte character environment. As Nathan Myers said[X3J16/94-0064R1,WG21/N0451R1], we should regard; the purpose of the 'widen/narrow' functions is purely to help adapt old char-oriented code (expecting all the characters can be represented by a single byte) for wide-character environment (or generalized character container type environment) when nothing trikky is intended. Extractors/inserters for the single-byte string use the narrow/widen conversion functions. The extracter converts a 'character' to the corresponding single-byte character to fill the string argument and the inserter converts a single-byte character to a 'character' in order to write them. As Jerry proposed in c++std-lib-1870, template > basic_ostream& operator<<(basic_ostream&, const basic_string&); will be provided. Its behavior relates to; locale::ctype ctype_facet = loc.use< locale::ctype >(); ctype_facet.widen (char* lo, char* hi, charT* to); And; template > basic_istream& operator>>(basic_istream&, const basic_string&); will be provided. Its behavior relates to; locale::ctype ctype_facet = loc.use< locale::ctype >(); ctype_facet.narrow (charT* lo, charT* hi, char* to); 3.9 Skinny character handling in the template IOStream classes (1) In principal, it is not suitable to support skinny character handling with the same name of the member functions as in the traditional IOStream classes. I believe that even one of the functions; ostream::write(char*,int), should be changed to support the character container type 'charT' and should be prohibited its original signature, basic_ostream::write(char*,int); The reason is as follows. If we had the same signature, error occurrences in compile time could not be expected in rewriting the existing source code developed for the traditional single-byte oriented environment. If an existing source code try to apply to the template IOStream classes(or wide-oriented IOStream classes), it should be rewritten by some humans hand. In such a case, we should check the every occurrence of inserting/extracting (skinny) 'char' type objects to determine whether it should be rewritten to use 'wchar_t' or not. In case we look over one or more occurrences of the inserting/ extracting 'char' type, we are happy to receive errors in compilation time. If we provided inserter/extractor for skinny 'char' type object(char, char*, char&), we could not expect the error in compilation time. (2) I agree with the necessity to input/output skinny character with the template IOStream classes as 'binary data input/output'. So we have to provide some way to do so; I am planning to introduce some new member functions to do so in th following principles. * No same signature is provided; In order to support skinny characters, either of the following is adopted (1) Introducing a new function(whose name differs from any of the traditional ones). Ex) not 'write(char*, int)' but 'write_byte(char*, int)'. (2) Using a basic_string because no existing programs ever use basic_string. * There are two types of functionalities in input/output skinny characters. According to the type, the behavior of handling skinny character is determined. (1) narrow/widen ... each skinny character is widen to the corresponding 'character'. The behavior can be characterized by the 'narrow'/'widen' member function in the 'locale::ctype<>' facet. (2) handling binary data ... the binary data in a certain object is written directly on the external sink, or a part of the external source stream is read into an object. The behavior of such binary input/output will be implementation-defined because it closely relates to the implementation of the external source/sink stream. In some stream implementation, this operation may be rejected in runtime. Under the condition we cannot assume whether the external source/sink is a byte stream, can we specify the behavior in the Standard? 4. Specification 4.1 Table Of Contents... The contents of the chapter in the Standard IOStream Library part(17.4) is as follows; 17.4 Input/output 17.4.1 Header 17.4.1.1 Template struct ios_baggage 17.4.1.1.1 Template struct ios_char_baggage 17.4.1.2 Repositional information classes 17.4.1.2.1 Template struct ios_pos_baggage 17.4.1.2.2 The specialized class POS_T, OFF_T 17.4.1.2.3 Type streamoff 17.4.1.2.4 Class streampos 17.4.1.2.5 Type wstreamoff 17.4.1.2.6 Class wstreampos 17.4.1.3 Class basic_ios 17.4.1.22 Class ios; 17.4.1.23 Class wios; 17.4.2 Header 17.4.2.1 Template class basic_streambuf 17.4.2.2 Class streambuf 17.4.2.3 Class wstreambuf 17.4.3 Header 17.4.3.1 Template class istream_iterator 17.4.3.1.1 Template class istreambuf_iterator::proxy 17.4.3.2 Class basic_istream 17.4.3.3 ws(basic_istream&) ... 17.4.3.4 Class istream 17.4.3.5 Class wistream 17.4.4 Header 17.4.4.1 Template class ostream_iterator 17.4.4.2 Template class basic_ostream 17.4.4.3 endl(basic_ostream&) 17.4.4.4 ends (basic_ostream&) 17.4.4.5 flush (basic_ostream&) 17.4.4.6 Class ostream 17.4.4.7 Class wstream 17.4.5 Header 17.4.5.1 Template class basic_smanip 17.4.5.2 Template class basic_imanip 17.4.5.3 Template class basic_omanip 17.4.5.4 Instantiations of manipulators 17.5.4.5 Template class smanip 17.5.5.6 Template class imanip 17.5.5.7 Template class omanip 17.5.5.8 Template class wsmanip 17.5.5.9 Template class wimanip 17.5.5.10 Template class womanip 17.4.6 Header 17.4.6.1 Template class basic_strstreambuf 17.4.6.2 Template class basic_istrstream 17.4.6.3 Template class basic_ostrstream 17.4.7 Header 17.4.7.1 Template class basic_stringbuf 17.4.7.2 Template class basic_istringstream 17.4.7.3 Template class basic_ostringstream 17.4.7.4 Class stringbuf 17.4.7.5 Class istringstream 17.4.7.6 Class ostringstream 17.4.7.7 Class wstringbuf 17.4.7.8 Class wistringstream 17.4.7.9 Class wostringstream 17.4.8 Header 17.4.8.1 Template class conv_baggage 17.4.8.2 Template class ios_conv_baggage 17.4.8.3 Template class basic_convbuf 17.4.9 header 17.4.9.1 template class basic_filebuf 17.4.9.2 Class filebuf 17.4.9.3 Class wfilebuf 17.4.9.4 Template class basic_ifstream 17.4.9.5 Class ifstream 17.4.9.6 Class wifstream 17.4.9.7 Template class basic_ofstream 17.4.9.8 Class ofstream 17.4.9.9 Class wofstream 17.4.9.10 Template class basic_stdiobuf 17.4.10 Header 17.4.10.1 Object cin 17.4.10.2 Object cout 17.4.10.3 Object cerr 17.4.10.4 Object clog 4.2 the baggage class: ios_baggage template struct ios_char_baggage; template struct ios_pos_baggage; // forward reference // The declaration of the baggage 'ios_baggage'. template struct ios_baggage {}; struct ios_baggage { typedef ios_char_baggage char_bag; typedef ios_pos_baggage pos_bag; }; struct ios_baggage { typedef ios_char_baggage char_bag; typedef ios_pos_baggage pos_bag; }; // // Baggage for the character container type. // template struct ios_char_baggage : public string_char_baggage {}; // No special definition/declaration is provided here. // The base class (or struct), 'string_char_baggage' provides // the common definitions common between the template string // classes and the template IOStream classes. // The following template definition of the ios_char_baggage class // is for a certain character container class 'CHAR_T'. struct ios_char_baggage { // the character container types, 'CHAR_T' and 'INT_T' // The implementor should provide definition of there types outside. typedef CHAR_T char_type; typedef INT_T int_type; // 'baggage' for the character container type, 'CHAR' static char_type to_char_type (int_type); static int_type to_int_type (char_type); static bool eq_char_type (char_type, char_type); static bool eq_int_type (int_type, int_type); static int_type eof(); static int_type not_eof(); // non-eof value, used in streambuf::overflow(); static bool is_eof(int_type); static char_type newline(); typedef locale::ctype ctype_type; static bool is_whitespace(ctype_type& ctype, char_type c) { return ctype.isspace(c); } // locale-dependent member functions, // non-static function, // refers the 'local' member function. }; // The default baggage for the character container type 'char', // and 'wchar_t'. struct ios_char_baggage { typedef char char_type; typedef int int_type; static char_type to_char_type (int_type c) { return c; } static int_type to_int_type (char_type c) { return c; } static bool eq_char_type (char_type c1, char_type c2) { return c1 == c2; } static bool eq_int_type (int_type c1, int_type c2) { return c1 == c2; } static int_type eof() { return EOF; } static int_type not_eof() { return ~EOF; } static bool is_eof(int_type c) { return c == EOF; } static char_type newline() { return '\n'; } typedef locale::ctype ctype_type; static bool is_whitespace(ctype_type& ctype, char_type c) { return ctype.isspace(c); } }; struct ios_char_baggage { typedef wchar_t char_type; typedef wint_t int_type; static char_type to_char_type (int_type c) { return c; } static int_type to_int_type (char_type c) { return c; } static bool eq_char_type (char_type c1, char_type c2) { return c1 == c2; } static bool eq_int_type (int_type c1, int_type c2) { return c1 == c2; } static int_type eof() { return WEOF; } static int_type not_eof() { return ~WEOF; } static bool is_eof(int_type c) { return c == WEOF; } static char_type newline() { return L'\n'; } typedef locale::ctype ctype_type static bool is_whitespace(ctype_type& ctype, char_type c) { return ctype.isspace(c); } }; // The baggage for pos_type , off_type. template struct ios_pos_baggage {}; struct ios_pos_baggage { // pos_type and off_type typedef streampos char_pos; typedef streamoff char_off; // If there are no classes derived from the 'basic_convbuf' class // among the narrow-oriented IOStream classes, no further // baggage are introduced here. Otherwise, the same // baggage as those for 'wstreampos' class are needed. }; // The baggage class for basic_convbuf has some additional features // for basic_convbuf::seekpos, seekoff member functions as // follows; template struct conv_baggage {} struct conv_baggage { typedef ios_char_baggage char_bag; typedef ios_pos_baggage pos_bag; typedef ios_conv_baggage conv_bag; }; // the definitions of the classes, wstreampos, wstreamoff, state_t are // provided in other place. template struct conv_baggage {}; struct ios_conv_baggage { typedef state_t conv_state; // the parsing state for the conversion function. typedef upos_t conv_upos; // positional information for the external source/sink // character stream. typedef wstreampos pos_type; typedef wstreamoff off_type; locale::result convin(locale*, conv_state&, const char*, const char*, const char*&, charT*, charT*, charT*&); locale::result convout(locale*, conv_state&, const charT*, const charT*, const charT*&, char*, char*, char*&); // the code-conversion function: the default conversion // function is depend on the locale_codecvt facet. Users can // easily customize the encoding scheme by specifying their // own conversion functions in a new conv_baggage class. pos_type& get_pos (conv_state&, conv_upos&); off_type& get_off (conv_state&, conv_upos&); // constructs a pos_type/off_type object from conv_state, // conv_upos. These are used to make the return value of the // 'basic_convbuf::seekpos/seekoff' protected member function. conv_state& get_posstate (pos_type&); conv_state& get_offstate (off_type&); conv_upos& get_posupos (pos_type&); conv_upos& get_offupos (off_type&); // extract the conv_state or the conv_upos value from a // pos_type/off_type object. These are used in the // 'basic_convbuf::seekpos/seekoff' protected member functions // to reposition the external source/sink multibyte character // stream and to restore the private member 'state'. }; // It is necessary to satisfy the baggage for 'basic_convbuf' in // the wstreampos/wstreamoff class object(pos_type /off_type for wide-oriented // stream) so as to support wfilebuf, which is a derived class from // the basic_convbuf. struct ios_conv_baggage { protected: typedef wstreampos pos_type; typedef wstreamoff off_type; ...(to be filled)... }; 4.3 basic_ios //## A part of the declaration is omitted because the part is quite //## the same as those in the traditional 'ios' class. template > class basic_ios { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } private: typedef basic_ios ios_type; //## Note: An idea to declare eof as a constant, that is, static const //## IEOF = baggage::char_bag::eof();, is rejected because it cause a //## constraint that charT and int_type should be integral //## types. public: class failure public: xmsg { ... (same as in the traditional ios) }; typedef T1 fmtflags; ... (same as in the traditional ios) basic_ios (basic_streambuf* sb_arg); virtual ~basic_ios(); operator void*(); int operator!() const; ios_type& copyfmt(const ios_type& rhs); basic_ostream* tie() const; basic_ostream* tie(basic_ostream* tiestr_arg); basic_streambuf* rdbuf() const; basic_streambuf* rdbuf(basic_streambuf* sb_arg); //## Note: template parameter coupling between basic_ios and //## basic_streambuf: Both basic_ios and basic_streambuf corresponding //## to it should take the same template parameter . We need //## not allow to make a couple of basic_ios and //## basic_streambuf. ... (same as in the ios) //## Specifying fill character int fill(int_type ch); //## Imbueing locale object. locale imbue (locale loc); locale getloc (); //## Note: imbue/getloc member functions based on Nathan's 93-0167R1, N0374R1 protected: basic_ios(); init(basic_streambuf* sb_arg); private: // basic_streambuf* sb; exposition only // basic_ostream* tiestr; exposition only // iostate state; exposition only // iostate except; exposition only // fmtflags fmtfl; exposition only // int prec, wide; exposition only // charT fillch; exposition only // Off course, fill character type should be templated // static int index; exposition only // int* iarray; exposition only // void** parray; exposition only //## the locale member object. // locale local; exposition only //## class locale object is immutable. So we do not worry about its size; }; 4.4 basic_streambuf template > class basic_streambuf { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: ~virtual basic_streambuf (); pos_type pubseekoff (off_type off, ios_seekdir way, ios_openmode which = ios::in | ios::out); pos_type pubseekpos (pos_type sp, ios_openmode which = ios::in | ios::out); basic_streambuf* pubsetbuf (char_type* s, streamsize n); int pubsync(); int_type sbumpc(); int_type sgetc(); int sgetn(char_type* s, streamsize n); int_type snextc(); int_type sputbackc(char_type c); int sputn(const char_type* s, streamsize n); protected: basic_streambuf (); char_type* eback() const; char_type* gptr() const; char_type* egptr() const; void gbump(int n); void setg(char_type* gbeg_arg, char_type* gnext_arg, char_type* gend_arg); char_type* pbase() const; char_type* pptr() const; char_type* epptr() const; void pbump(int n); void setp(char_type* pbeg_arg, char_type* pend_arg); virtual int overflow(int_type c = eof()); virtual int pbackfail(int_type c = eof()); virtual int underflow(); virtual int uflow(); virtual int xsgetn(char_type* s, streamsize n); virtual int xsputn(const char_type* s, streamsize n); virtual pos_type seekoff(off_type, ios::seekdir way, ios_openmode:: which = ios::in | ios::out); virtual pos_type seekpos(pos_type sp, ios_openmode:: which = ios::in | ios::out); virtual basic_streambuf* setbuf(char_type* s, streamsize n); virtual int sync(); //## The following two member functions are introduced to support Raw //## byte I/O, whose behaviors are implementation-defined. virtual int write_byte (const char* buf, int len); virtual int read_byte (char* buf, int len); private: char_type* gbeg; char_type* gnext; char_type* gend; char_type* pbeg; char_type* pnext; char_type* pend; }; 4.5 basic_istream template > class basic_istream : virtual public basic_ios { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } private: typedef basic_istream istream_type; typedef basic_ios ios_type; public: basic_istream(basic_streambuf* sb); virtual ~basic_istream(); int ipfx(int noskipws = 0); void isfx(); istream_type& operator>>(istream_type& (*pf)(istream_type&)); istream_type& operator>>(ios_type& (*pf)(ios_type&)); istream_type& operator>>(char_type* s); istream_type& operator>>(char_type& c); istream_type& operator>>(short& n); .... istream_type& operator>>(basic_streambuf& sb); //## Note: istream& operator>>(streambuf& sb), Both istream and //## streambuf takes exactly the same charT and baggage. int_type get(); istream_type& get(char_type* s, streamsize n, char_type delim = newline()); //## Note: Need newline character for specifying delim as istream::get //## parameter. istream_type& get(char_type& c); istream_type& get(basic_streambuf& sb, char_type delim = newline()); //## Note: newline character depends on the locale?! istream_type& getline(char_type* s, streamsize len, char_type delim = newline()); istream_type& ignore (streamsize n = 1, int_type delim = newline()); istream_type& read(char_type* s, streamsize n); //## A new member function for supporting Raw byte I/O //## in which it performs 'rdbuf().read_byte (s, n);' istream_type& read_byte (char* s, streamsize n); int_type peek(); istream_type& putback(char_type c); istream_type& unget(); streamsize gcount() const; int sync(); private: // streamsize chcount; exposition only }; 4.6 basic_ostream template > class basic_ostream : virtual public basic_ios { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } private: typedef basic_ostream ostream_type; public: basic_ostream (basic_streambuf* sb); virtual ~basic_ostream(); int opfx(); void osfx(); ostream_type& operator<<(ostream_type& (*pf)(ostream_type&)); ostream_type& operator<<(ios_type& (*pf)(ios_type&)); ostream_type& operator<<(const char_type* s); ostream_type& operator<<(charT c); ostream_type& operator<<(short n); .... ostream_type& operator<<(basic_streambuf& sb); ostream_type& put (); ostream_type& write (const char_type* s, streamsize n); //## A new member function for supporting Raw byte I/O //## in which it performs 'rdbuf().writebyte (s, n);' ostream_type& write_byte (const char* s, streamsize n); ostream_type& flush(); }; 4.7 basic_smanip //## Note: terrible triple template parameters! //## basic_smanip!!! Shall we provide another baggage //## class for manipulators? template > class basic_smanip { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_smanip (ios_type& (*pf_arg)(ios_type&, T), T); // ios_type& (*pf)(ios_type&, T); exposition only // T manarg; exposition only }; 4.8 basic_strstreambuf template > class basic_strstreambuf : public basic_streambuf { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_strstreambuf (streamsize alsize_arg = 0); basic_strstreambuf (void* (*palloc_arg)(size_t), void* (*pfree_arg)(void*)); basic_strstreambuf (char_type* gnext_arg, streamsize n, char_type* pbeg_arg = 0); basic_strstreambuf (const char_type* gnext_arg, streamsize n); //## Note: null character constraint is needed in ios_baggage. virtual ~basic_strstreambuf(); void freeze (int = 1); char_type* str(); streamsize pcount(); protected: // virtual int_type overflow(int_type c = eof()); inherited // virtual int_type pbackfail(int_type c = eof()); inherited // virtual int_type underflow(); inherited // virtual int_type uflow(); inherited // virtual streamsize xsgetn(char_type* s, streamsize n); inherited // virtual streamsize xsputn(const char_type* s, streamsize n); inherited // virtual pos_type seekoff(off_type off, ios::seekdir way, inherited // ios::openmode which = ios::in | ios::out); // virtual pos_type seekpos(pos_type pos, inherited // ios::openmode which = ios::in | ios::out); // virtual basic_streambuf* setbuf(char_type* s, streamsize n); inherited // virtual int sync(); inherited private: // typedef T1 strstate; exposition only // static const strstate allocated; exposition only // static const strstate constant; exposition only // static const strstate dynamic; exposition only // static const strstate frozen; exposition only // strstate strmode; exposition only // streamsize alsize; exposition only // void* (*palloc)(size_t); exposition only // void (*pfree)(void*); exposition only }; 4.9 basic_istrstream template > class basic_istrstream : public basic_istream { public: basic_istrstream(); basic_istrstream(const char_type* s); //## Note: Need 'null character' to provide this constructor. Suitable? basic_istrstream(const char_type* s, streamsize n); basic_istrstream(char_type* s); basic_istrstream(char_type* s, streamsize n); virtual ~basic_istrstream(); basic_strstreambuf* rdbuf() const; private: // basic_strstreambuf sb; exposition only }; 4.10 basic_ostrstream template > class basic_ostrstream : public ostream { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_ostrstream(); basic_ostrstream(char_type* s, int n, openmode mode = out); //## Comment: Is 'openmode' really 'ios::openmode'? virtual ~basic_ostrstream(); basic_strstreambuf* rdbuf() const; void freeze(int freezefl); char_type* str(); int pcount() const; private: // basic_strstreambuf sb; exposition only }; 4.11 basic_stringbuf template > class basic_stringbuf : public basic_streambuf { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_stringbuf(ios::openmode which = ios::in | ios::out); basic_stringbuf(const basic_string& str, ios::openmode which = ios::in | ios::out); basic_string str() const; void str(const basic_string& str_arg); protected: // virtual int_type overflow(int_type c = eof()); inherited // virtual int_type pbackfail(int_type c = eof()); inherited // virtual int_type underflow(); inherited // virtual int_type uflow(); inherited // virtual streamsize xsgetn(char_type* s, streamsize n); inherited // virtual streamsize xsputn(const char_type* s, streamsize n); inherited // virtual pos_type seekoff(off_type off, ios::seekdir way, inherited // ios::openmode which = ios::in | ios::out); // virtual pos_type seekpos(pos_type pos, inherited // ios::openmode which = ios::in | ios::out); // virtual basic_streambuf* setbuf(char_type* s, int n); inherited // virtual int sync(); inherited private: // ios::openmode mode; exposition only. }; //## Note: same 'charT' type string can be fed. 4.12 basic_istringstream template > class basic_istringstream : public basic_istream { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_istringstream(ios::openmode which = ios::in); basic_istringstream(const basic_string& str, ios::openmode which = ios::in); virtual ~basic_istringstream(); basic_stringbuf* rdbuf() const; basic_string str() const; void str(const basic_string& str); private: // basic_stringbuf sb; exposition only }; 4.13 basic_ostringstream template > class basic_ostringstream : public basic_ostream { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_ostringstream(ios::openmode which = ios::out); basic_ostringstream(const basic_string& str, ios::openmode which = ios::out); virtual ~basic_ostringstream(); basic_stringbuf* rdbuf() const; basic_string str() const; void str(const basic_string& str); private: // basic_stringbuf sb; }; 4.14 basic_convbuf template > class basic_convbuf : public basic_streambuf { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } private: typedef baggage::conv_bag::conv_state state_type; typedef baggage::conv_bag::conv_upos upos_type ; public: basic_convbuf(); //## Note: In order to decrease the number of template parameters of //## codecvt, (i) we assume that one of the pair types is 'char'(or //## byte), (ii) we provide the two-way conversions(from char to charT //## and vice versa) in the locale_convbuf. So the definition of //## locale_convbuf might; //## //## template //## class locale_codecvt : public locale::features { //## public: //## enum result { OK = 0, PARTIAL, ERROR } //## protected: //## virtual result do_convin(stateT& state, const char* from, const char //## from_end, const char*& from_next, charT* to, charT* to_limit, //## charT*& to_next); //## virtual result do_convout(stateT& state, const charT* from, const charT //## from_end, const charT*& from_next, char* to, char* to_limit, //## char*& to_next); //## public: //## result do_convin(stateT& state, const char* from, const char //## from_end, const char*& from_next, charT* to, charT* to_limit, //## charT*& to_next); //## result do_convout(stateT& state, const charT* from, const charT //## from_end, const charT*& from_next, char* to, char* to_limit, //## char*& to_next); //## private: //## static const size_t id_; //## public: //## static size_t id() { return id_; } //## locale_codecvt (size_t refs = 0) : locale::features(refs) {} //## ~locale_codecvt(); {} //## }; virtual ~basic_convbuf(); //## Note: If a locale object is imbued into a basic_istream/ostream, //## it use 'attach' member function in order to 'imbue' its //## locale::codecnv feature. protected: // virtual int overflow(int_type c = eof()); inherited // virtual int pbackfail(int_type c = eof()); inherited // virtual int underflow(); inherited // virtual int uflow(); inherited // virtual streamsize xsgetn(charT* s, streamsize n); inherited // virtual streamsize xsputn(const charT* s, streamsize n); inherited // virtual pos_type seekoff(off_type, ios::seekdir way, // ios_openmode:: which = ios::in | ios::out); inherited // virtual pos_type seekpos(pos_type sp, ios_openmode:: which = ios::in | ios::out); inherited // virtual basic_streambuf* setbuf(charT* s, streamsize n); inherited // virtual int sync(); inherited private: // stateT state; exposition only }; 4.15 basic_filebuf template > class basic_filebuf : public basic_convbuf { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_filebuf (); virtual ~basic_filebuf(); int is_open() const; basic_filebuf* open(const char* s, ios::openmode mode); // basic_filebuf* open(const char* s, ios::open_mode mode); basic_filebuf* close(); protected: // virtual int_type overflow(int_type c = eof()); inherited // virtual int_type pbackfail(int_type c = eof()); inherited // virtual int_type underflow(); inherited // virtual int_type uflow(); inherited // virtual streamsize xsgetn(char_type* s, streamsize n); inherited // virtual streamsize xsputn(char_type* s, streamsize n); inherited // virtual pos_type seekoff(off_type off, ios::seekdir way, // ios::openmode which = ios::in | ios::out); inherited // virtual pos_type seekpos(pos_type sp, ios::openmode which = ios::in | ios::out); inherited // virtual basic_streambuf* setbuf(char_type* s, streamsize n); inherited // virtual int sync(); inherited private: // .... exposition only }; 4.16 basic_ifstream template > class basic_ifstream : public basic_istream { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_ifstream (); basic_ifstream (const char* s, ios::openmode mode); virtual ~basic_ifstream (); basic_filebuf* rdbuf() const; bool is_open (); void open (const char* s, ios::openmode mode); // void open (const char* s, ios::open_mode mode); optional void close (); private: // filebuf fb; exposition only }; 4.17 basic_ofstream template > class basic_ofstream : public basic_ostream { public: typedef charT char_type; typedef baggage::char_bag::int_type int_type; typedef baggage::pos_bag::pos_type pos_type; typedef baggage::pos_bag::off_type off_type; int_type eof() { return baggage::char_bag::eof(); } char_type newline() { return baggage::char_bag::newline(); } public: basic_ofstream (); basic_ofstream (const char* s, ios::openmode mode); virtual ~basic_ofstream (); basic_filebuf* rdbuf() const; bool is_open (); void open (const char* s, ios::openmode mode); // void open (const char* s, ios::open_mode mode); optional void close (); private: // filebuf fb; exposition only };