From METCALF@crnvma.cern.ch Mon Nov 11 13:41:09 1991 Received: from danpost2.uni-c.dk by dkuug.dk via EUnet with SMTP (5.64+/8+bit/IDA-1.2.8) id AA18172; Mon, 11 Nov 91 13:41:09 +0100 Received: from vm.uni-c.dk by danpost2.uni-c.dk (5.65/1.34) id AA02906; Mon, 11 Nov 91 12:41:06 GMT Message-Id: <9111111241.AA02906@danpost2.uni-c.dk> Received: from vm.uni-c.dk by vm.uni-c.dk (IBM VM SMTP V2R1) with BSMTP id 0879; Mon, 11 Nov 91 13:41:30 DNT Received: from CERNVM.cern.ch by vm.uni-c.dk (Mailer R2.07) with BSMTP id 3261; Mon, 11 Nov 91 13:41:27 DNT Received: from CERNVM.CERN.CH (METCALF) by CERNVM.cern.ch (Mailer R2.08) with BSMTP id 4207; Mon, 11 Nov 91 13:31:26 SET Date: Mon, 11 Nov 91 13:30:53 SET From: Michael Metcalf Subject: Report on NAG's f90 To: sc22wg5@dkuug.dk X-Charset: ASCII X-Char-Esc: 29 EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH A First Encounter with f90 Michael Metcalf _______________ CERN, Geneva, Switzerland (metcalf@cernvm.cern.ch) Abstract A few weeks before the formal publication of the International Fortran 90 Standard, the world's first f90 compiler was announced. CERN was invited to perform beta-testing, and the opportunity was taken subsequently to assess the impact of Fortran 90 on the CERN Program Library. 1. Background Given the long development time of the Fortran 90 standard and the gloomy predictions about the availability of compilers, the viability of the language, and even the difficulty of imple- menting it fully, it was with some surprise that we learned of the announcement of a full f90 compiler by NAG (the Numerical Algorithms Group, Oxford) on 10 June of this year. This fol- lowed the completion of the standard by WG5 and X3J3 in the spring, and preceded the formal publication of the standard by ISO in August [1]. In view of the discussion currently taking place in the HEP community on the merits and demerits of various programming languages, particularly in thinking about experiments at the next generation of accelerators, it was clearly useful to CERN as well as to NAG to accept the invitation to participate in the beta-tests. The beta-test served a number of purposes: * familiarization with the language and the compiler; * an assessment of the impact of f90 on the CERN Program Library in terms of the effort required to implement it; * stress testing; * an evaluation of the compiler itself. Stress testing was an attempt to break the compiler by present- ing it with large files to compile. The assumption here was that NAG itself would perform tests in-house on large numbers of individual routines, but HEP requires successful compilation of large files, some of them containing what is often less than 'clean' code. The test was successful on all counts: by the time it had ended most small test programs had been got to work, KERNNUM and JETSET had been installed and tested, and large amounts of other Library code had been successfully compiled. Following the arrival of Release 1.0a(129) at CERN, further tests were completed, and these are reported below. All the tests were performed on an HP/Apollo 9000/425 with 12Mbytes of memory under Domain OS 10.3 and comparisons are with ftn Rev. 10.85(260) with optimization. 2. The NAG compiler The compiler itself is available on a number of unix platforms, and is invoked simply by typing f90 name for a file in the new free source form, or f90 name.f for an existing fixed form program. This is followed by the usual a.out command for execution. Thus, its use is simple and straightforward. Various options, in particular -O for opti- mization and -c to skip linking, are available. The compiler uses C as an intermediate language, and thus relies on the native C compiler to produce assembler and then object code. There is an obvious optimization penalty involved here, and another object of the test was to determine its mag- nitude. The compiler performs six passes: i. lexical and syntactic analysis, building the symbol table and an abstract syntax tree; ii. semantic analysis, annotating the parse tree and filling in the symbol table; iii. code generation by parse tree transformation; iv. output of C source code from the flattened parse tree; v. compilation using the native C compiler; vi. linking using ld and the f90 run-time libraries supplied by NAG. Each pass is distinct from the others in order "to improve maintainability and to allow the re-use of components". This is an important statement: it opens up the possibility of hardware vendors attaching the NAG front-end to a native back-end, and marketing the resulting product as a native compiler. In the HEP environment, where huge programs are ported to many platforms in many countries, the idea of having a common front end to all Fortran compilers is very attractive and one we should consider encouraging. It would be a significant simplification for the providers of libraries and for those responsible for our major production programs, and fits in well with the unix concept of portability. Also, it would overcome the fact that, by its very nature, the NAG com- piler can never be as highly optimized as a native one and, therefore, is not by itself suitable for large-scale production. The compiler performs extensive checks. On the first pass it makes a complete check of the syntax, and issues warnings and error messages as problems are detected. The second pass starts only if no errors are detected on the first. Here the semantics are checked, and the compiler is particularly good, but not perfect, at detecting such things as variables which are used but not defined, inconsistent use of arguments, and breaches of the typing rules. It sometimes takes a while to get through this pass successfully, but the reward is then a program with fewer errors than is usual with some compilers, and if inter- face blocks are also made available, then this is even more the case. The compiler performs a level of error checking often only achievable by the use of additional tools. All diagnostics are issued with respect to the Fortran file line numbers, even if issued during the C pass. This is possi- ble as the original names and line numbers are passed on to the C step using the #line directive to ccp. This makes the use of a debugger possible, but I never had to resort to the use of one during these tests. However, to test the principle, I introduced a deliberate error at line 22 of the following code, changing a .LT. to .GT., leading to the extraction of the square root of a negative number. 22 IF (D.GT.0.) THEN 23 COMP = CMPLX(-B/(2.*A), SQRT(-D)/(2.*A)) : 27 ELSE 28 SQRTD = SQRT(D) 29 REAL1 = (-B + SQRTD)/(2.*A) 30 REAL2 = (-B - SQRTD)/(2.*A) : 32 ENDIF I abandoned my attempt to use dbx after it had crashed OS on a first attempt then, after it had identified successfully a floating point exception at line 29 with ftn, it had finally given a segmentation fault running the f90 test. Turning to dde instead, this too identified the exception at line 29 for ftn. With f90, it identified an exception in the square-root function, and by invoking the trace option it told me the sqrt in question had been called at line 28. For information, the C code of the snippet above looks thus: # line 22 "debug.f" if (d_>0.) { # line 23 "debug.f" tmp3.im = sqrt_r((tmp2 = -d_ , &tmp2))/(2.*a_); # line 23 "debug.f" tmp3.re = -b_/(2.*a_); # line 23 "debug.f" comp_ = tmp3; ... # line 27 "debug.f" } else { # line 28 "debug.f" sqrtd_ = sqrt_r(&d_); # line 29 "debug.f" real1_ = (-b_ + sqrtd_)/(2.*a_); # line 30 "debug.f" real2_ = (-b_ - sqrtd_)/(2.*a_); ... The size of the compiler on the test platform was 1.2Mbytes. On the Apollo, the -frnd option is taken by default. 3. The beta-test The beta-testing began in early June and proceeded, part-time, for about four months. CERN was supplied with an advance copy of the compiler, and the first trials were sim- ply to get small tests programs, a 1500-line source form conversion program, and some parts of the CERN Program Library to compile and execute correctly. As problems were found, they were reported to NAG by e-mail and typically corrected within 24 hours. A new tape was sent by express mail about every two weeks. The collaboration with NAG at this stage was excellent, enabling difficulties to be resolved quickly and efficiently. By the end of the beta-test, all test pro- grams were working, apart from: * a test which works but should fail, because the INTENT attribute is not yet implemented (planned for Release 2). In addition, two further errors were detected after the test: * A segmentation fault when raising an integer to a double-precision power (but a correction to the .h file was supplied by return when the error was reported). * In a construct whereby part of an array is initialized by DATA statement operating through an EQUIVALENCE statement, the array is initialized from the first array element, even if the EQUIVALENCE statement explicitly references an ele- ment other than the first. 4. Source form Fortran 90 allows the use of a new source form not tied to the rigid notion of fixed positions on a line. In order to facili- tate the conversion of existing code to the new form, a tool has been written. Part of the beta-test was to get this program to work, and to check that it produced valid output by running its own output through the f90 compiler. The program exits in two forms, one using ANSI FORTRAN 77, the other using the new source form itself and having all 'deprecated' features [2] replaced by their Fortran 90 equivalents. It uses most of the static, non-array statements of the new standard, and thus is itself a quick test of a compiler. Both versions were made to work correctly. We note, however, that, as is so often the case with early versions of Fortran compilers, opti- mization of character handling is not considered to be impor- tant, and the program runs four times slower than under ftn. NAG claims this will be improved with Release 2. The compila- tion time itself was 44.6 s, 1.7 times longer than with ftn. Apart from the conversion, the program can: * ensure that blanks are used correctly in the code (they are significant with the new source form); * indent DO-loops and IF-blocks; * replace CONTINUE by END DO, where appropriate; * add subprogram names to END statements; * change non-standard length specification syntax, like INTEGER*2, by the Fortran 90 equivalent, in all contexts (but the value itself must often be changed by a further global edit, see below); * produce interface blocks automatically - useful when wish- ing to use this important new facility with existing libraries. 5. Required code transformations During the course of testing, it was established that a limited number of changes to the codes was necessary. They mostly con- cern features that one might describe as being on the edge or beyond the old standard. With one exception they were trivial to implement. 5.1 Non-standard length specifications _____________________________________________ The programs contained a small but significant number of non-standard length specifications: INTEGER*2 REAL*8 COMPLEX*16 Under Fortran 90, there is a standard syntax for the (non-standardized) length specifier values. Under the NAG compiler the statements must become: INTEGER(2) REAL(2) COMPLEX(2) The conversion program described above was sometimes used to perform the syntactical transformation; the actual value was then changed by a global edit, where necessary. 5.2 'Generic' subprograms _________________________ The Program Library contains a number of entries each of which accepts arguments of various types, for instance to copy one array to another regardless of the types of the arrays. Where two or more inconsistent references to such an entry are detected in a given subprogram, the compiler signals an error, and there are two solutions available: i. if there are only a few such occurrences, the references can be replaced by array assignments; ii. if there are many, the subprograms in question should have a USE statement to a module added, for instance use kerngen where the module contains the appropriate interface blocks to ensure that the correct specific names are used: module kerngen interface ucopy module procedure ucopy_rr, ucopy_ii end interface contains subroutine ucopy_rr(a, b, n) real :: a(*), b(*) call generic_ucopy(a, b, n) end subroutine ucopy_rr subroutine ucopy_ii(i, j, n) integer :: i(*), j(*) call generic_ucopy(i, j, n) end subroutine ucopy_ii end module kerngen and they in turn call a generic name that has to be added to the library code as an ENTRY point: subroutine ucopy(a, b, n) entry generic_ucopy(a, b, n) real a(*), b(*) b(:n) = a(:n) end The module and library code are prepared by the library writers, the user needing to add just one line to the call- ing routines. 5.3 Hexadecimal constants _________________________ Some occurrences of hexadecimal constants in DATA statements were discovered. They are trivially converted to the now standard syntax Z'dddddddd'. 5.4 Hollerith constants _______________________ A quite frequent problem was the widespread use of Hollerith constants in DATA statements. This was a real nuisance to con- vert. The proper solution is to replace their use by CHARACTER constants, however, in order to get the evaluation completed as quickly as possible, the constants were replaced by references to the transfer function. An example is to replace DATA I/4HABCD/ : WRITE (6, '(A4)') I by INTEGER :: I = transfer('ABCD', 0) : WRITE(6,'(A4)') transfer(I, '1234') ! The second argument ! fixes the type (and ! length) of the result. I recommend the elimination of as much old code containing heavy use of Hollerith constants as possible, before conver- sion. (Note that the Hollerith edit descriptor in FORMAT state- ments is still accepted.) 5.5 Typing functions ____________________ A small problem that arises, not from the standard but from the NAG implementation which goes through C, is the need to type some function names that are referenced in EXTERNAL statements. Whenever an EXTERNAL statement is encountered, it is assumed that it is a reference to a subroutine unless its clearly not so due to presence of a function reference (a message to this effect is given). Where an external function name appears only as an actual argument this assumption is wrong and will be sig- nalled as wrong in the C pass. To avoid the problem, a type statement is sufficient, as this indicates unambiguously to the compiler that the reference is to a function: EXTERNAL FUNC REAL FUNC NAG plans to remove this limitatuon in Release 2. 5.6 Blanks in literal constants _______________________________ There were some occurrences of blanks in literal constants, e.g. DATA C(1) / 8.33333 33333 33333 33D-2/ This was never seen as a problem, however, as they occurred only in code which was anyway passed through the source form converter, which removes them. They are, of course, still allowed with the old source form. 5.7 SAVE attribute __________________ It has been a standard practice to install Library code on the Apollo using the -save option, as it was known that to do oth- erwise caused problems and there had never been time to inves- tigate them. I chose not to do this (there is anyway no such option in the NAG compiler), preferring to sort it out. The fact that all entities in DATA statements automatically acquire the SAVE attribute under Fortran 90 solves most such diffi- culties, and I discovered only one instance where an explicit SAVE statement had to be added. It was in a construct of the form REAL A(100) DATA K/0/ : * Define some values in A(1),..., A(K) and K itself : On a second call to the routine, the previously defined value of K was still available, but the values in A were gone. It was sufficient to give A the SAVE attribute. 5.8 DCMPLX __________ Some of the code uses the double-precision complex, non-standard intrinsic function DCMPLX. This can be replaced directly by the appropriate reference to CMPLX using the new optional third argument to specify the kind of the result. Where it is used extensively, this is most easily done by adding a statement function as in complex(2) dcmplx real(2) arg1, arg2 dcmplx(arg1, arg2) = cmplx(arg1, arg2, 2) For better portability, named constants should be used to store the kind type parameter values, here 2 for both types. 5.9 DREAL and DIMAG ___________________ Similarly, the double-precision, non-standard intrinsic functions DREAL and DIMAG can be replaced directly by references to REAL, with the new, optional second argument to specify the kind of the result, and to AIMAG. This may again be achieved using statement functions: complex(2) arg real(2) dreal, dimag dreal(arg) = real(arg, 2) dimag(arg) = aimag(arg) 5.10 Assumed-length characters _____________________________________ There were some instances of code specifying a fixed length (99) for character dummy arguments. Where shorter actual argu- ments were passed at run time, an error message was (correctly) given. The obvious cure was to correct the code with a *(*) assumed-length specification. 5.11 EQUIVALENCE restrictions _____________________________ A point not found during this evaluation, but detected by another reviewer at NASA [3], was that Fortran 90 does not allow the equivalence of short integers to full-word integers. Once again, the transfer function can be used to cir- cumvent this problem. 5.12 Interface blocks _____________________ During the automatic generation of interface blocks, in which no COMMON blocks or PARAMETER statements are copied, it hap- pens, rarely, that part of a specification is missing, or that an incomplete part of the specification of a local variable is copied. In SUBROUTINE SUB(X) REAL X(N), LOCAL(M) END the specifications of M and N are missing. The repair is triv- ial (add the statement defining N, and remove LOCAL complete- ly). The compiler detects this problem. Similarly, the program can spuriously copy the CHARACTER declaration of an assumed-length character symbolic constant, as well as BLOCK DATA declarations. These are all detected by f90 and have to be deleted. Solving all these in the program itself would go beyond its current level of sophistication, and was not judged worthwhile. 6. The evaluation 6.1 KERNNUM ___________ KERNNUM is a set of 210 subprograms, totalling 11124 lines of code. It forms the kernel of the mathematical part of the CERN Program Library. A test program of a further 10400 lines exer- cises all the entries over legal and illegal data sets. The test was rather straightforward in that only three lines of code had to be added: one type declaration of an EXTERNAL name, and two SAVE statements. The test program itself required type statements for 15 EXTERNAL names, and one occurrence of a Hol- lerith string had to be changed to CHARACTER. The compilation time of KERNNUM was 295 s, a factor 1.14 longer with f90 than with ftn. The execution time of the test program was 148 s, a factor 1.31 longer. The test program imposes stringent preci- sion requirements, necessitating the use of the -frnd option with each compiler. The test revealed a number of problems that remain to be resolved: both compilers give an illegal instruction in the test of F011, and this is assumed to be an error in the test program. The entry was removed. One routine (D209) fails an accuracy test with ftn, and two different routines (C300 and entry RMBIL in F003) fail accuracy tests with f90. A module containing interface blocks to the whole of KERNNUM was generated automatically by the conversion program and, after fixing one error (see Section 5.12), was successfully compiled and tested (this took 20 minutes from start to fin- ish). Compiling a module with a name kernnum produces a file called kernnum.mod, and if the compiler encounters a USE state- ment as in use kernnum call ranf(x) ! ranf is a function it searches automatically in that file and, in this case, detects that the function reference is incorrect. 6.2 KERNGEN ___________ KERNGEN contains 248 subroutines totalling 8392 lines. It forms the general purpose kernel of the Library - basic mathematical routines, bit, byte and character handling, and various utili- ties. It is exercised using a test program of 4450 lines. Both the library and the test program were replete with Hollerith constants in DATA statements, and these were all replaced as described in Section 5.4, no mean task. In addition, the test program required interface blocks for eight generic entries, and the corresponding ENTRY points had to be added to the library. A small number of other changes were made to the library: a few occurrences of INTEGER*2, a few INTEGER/CHARACTER equivalences, and some calls to system rou- tines - these were mainly in utilities which are system depen- dent, and which were removed for the test or replaced by C equivalents. Some other minor changes were required, including providing the Apollo's non-standard intrinsic functions rshft, lshft and and as external functions. The code took 131 s to compile and 0.8 s to execute. The execution time is 60% longer than under ftn, but in its new form the code makes a large number of references to external functions like rshift. It would be possible to replace these by references to the Fortran 90 bit intrinsics, which f90 puts in-line, if deemed necessary. 6.3 GENLIB __________ This is a library of 413 subprograms running to 31,342 lines of mainly mathematical code. A significant part of it is exercised by a 4000-line test program. A small number of conver- sions as already described were necessary, as well as five changes to prevent branches into DO-loops (by adding PRINT and STOP statements). The main change was to add state- ment functions for DCMPLX, DIAMAG and DREAL, as described in 5.8 and 5.9. The library compiled in 952 s (1.05 times longer than with ftn, but in less than half the real time), and the test ran for 42 s, a factor 1.53 longer than with ftn (all code compiled with the -frnd option, necessary to make G110 pass). Once again, a few loose ends remain to be cleared up - one routine fails under ftn (D113 in a loop), and two fail under f90 (C206 with wrong results and D113 in a loop). 6.4 FATMEN __________ The 30,000 lines of FATMEN, a distributed file and tape manage- ment system written by J. Shiers compiled, apparently suc- cessfully. Further testing awaits the provision of more library code. 6.5 GROPAL __________ In an attempt to determine how much code the compiler can han- dle in one run, a large physics analysis code kindly provided by S. O'Neale was compiled. The compiler stopped after run- ning out of memory at line 97,012. A run with more memory would either complete, hit a symbol table limit or run out of stack space (which gives a segmenation fault), depending on the machine configuration. This number seems high enough for most applications. 6.6 JETSET __________ One of the major physics codes used in the evaluation was the JETSET program kindly provided by T. Sjostrand (U. of Lund and CERN/TH). This is an event generator extensively used in simu- lations. It is a stand-alone program of 10,000 lines written in pure FORTRAN 77, but containing many complicated expressions. It was intended to make a comparison with ftn, but this proved impossible. Compiled under ftn, with or without optimization, JETSET produced very wrong results. (A test on a DN3500 indicated the problem was with the run-time library rather than the compiler itself.) It was impressive that f90 was more successful. In spite of a rather long compile time (390 s, 2.7 times longer than ftn), JETSET worked correctly, the standard test running for 71 s (a reasonable figure accord- ing to the author). No changes to the code were required. An attempt with optimization produced an error after 14 s in the SQRT function, indicating an error in the C compiler. Thus, f90 succeeded where the native ftn compiler failed and revealed a problem even in the C compiler. 7. Conclusions Taking the four objectives of the beta-test in turn, we can conclude that i. it was useful in gaining experience in real-world use of Fortran 90, even if mainly in a FORTRAN 77 con- text; ii. rather than just assessing the impact on the CERN Pro- gram Library, it was actually possible, with a reasonable effort, to convert the code of the subroutine library to run under f90; iii. the compiler can handle sufficient amounts of code in large runs; iv. the compiler itself is, at least for FORTRAN 77 code, now rather reliable. As a consequence of this, CERN decided to purchase a site-wide licence for a number of platforms. This will enable further testing and evaluation to be carried out, par- ticularly by those interested in testing out some new ideas in data structuring, but also with more immediate targets, such as seeing what the implications are for major codes such a GEANT. Finally, CERN now has some 80,000 lines of code working under a Fortran 90 compiler, and this will surely be useful in testing new compilers as they are introduced by the hardware vendors. Acknowledgements I thank Miguel Marquina for providing me with the library code and its test programs, Malcolm Cohen and Robert Iles (NAG) for their excellent support and quick responses whilst performing the tests and evaluation, and Rene Brun, Federico Carminati and David Williams for their support and interest in this project. Bibliography 1. ISO, ISO/IEC 1539 : 1991, ISO, Geneva, Switzerland. 2. M. Metcalf and J. Reid, Fortran 90 Explained, (Oxford Uni- versity Press, Oxford and New York, 1990). 3. R. Maine, Review of NAG Fortran 90, to appear in Fortran Journal, 1991.