From owner-sc22wg5@open-std.org Tue Nov 11 10:20:10 2008 Return-Path: X-Original-To: sc22wg5-dom7 Delivered-To: sc22wg5-dom7@www2.open-std.org Received: by www2.open-std.org (Postfix, from userid 521) id 5F37EC178DC; Tue, 11 Nov 2008 10:20:10 +0100 (CET) X-Original-To: sc22wg5@open-std.org Delivered-To: sc22wg5@open-std.org Received: from ppsw-6.csi.cam.ac.uk (ppsw-6.csi.cam.ac.uk [131.111.8.136]) by www2.open-std.org (Postfix) with ESMTP id 8D40CC178D9 for ; Tue, 11 Nov 2008 10:20:08 +0100 (CET) X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-1.csi.cam.ac.uk ([131.111.8.51]:58519) by ppsw-6.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:25) with esmtpa (EXTERNAL:nmm1) id 1KzpPs-0004iN-Jf (Exim 4.70) for sc22wg5@open-std.org (return-path ); Tue, 11 Nov 2008 09:20:08 +0000 Received: from prayer by hermes-1.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:nmm1) id 1KzpPs-00074P-2h (Exim 4.67) for sc22wg5@open-std.org (return-path ); Tue, 11 Nov 2008 09:20:08 +0000 Received: from [83.67.89.123] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.1); 11 Nov 2008 09:20:07 +0000 Date: 11 Nov 2008 09:20:07 +0000 From: "N.M. Maclaren" To: WG5 Subject: Re: [ukfortran] (SC22WG5.3655) (j3.2006) FW: Notes on WG5 coarray papers Message-ID: In-Reply-To: <20081111083358.6613AC178D9@www2.open-std.org> References: <49137AD3.1070402@lrz.de> <20081110124449.AEBF1C178E0@www2.open-std.org> <20081111030258.C9550C178D9@www2.open-std.org> <49193B79.8030807@lrz.de> <20081111083358.6613AC178D9@www2.open-std.org> X-Mailer: Prayer v1.3.1 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Sender: owner-sc22wg5@open-std.org Precedence: bulk This isn't a full response, and doesn't represent the BSI, but I am sending my initial reactions to a selection of points; Most of them would be easier to discuss face-to-face, as they are not simple. > 1.2 VOLATILE coarrays (8.5.1, para6): > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > ---------------------------------------------------------------------------- > NOTE 8.29a A scalar coarray that is volatile and of type default integer, > default real, or default logical is 'atomic' in the sense that read > accesses from any other than those altering its value will obtain either > the value previous to an alteration, or the value after an alteration. It > remains the programmer's responsibility to prevent race conditions for > such volatile coarray references by suitable formulation of the algorithm > (ref. to example in Note 8.38). > ---------------------------------------------------------------------------- This NOTE goes to the heart of the problem. There are two separate concepts involved, corresponding to your two sentences: Whether accesses are atomic. The current wording isn't enough, because arrays are not a separate type from their elements in Fortran (unlike modern languages). However, that's the 'easy' one :-( Whether volatile coarray accesses cause any ordering, beyond that caused by normal object accesses and image control statements. And that's the killer. Roughly, in current Fortran, accesses are grouped into totally ordered sets, in the mathematical sense. End of story. The sets are those that have no ordering defined by the language (e.g. those within an expression). That fails as soon as we introduce parallelism, where there can be at most a partial ordering and more usually only a pre-ordering. Fortran COULD say that the behaviour is AS IF there is single execution order, but that is sequential consistency. That is foul to implement on most systems and inefficient on all large ones, for non-handshaking accesses like VOLATILE coarrays. Neither Intel nor AMD guarantee it even at the hardware level, so it wouldn't be reliable even on them unless special action was taken. And, currently, it doesn't. It doesn't say anything. > In any case, we have here an extension of the semantics of VOLATILE for > the indicated coarray objects, as compared to the original definition in > 1.1. This additional property is needed for the code in Note 8.38 to > work. Note that formally the VOLATILE attribute is only exploited on > image Q in that example. Due to the additional semantics, a reliable > implementation for commodity clusters will probably incur an even larger > overhead than normal VOLATILE objects. Yes, I agree - with one exception. Where does it say that in normative wording? > 2. Comments on paper N1745: > =========================== > > In section 1, the author advocates three possibilities for a better > approach. ... > > The second one, atomic datatypes and operations, is in effect what is > already there (see 1.2 above). ... The issue is not the atomicity but the ordering, and Fortran defines ordering only for statements, not accesses within statements. [ Please let's not start the impure function 'debate' yet again! ] > The VOLATILE attribute would then still be advantageous since it > provides the effect of an object-specific SYNC MEMORY, saving on overhead > if many other memory operations are outstanding (an additional SYNC > MEMORY would otherwise be required within the spin loop of Note 8.38!). Grrk. As they necessarily occur within an expression, that implies an ordering of expression evaluation and object access within it, which Fortran doesn't have, in general! The point about a proper atomic interface is that it would use subroutines, which can be invoked only as complete statements, and hence are ordered. > In section 2, some examples are provided to illustrate inconsistencies > in the standard. As Malcolm says, arrays are just collections of scalars. It would be trivial to rewrite most of my array-using examples with purely scalar access, and one of the facilities that don't imply ordering, such as DO CONCURRENT. > Example 2.4 ("Protected Context") is non-conforming since the cited > restriction is violated by the object in question being VOLATILE within > the scope of the DO loop. This would apply even if the CASE(1) statements > were not present. Malcolm has addressed this, but there is an extra wrinkle. Is there a difference in meaning between "be redefined" and "become redefined"? In formal English, there is - but, in the Fortran standard? It makes a BIG difference for VOLATILE. > Finally, the reference to the C++ standardization efforts with respect > to the memory model (actually at the end of N1744) appears to be relevant > for shared memory processing, but not necessarily for segmented memory > processing. I can assure you that it is! > 3. Concerning Paper N1744: > ========================== > > Section 1 (Sequential Consistency): > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > The author seems to believe that in absence of the user defining a > suitable synchronization sequence the processor should impose one. This > appears contrary to the spirit of coarray programming which intends to > minimize the constraints as far as possible to achieve improved > performance. Absolutely NOT! My belief is that the standard should specify, clearly and explicitly, what ordering properties are defined, and what ones are not. N1744 is asking for clarification of points on which there is scope for more than one reasonable interpretation. Actually, it is YOU who are assuming it! Unless I am much mistaken, sequential consistency is mathematically equivalent to there being a partial ordering. And, if your interpretation of SYNC MEMORY is correct, then segments are partially ordered. I have no problem with this! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1@cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679