These minutes were approved at Meeting #3.
Meeting Location:
British Standards Institution
389 Chiswick High Road,
London W4 4AL
Room 513
Tel: Int + 44 20 8996 9000
Host:
Host Contact information:
Derek Jones
Email: Derek Jones
Phone: Int + 44 1252-520667
The meeting started late, at about 9:30 am, because several of the members were late due to transport delays. Not all attendees were present when the meeting commenced.
Due to the late start, the convener refrained from making comments.
The attendees introduced themselves. (This list includes those who arrived after the meeting commenced.):
It was noted that Erhard Ploedereder no longer represents the German national body, but instead serves as a liaison from Ada-Europe.
It was noted that Brian Wichmann, Rod Chapman, and Robert Seacord sent regrets that they were unable to attend the meeting.
The convener proposed that John Benito serve as meeting chair. There was no objection and the appointment was approved.
The meeting chair proposed that Jim Moore serve as meeting secretary. There was no objection and the appointment was approved.
John Benito provided a quick outline of meeting procedures. He noted that the group permits anyone to speak and that straw polls may record individual positions. Of course, we must be aware that decisions are made by voting that occurs in our parent organization, SC 22.
Jim Moore reviewed the action item log. The meeting agreed on some changes to the log:
#01-05: Clive Pygott has been appointed by MISRA C++ to serve as the liaison to OWGV. Clive will forward the note that appointed him. This action item is closed.
#01-08 and #01-11: These two items concern the recruiting of liaison representatives. Although the responsible individuals had performed the requested action, the desired results have not yet been achieved. So it was decided to carry these as open items. In short, we await response from WG21 (C++) and the ECMA committee on Eiffel.
#1-13: We are still attempting to recruit participation from an appropriate body in the Java committee.
Some action items were closed as a result of activity later in the meeting. These include #01-09, #01-10, and #01-16. Also items number #02-xx were added to the log as a result of the meeting.
The proposed meeting agenda [N0029] was considered. It was decided to add MISRA C to the list of liaison reports (MISRA C and MISRA C++ would be items 2.7.1 and 2.7.2 respectively). It was also decided to entertain a presentation by Paul Caseley of the UK MOD on Friday morning. Administratively, this presentation would be treated as item 9 on the agenda and adjournment changed to be item 10.
The draft minutes of meeting #1 [N0025] were considered. One correction was made:
With this correction, the minutes were approved. The corrected minutes will be posted to the log as [N0025r].
The chairman noted that several documents had been submitted for consideration at this meeting. Each will be considered at an appropriate time in the meeting:
1.9.1 Moore's draft 1 response to Action Item 01-10, "Vulnerability Descriptions" [N0030]
1.9.2 Seacord's draft response to Action Item 01-09, "Levels" [N0031: txt, jpg]
1.9.3 Wichmann's paper on "Predictable Execution" [N0032]
1.9.4 Jones's paper on "Culture and Formal Education Issues" [N0033]
1.9.5 UK revision of proposed base document [N0034]
The next meeting, Meeting #3, will be at INCITS, Washington, DC, for three days, 11-13 December 2006, hosted by John Benito and INCITS. We discussed strategies for subsequent meetings. Tentatively, Meeting #4 will be held April 30-May 2 at Kingston University, Kingston-on-Thames, UK, hosted by Les Hatton and Derek Jones. (There was a suggestion that 30 April might be a bank holiday but Derek Jones subsequently decided that it was not.
We decided that it would be desirable to have Meeting #5 in July 2007 and Meeting #6 in October 2007.
John Benito took Action Item #02-01 to contact SC22 representatives who voted for the work item and ask them if they will host a meeting. The goal is to increase participation from national bodies other than the US and the UK. Canada offered to host Meeting #5 in Ottawa if there are no other volunteers. Erhard Ploedereder offered to host a meeting in Stuttgart. John Benito said that the Netherlands may be interested in hosting a meeting.
INCITS J3 believes that the OWGV is an important effort. They probably will not be able to implement any results in the Fortran 2008 (they are planning to go into CD next summer) but would aim for later revisions. Nagle believes that J3 might react well if OWGV describes generic problems and J3 has the opportunity to determine how to apply the generic descriptions to Fortran.
As a result of discussion we reached Tentative Decision #02-01 that the OWGV Technical Report should be organized around generic descriptions of weaknesses and that annexes would provide the language-dependent descriptions of the weaknesses. Our liaison representatives would ensure that the language committees provide appropriate material to be used in the annexes.
We also reached Tentative Decision #02-02 that we should ask the language committees for a list of all "processor-dependent" or "implementation-dependent" features of their languages.
No report was received.
WG9 has completed technical work on the amendment to the Ada standard that defines "Ada 2005"; the amendment is about to go to JTC 1 for FDAM ballot. WG9 has a sub-group, the "Annex H Rapporteur Group", that works on issues related to safety and reliability. WG9 is interested in providing material to the OWGV once the interests have been specified. In general, it will produce a better report if working groups provide their own responses to the generic problems specified by the OWGV.
The C working group has not met since the previous meeting of OWGV, so there is nothing further to report.
Note: This is only a placeholder in the agenda; liaison has not yet been established.
No report was received.
They are currently working on a Technical Corrigendum to the MISRA C document as well as a test suite. Both should be completed by the end of this year. A proposal for a third edition of MISRA C is currently being discussed. Hacking is becoming a serious problem in automotive systems. So the work of OWGV is of interest to MISRA and improvements to address vulnerabilities identified by OWGV might find their way into the proposed third edition. This work might also pull along some tool suppliers.
Clive made a presentation, "Objectives of Coding Standards and MISRA C++" [N0038], reporting on the Defence Aerospace Research Partnership C++ workshop held in April 2006, and mentioned that additional material was available in a second document [N0039]. Participants were asked, "what do you want to see in a generic software vulnerabilities standard?" Their responses included:
The workshop summarized the "reasons for coding standards" as:
During the OWGV discussion of the report, it was agreed that "tractability for analysis" should have been added to the list.
One of the sources cited for the MISRA C++ report is a QinetiQ report on vulnerabilities that collected implementation defined items from ISO standards. John Benito asked Clive Pygott to make the report available to OWGV. Clive accepted that as Action Item #02-02. (Shortly after the meeting, this action item was completed; the document is [N0043]).
The report prompted some discussion of the criteria for guidance. It was agreed that we prefer guidance based on evidence. It should be acceptable to provide rules based on preponderance of expert belief, but the rationale should explain the basis for belief so that readers can determine if the rule is applicable to their situation. This was recorded as Tentative Decision #02-03. Olwen Morgan said, "In an industrial-strength environment rules must be machine-verifiable because of the volume of code that is involved."
No report was received.
No report was received.
No report was received.
There were no other liaison reports.
At this point in the agenda, we recessed for lunch. In the afternoon, we resumed with agenda item 3.
John Benito reviewed the group's annual report [N0026] to SC22 (which was also published by SC22 as [N4078]). After a brief discussion, the participants agreed on the following as Decision #02-04: OWGV will recommend to SC 22 that OWGV should be continued for another year with John Benito as convener and Jim Moore as secretariat.
With no draft of the TR yet in place, the participants agreed to move to discuss documents submitted by participants.
We first considered Brian Wichmann's paper on "Predictable Execution" [N0032]. In Wichmann's absence, the paper was presented by Clive Pygott.
Sometimes a language standard provides insufficient information to predict execution. Sometimes the information that is provided is unhelpful; for example, in the case of arithmetic overflow, the Ada standard says "raise an exception". So one might have guidelines that aim to restore predictability. Olwen Morgan said that any claim of predictability beyond "stepwise predictability" is stymied by the halting problem. Jim Moore suggested that we may not need full mathematical predictability, but only the notion of predictability to inform the phrasing of guidelines. Erhard Ploedereder proposed that "reliability" is the goal. This would be an ideal and programmers would take various measures to more closely approach the ideal. Nagle said that predictability means that the execution of the software does not violate the model understood by the reasonably competent programmer. Wichmann's paper said that if you use floating point, you may need numerical analysis expertise to deal with issues of numeric stability. Section 3 of the paper suggested a possible set of categories for problem areas in language specification. This would be a starting point, not an exhaustive list. (For example, Wichmann doesn't deal with temporal characteristics at all.) There are possibly other lists of mistakes that programmers make in using the languages, including uninitialized variables, buffer overruns, etc. As a result of discussion, we reach some tentative decisions:
Decision #02-05: OWGV will view "predictability" not as a mathematically provable property but as an ideal that we hope to approach. We would apply the test that a piece of code is not "predictable" if it is possible for it to violate the model of execution understood by a "reasonably competent programmer".
Decision #02-06: Step-wise predictability is a useful criterion for providing advice, at least when considering serial execution. (Note that some non-determinism is permitted.)
Decision #02-07: View floating point as "out of bounds" at least for starters.
Decision #02-08: Use Wichmann's section 3 of [N0032] as a starting point for a categorization of problem areas in language description (but not language usage).
In dealing with the halting problem, for some levels of integrity, one might insist that any specific program must be analyzed as predictable in order to be acceptable. Non-analyzable programs would simply not be allowed.
We next considered Jim Moore's draft [N0030] of a response to Action Item 01-10. This paper was to have been a collaboration among Moore, Derek Jones and Larry Wagoner. Because of timing and email problems, though, they were unable to resolve differences prior to the meeting. It was decided that Moore's draft and their concerns would be openly discussed at the meeting. In the absence of Wagoner, John Benito represented the position described in his email comments.
Derek Jones said that he generally agrees with the proposal, but has some detailed concerns. Larry Wagoner wants a dedicated section in the technical report for each language. Chris Hills said that the master document should be generic and should refer to Annexes that contain language-specific guidance. Olwen Morgan suggested that the language-specific material could be copied from the body into the annex. Hills responded that there could be references instead. Clive Pygott said that the annexes would expand the material from the generic section.
There was some discussion of classifying the vulnerabilities and some discussion of distinctions among "(taxonomical) classification", "categorization", and "codification". Jim Moore said that there are thousands of possible weaknesses and they need to be classified so that readers can make some sense of them. Derek Jones suggested that we might order them by importance and that we might choose to pick the top 100. He went on to suggest that we might only have time to produce wording (over the coming year or two) for 20 or so generic weaknesses (which might expand to many recommendations for a particular language).
As a result of discussion, the group reached a number of decisions"
Decision #02-09: Use a faceted classification scheme as a method of organizing the vulnerabilities.
Decision #02-10: The reference in [N0030] to "dialects" risks leading us into vendor-specific issues. Remove the reference to dialects.
Decision #02-11: Remove the section on profiles from [N0030]. Instead, give advice on writing profiles. Profiles might arise from the faceted classification scheme.
These decisions will be implemented in an initial draft document to be prepared by John Benito and Jim Moore. (See below.)
Robert Seacord and Tom Plum were given Action Item 01-09 to propose a set of levels. In correspondence, Plum suggested that some other concepts need to be agreed upon, such as critical vulnerability, before a scheme of levels could be developed. Seacord submitted a chart describing the CERT approach [N0031: txt, jpg]. In Seacord's absence, OWGV discussed the suggestion described in his email note. The action item was closed.
Of the three factors he considers in computing a level, OWGV agrees that consequence and cost of remediation are important. In the technical report, we would treat these issues but we don't agree that they should be reduced to numerical values. Instead, we would treat them in a narrative fashion. With respect to the proposed factor of likelihood, the OWGV believes that exploitable vulnerabilities should be regarded as having a likelihood of 100%; so this factor would not be useful.
This was recorded as tentative Decision #02-12: Consequence and cost of remediation are important in discussion of "levels". In the technical report, we would treat these issues but not in a numerical fashion. Instead, we would treat them in a narrative fashion. With respect to the possible factor of likelihood, exploitable vulnerabilities should be regarded as having a likelihood of 100%.
[Upon reading the first draft of these minutes, Seacord responded that the group may have misunderstood his intent with regard to the factor of likelihood. He wrote:
Unfortunately much of this goes to definitions that we may or may not have agreed upon yet. I am still using the word vulnerability to mean a set of conditions that can allow for a successful exploit. This normally means that a security "flaw" must be combined with other factors to become a software vulnerability. For example, an unbounded string copy while clearly a flaw will not be exploitable unless an attacker can control (or at least influence) the contents of the string being copied.
The "likelihood" of the metric means, how likely is it that a security flaw would result in a vulnerability? Freeing memory multiple times in C, for example, is definitely a flaw, but can usually only be exploited under very narrow conditions--so the likelihood of this becoming a vulnerability is low.
As always, decisions can be reconsidered. -- JWM]
Derek Jones presented document [N0034], the revision of the UK's previous proposal for a base document. During discussion, the following changes were agreed:
Section 2.1 described the scope: "Applicable to any computer programming language. Applicable to software written, reviewed and maintained for any application. Applicable in any context where assured behaviour is required, e.g. security, safety, mission/business criticality etc." It was agreed that our scope is not all languages; it is the languages which we actually treat. The word "any" should be removed.
Section 2.2 described what is out of scope: "This technical report does not address software engineering and management issues such as how to design and implement programs, using configuration management, managerial processes etc. The specification of the application is not within the scope." It was decided that "software engineering and management issues" should be related to the terminology and concepts of JTC 1/SC 7. OWGV should strive to use the same term when it means the same thing and a different term when it does not.
Section 2.3 described a "cautious approach":
The impact of the guidelines in this technical report are likely to be highly leveraged in that they are likely to affect many times more people than the number that worked on them. This leverage means that these guidelines have the potential to make large savings, for a small cost, or to generate large unnecessary costs, for little benefit. ... For these reasons this technical report has taken a cautious approach to creating guideline recommendations. New guideline recommendations can be added over time, as practical experience and experimental evidence is accumulated.
It was decided that the OWGV technical report should introduce issues in an informative manner that other documents may address in a prescriptive fashion. The OWGV will attempt to take an evidence-based approach, but it understands that sometimes the only available evidence is a preponderance of expert opinion. However, the mere fact that some naked guideline exists unexplained in various documents is not sufficient grounds to include it – equally unexplained – in our document.
In section 3.1, to the list of "knowledge failures", the meeting decided to add the cost of obtaining the necessary knowledge.
In section 3.2, it was decided to remove the first paragraph. In the second paragraph, instead of "provide guidelines", the text should read "identify issues". The last bullet item in the list should address the use of tools rather than the production of tools. That issue should be addressed in a distinct paragraph suggesting that vendors may rally around the document, and make tools more readily available and economical.
It was agreed to remove 3.2.2.
It was agreed that 3.3 should be rewritten to address this intention: When you work on a different implementation, some of the things that you thought were true, are not true anymore. We can rephrase this to talk about source code portability across various implementations. Move the rationale about people portability into the intended usage of the guide. We might also discuss the issue that one should routinely use a variety of compilers and implementations.
Erhard Ploedereder said that there are generic issues with languages. We should include a general discussion of portions of languages that are likely to get us into problems – like exception handling and a single integer type. This should be included in the base document.
During the discussion, there was some disagreement among the UK delegation and the OWGV agreed that we would use the document as a basis of discussion rather than as a firm UK position.
At this point in the discussion, we recessed and then resumed on Friday morning by discussion Section 4 of the UK proposed base document.
Section 4, 2nd paragraph: We decided to change "unpredictable" to "unintended". Also change "prediction", etc.
The meeting decided to remove sections 5.1 and 5.2; "conformance" is not appropriate in a type 3 TR.
There was extended discussion of sections 5, 6, and 7 of the proposed document. Ultimately, it was agreed that describing vulnerabilities (as described in the New Work Item Proposal) is distinct from guidance on how to produce language-specific guidelines (as suggested by the UK proposal). The latter can be regarded as a future work item. If material appropriate to this is produced, we will save it for potential future work items. (This was logged as decision #02-14.) So, sections 5, 6, and 7 should be set aside for potential work on future items. Also, section B should be set aside for the same purpose. Finally, Section D--guidelines considered but rejected--should be incorporated in a Rationale section. (This was logged as decision #02-15.)
These decisions have been summarized as Decision #02-13. These decisions will be implemented in an initial draft document to be prepared by John Benito and Jim Moore. (See below.)
Steve Michell suggested that the TR should provide advice on how it should be applied. He suggested that TR 15942 might be the language-specific annex for Ada. There was some disagreement. Michell took Action Item #02-03, "Make a contribution describing how to apply TR 15942 in the work of the OWGV."
At this point in Friday's meeting, we performed agenda item 9, out of order; it is reported below.
At this point, we recessed for lunch and resumed with the following item ...
Derek Jones reviewed his submission [N0033]. In discussing the paper, the group reached some decisions about items that might be included in the OWGV technical report:
These were logged as tentative decisions #02-16 and #02-17.
We had a brief discussion of how to make progress on the work between meetings of OWGV. It was the consensus that discussion on the mailer will go better if we operate within the framework provided by a draft technical report. Accordingly, an action item was assigned:
Action Item #02-04 [Benito and Moore]: Synthesize a document outline from Moore's proposal, as amended by the meeting, and the UK base document, as amended by the meeting. Prepare the outline in time for distribution via the post-meeting package.
The meeting participants agreed that it was not necessary for the document to be perfected, but merely "good enough" to use as a framework for progress in the near term. The meeting secretary reserved document number [N0040] for the draft TR.
The meeting then discussed possible sources for vulnerabilities to treat within the draft TR. Several persons volunteered to provide items:
In addition, Chris Hills took an action item:
The group reviewed the prospects for future meetings. Erhard Ploedereder had already volunteered Stuttgart,Germany, and Steve Michell, Ottawa, Canada. It was agreed that it may be helpful in recruiting national body participation if we agree to hold meetings in nations who might be interested in participating. It was agreed that alternative hosting might be acceptable beginning as early as Meeting #4, April 2007, which is currently tentatively scheduled for the UK. Two action items were accepted:
No business to discuss.
The meeting set October 6 as the date for the post-meeting mailing and November 20 as the date for the pre-meeting mailing.
Jim Moore did a quick review of decisions reached during the meeting. They are recorded in the decision log.
There are no formal resolutions. It was suggested that this should be removed from the agenda of future meetings.
Jim Moore did a quick review of action items accepted during the meeting. They are recorded in the action item log.
In behalf of the group, the meeting chairman John Benito expressed appreciation to Derek Jones and BSI for hosting the meeting. He also thanked Dinkumware for providing the meeting Wiki.
This agenda item was taken out of order and was actually performed on Friday morning.
Paul Caseley made a presentation that is not yet available to us. He will eventually send us a cleared version (Action Item #02-12). Number [N0041] has been reserved in the document log.
The meeting was adjourned at approximately 4:30 pm.
