Navy Aircraft Acquisition - Source Selection

Exhibit A-1. A retype of a 1977 paper for VPI.

Note: A similar paper was written for a "corporate memory" series. (Latter probably never completed.) GAS

HISTORY

NAVY AIRCRAFT ACQUISITION PROCESS

SOURCE SELECTION

The Navy's record of success in aircraft development has been acknowledged to be superior to that of any other service or agency. Although the factors which made that record possible were well known to those involved in compiling it, the documentation of the acquisition process as actually employed for naval aircraft is somewhat sketchy, as little of it appeared in official instructions or directives through much of our history. In recent years, under pressure for standardizing all Navy and Defense procurement actions, the documentation has increased greatly, but the procedures now described differ materially from those employed so successfully by NAVAIR and its predecessor agencies, BuWeps and BuAer. This paper will examine the source selection phase of the Navy's aircraft acquisition process as employed in the past and primarily in the period from 1930 to 1970 when the Navy started over 100 designs and produced approximately 100,000 aircraft for its own use.

"Source Selection", a term initially applied to one method of selecting a contractor for a new development has become a generic term encompassing all methods of selection. Prior to World War II, design competitions were used by all services as the means by which a manufacturer was selected to receive a development contract for a new aircraft. As the name implies, in a design competition, the government selected a design which best met its objectives from a number of proposals submitted by industry. In the 1950's, the Air Force changed its procedures to select not a "design", but only a "source". This step was taken, it was said, to avoid unnecessary expenditure of technical resources by industry. Conceptually, only brief management proposals were to be required, with the major costs associated with making a design proposal deferred until after a contract was let. Not unexpectedly, management proposals proved inadequate for making justifiable selections, and eventually "source selection" proposals became equivalent to those required in a design competition. The Air Force, and later OSD, however, retained the terminology of "Source Selection" which now appears in all directives on the subject.

The Navy's aircraft selection process evolved over the years modified as necessary to suit changing time, organizations, management theories, technologies, etc., but always retaining the basic philosophy of selecting that design which was estimated by the Navy's own experts to best satisfy the fleet needs. In the early days of the Bureau of Aeronautics (and incidentally the aircraft industry), the law required, in aircraft competitions, that the government inform all competitors of the criteria and weighting factors to be employed in making a selection. It is presumed from this evidence that numerical scoring systems were in vogue during this period in which both experimental and production contracts were let in competitive bidding procedures similar to those of today's forma advertising. By the 1930's, the procedures and law changed so that design competitions were continued as the method of choice for selecting experimental models, while production contracts were negotiated with the manufacturer who had developed the prototypes. A factor in the procurement procedures of that era was that the entire contractual procedure was actually the responsibility of the Bureau of Supplies and Accounts (Bu S&A), although virtually all the contractual documents were drafted by BuAer. This led BuAer, in the interests of efficiency, to use "informal" proposal requests, competition procedures and pre-contract negotiations before getting officially involved with Bu S&A. By World War II, contracting authority was granted to the bureaus, but the "informal" approach continued for many years with little involvement of "Contract", "Legal", or "Procurement" personnel until firm decisions on contractor selection and contractual action had been made. Major system acquisitions were thus handled differently than other procurements for most of the bureau's history.

The organizational structure of BuAer was always a "functional" one with a minimum of "project" personnel. Under pressure from management theorists, a "matrix" organization has now evolved with a continuing trend toward larger numbers of management personnel and comparatively fewer functional specialists. There has been a concurrent trend toward greater reliance on field station or contract personnel to provide the technical expertise formerly resident within the basic organization. Under the old functional arrangement, there was little or no need for administrative instructions covering the normal work of the bureau, since each organizational entity's responsibilities had long been established and were regularly performed. One of the functions delegated by the Chief of the bureau was that of conducting design competitions for new aircraft and missiles to a small group once called "Design Coordination" in "Engineering" and now designated as "Evaluation" in "Material Acquisition". The responsibility for evaluating new designs was written into the duties of each of the bureau's function entities, and into virtually all of the position descriptions of individual engineers. With a single division in charge of all competitions, and with each technical division participating in the evaluation process, there was obviously no need of administrative paper work detailing a process continually being practiced. This is the explanation for the lack of documentation within the bureau's operating directives for the competition process. There was no disagreement on the procedures which were described as "well established" and were reported regularly to new officials, investigating groups, congressional committees, and of course in the memoranda which documented each competition. However, there were occasional recommendations for issuing an operating instruction from groups outside the bureau, usually as a result of management audits or surveys. Invariably such groups supported the Navy's competition procedures, but deplored the lack of a general paper detailing them. In the late 1950's, a routine revision of the general procurement regulations was issued which was then interpreted to include major system acquisitions. This step brought "Contract" personnel into the acquisition procedure from the beginning of the procurement. Until this time, "Contracts" entered the process after the competition was decided, just as "Bu S&A" did in the earlier era. It should be noted that there had been no problems experienced which indicated a need for such action. The greater formality took more time and effort, and although deplored as unnecessary, it has continued. From this point on, it became impossible to draft an operating instruction on which agreement could be reached as to the precise division of responsibilities between the various parts of the bureau organization. For example, at that time "Contracts" insisted that any directive show that contractor selection was their responsibility while in practice it had never been. In each new aircraft acquisition from then on, technical personnel continued to run the competitions in fact, although with greater and greater controls imposed by "procurement" personnel.

The reason for using design competitions in the selection process was that, as implied earlier, several manufacturers were both normally available to undertake any new development and were so nearly equally qualified that a justifiable choice between them was impossible. A design competition allowed each bidder to demonstrate his competence, and gave the government adequate information for selecting the best design and for justifying that selection to everyone, including the losing bidders. It should be obvious that this type of a competition could succeed only when the rules of the game were understood by all concerned and enforced by those running the event. To this end it was, and is, important that the weapon system be defined as well as possible so that in essence, the airframe was the principal variable. Design competitions were held, therefore, after all conceptual weapon system development had been completed, military worth established, and provision made for budget support. All Navy competitions were unfunded until OSD required compensation for the bidders in the late 1960's. When no funding is provided, marginally qualified contractors normally drop out early, while very small (and unqualified) companies are discouraged from submitting proposals and wasting their resources. When funding is provided, the entire selection process becomes more complex and consumes more time since one must first conduct a preliminary competition to limit the number of bidders, and then defining the tasks to be performed, awarding the contracts, and administering them. There is no apparent benefit to the government to offset the time and cost involved.

Returning to the competition process as actually practiced:

Initially, simple letters to our major contractors solicited their interest, and these were followed by "invitations" (not requests) to submit "informal proposals", again in letter form. Now, a forthcoming request to industry to submit proposals is first advertised in the Commerce Business Daily and is followed by an RFP.

The major ingredients of the "invitations to bid" were: a description of the intended procurement, the "schedule", a type specification for the airplane, and a specification addendum detailing the proposal data required. In the period of "cost plus" contracting, detailed definition of data and flight testing was deferred until after selection of the winning contractor. When "fixed price" contracting was employed, specification addendums covering both data and tests requirements became necessary. Specifications were drafted by a small group charged with that responsibility, but with the assistance of all functional divisions. Coordination with "Requirements" personnel was, of course, a normal routine.

During the proposal preparation period, contact with contractors was minimized. Questions from the bidders were directed to the "Evaluation" Division, who answered them immediately, if possible, or if coordination with other divisions was necessary, as soon as practicable. Clarifications and corrections to the specifications, if necessary, were then distributed to all bidders officially.

After proposal submission, the evaluation process proceeded with all the functional divisions evaluating each design and reporting back to "Evaluation" their ratings, rankings, and recommendations for necessary changes in design, data, tests, or schedule.

On the basis of the evaluated data, the least desirable designs were dropped from consideration. The remaining designs were reevaluated incorporating those changes which would be incorporated if each of the designs were to be procured. This part of the process, unilateral by the bureau, put all competitors on as near an equal basis as possible for final comparisons and selection of a winner.

The selection decision originated within the "Evaluation Division" with the active collaboration of the "Project officer" (or Class Desk). The competition results were then reduced to memorandum form in which the "Assistant Chief, R&D" recommended a course of action to the "Chief", via the other "Assistant Chiefs". In later years a Program or Project Manager, if assigned, became a collaborator in the selection decisions, all of which were reached informally by common agreement.

A final step in a design competition was to compare the winning design with other aircraft and with the characteristics which had been used in justifying the project. When the comparison was favorable to the winning design, procurement action was recommended. If the comparison was unfavorable, a rejustification analysis had to be performed.

After the decision, a procurement request started official contractual action. Specifications, contract terms and conditions were negotiated with the selected contractor. The remaining bidders were debriefed individually by the "Evaluation" Division, and given the bureau's estimates and comments on their own designs.

The Navy's practices differed from those of other services in a number of areas, some of which are discussed below:

In contrast to the Navy practice of using its entire functional organizations, the other services adopted an ad hoc team concept, sometimes situated at remote locations. The evaluation team leader was normally the "Program Manager", usually conducting his first (and only) competition and utilizing only a small fraction of the total expertise available in that service.

Numerical scoring procedures were standard in the other services with all proposals being "raw" scored in detail areas against "standards" and later "weighted" by previously established "criteria". Internally, the Air Force considered the scoring results as a useful tool in reaching a decision, but never regarded them as necessarily conclusive. Secretarial and Congressional levels often regarded the scoring results as absolute, however, as evidenced by the TFX investigation. Numerical systems were found unnecessary by the Navy, and much less efficient than simple comparison and elimination techniques which permit maximum concentration on the best designs. (With quantitative scoring systems, even the poorest designs are scored in the same detail as the best ones.)

Much less paper work was involved in the Navy system from start to finish of a competition. A one or two page memorandum to the participating divisions was the only "Source Selection Plan" required. Proposals were evaluated against specification requirements, obviating the need for preparation of generalized "Criteria" and "Standards". Division evaluation results were reported in memoranda running from a single page to an average of perhaps 10 pages. The memorandum summarizing the competition was usually less than ten pages but with enclosures which sometimes produced a ½ inch thick document. In contrast, the Air Force TFX evaluation report was 2 3/4 inches thick, while the Army's HLH evaluation occupied half a file drawer.

The other services separated the functions of evaluation and selection, with the latter normally made at the Secretarial level, sometimes without even a recommendation from those who had actually evaluated the proposals. Few senior defense officials have either the background or can take the time required to perform an actual selection in a major weapons system competition. They invariably must rely on a highly condensed briefing of the competitive proposals and evaluation data given to them by some who are more nearly qualified to make the decision. The lack of a recommendation was required by some officials in order to eliminate any possibility of a "reversal" of a source selection decision.

By using personnel from within its own organization, and working within their own offices, elaborate security controls were not needed in the Navy system. All personnel handling the proposals had adequate security clearances and were experienced in safeguarding sensitive information. Written pledges of forthrightness, lack of conflicts of interest, and strict control over evaluation data were never found necessary, since all these factors were part of each individual's normal responsibilities.

The other services ability to do accurate independent estimating of weight, performance, cost, etc. was greatly prejudiced by their personnel assignments, ad hoc approach, scoring methods and undue security controls. Without this ability, the design competition process cannot work, as each contractor tends to out promise all others, leading to so-called "lying contests". Only a strong technical group can prevent the problem.

After OSD standardized the source selection process along the lines of the Air Force system, the Navy was forced to modify its practices to indicate compliance. An "Evaluation Board" made up of the Division Directors involved in the evaluation was named, while the "Advisory Council" included all the "Assistant Chiefs", and the "Authority" was the "Chief". The basic system remained the same, the appearance was changed. In more recent years, the system has been compromised even more with ever increasing emphasis on legalistic procedures, none of which appear to have been instituted because of actual problems encountered with the older acquisition procedures. Under the simpler methods used, there was never a formal protest of an award in a Navy aircraft competition. LTV 's protest in the F-18 program, although rejected by the GAO, was occasioned in part by a lack of understanding of the rules of the particular game to be played, and would undoubtedly never have occurred under our older, more open and straightforward procedures.

The Navy's system worked very well over many years. No other system has been shown to be nearly as effective, or cost effective. Every effort should be made to return the process to as simple a form as possible. Awards should continue to be made on the basis of the best design at a reasonable price to meet real requirements of the fleet.