Exhibit A-5. A retyped paper on Program Analysis and Evaluation that was part of a NAVAIR training program. Similar pitches were given before retirement, but this was after that and done as "consultant". 1975. GAS

[Ed Note: Charts are included except where repetitive. Chart 1 is the first chart of the Exhibits -- "Naval Aircraft Starts". Chart 8 is the Cumulative Costs chart shown under VF-2, and Chart 9 is the TFX Cost Estimates chart shown under VF-2]

SESSION 14A

PROGRAM ANALYSIS

AND

EVALUATION

Overview -- Aircraft Design and Development

Session No. 30A -- Summary of Navy Program Requirements

Abstract -- Some insight is provided on the history of the Navy's aircraft development process, the reasons for changes as they occurred, and the lessons learned from the successful and unsuccessful programs.

Subject: Navy Aircraft Programs, History and Lessons Learned

1. Santayana observed that, "those who fail to heed history are doomed to repeat it." The purpose of this presentation is to provide some of our naval aircraft history and give a few of the lessons learned from both successes and failures, so that the mistakes of the past can be avoided in the future.



2. Naval aviation started in 1910 when a civilian pilot, Eugene Ely, took off from the USS Birmingham and landed in Willoughby Spit. The following year, he took off from, and landed successfully on, a platform on the USS Pennsylvania. In that same year, the first appropriation for naval aviation $24,000, was approved by the Congress. When World War I was entered by the USA in 1917, the Navy had a total of 54 airplanes. When it ended a year and a half later, the number of airplanes on hand totaled 2107. Obviously that growth rate could not be sustained in peacetime, but naval aviation continued its progress into the modern era. Chart 1 shows production Navy airplane deliveries for the years of 1935-1970. During World War II a peak of 22000 was reached. Korea produced a slight bulge, but deliveries reached barely 10% of those during WWII. The SEA fracas provided for only a modest increase overall from peacetime practice, with total production purchases running only about 1000 involving costs.



3. A month's worth of production Hellcats of WWII is more than the total planned buy of a new fighter, or a V/STOL. This great reduction in production rate, on the order of 10:1 is a major contributor to our current problems.



4. Chart 2 shows all of the new airplanes started through the same time period of 1935-1970. During the decade of the 40s, on the average, six new airplanes per year were started, only slightly less than the average of the preceding five years. During the 50s, the average of new starts dropped to about four, while in the 60s, the average was just over one. The outlook continues to be bleak.



5. Charts 3 and 4 show plots of cumulative aircraft deliveries for a number of models versus time from initial go-ahead, and versus time from initial first flight. The following observations can be made:



1. The F8F-1 was the most rapid of the Navy's combat airplane developments. Started in WWII, it was the light weight fighter (LWF) of its day, designed for the air combat maneuvering arena. It differed from today's LWF concept, however, in providing full armament capability and adequate (barely) fuel. Sacrificed were only the nonessentials. There was no seat adjustment provided. Altitude performance was sacrificed by use of a single stage, two-speed supercharged engine. (A two stage engine under development was planned as a growth step.) To save structural weight, "safety wing tips" were provided, and the overall factor of safety reduced from 1.5 to 1.4. The design was in today's parlance, a minimum risk project, built on the firm background of the F6F series, using a proven engine and propeller, and the best available armament. Production plans called for reaching a 500 per month rate in about two years from go-ahead. When the war ended, a squadron had deployed and was on its way to Japan, and production reached about 200 per month.



2. The A-1 (started as the BT2D) was the second most rapid development of the era. It was a solid development built on the failure of the SB2D, a two-place airplane with remotely controlled top and bottom aft firing turrets, and with a bomb bay. The airplane was too heavy, overly complex, and few of its innovative features developed on schedule, or performed as predicted. The power plant installation was salvaged and became the base for the very successful, much simpler, single place A-1. At that time, the fly-before-buy concept had arisen again, after the WWII hiatus, so the XBT2D-1 was a prototype on paper, with no formal production release until first flight. Almost as soon as the wheels lifted off, the contractor was given formal coverage for the production effort which had been underway since initiation. Douglas, El Segundo, with this airplane started a series of well designed naval aircraft with emphasis on weight control achieved by design simplicity, good structural load paths, and a strong defense against non-essential requirements.



3. The F-8, started as the XF8U-1, has served as a model of what a successful airplane development should be. The requirement was sound, the design competition tough, and the development well managed by both the Navy and the contractor. The design followed the quite unsuccessful F7U at Chance Vought, and proved conclusively that selection decisions can safely be made on the merit of a design, and not on the record of the last development. There were, and are, many who fail to heed history in this regard. The F-8, while labeled as a prototype development by some because of the XF8U-1 designation applied to the first two aircraft was actually a concurrent program, designed from the outset for production, and with a production contract let prior to first flight. This procedure allows a development cycle of about four years from contractual go-ahead to fleet use, or about five years from start of specification preparation for a competition.



4. The Army's OH-6 is noted on these charts to illustrate the length of a development cycle using a full fly-before-buy approach. In that development, the Army purchased five helicopters each from Bell, Hiller, and Hughes, and conducted a fly-off. After a two step procurement competition, Hughes was awarded the production contract. It can be seen that this simple, light weight helicopter required 2-3 years longer to get into the fleet than much more demanding projects.



5. F3H and F4D designs were almost "prototypes" with production release given prior to first flight for designs which incorporated major modifications. Both were started as "interceptors" and both were produced as "general purpose" fighters in the terminology of that day. Both required replacing the unsuccessful J40 engine during development. In this case, the initial requirement was faulty, the engines were not ready to be specified for an airplane development, and the overall results were predicted by some.
   



6. Chart 5 provides a closer look at the airplanes started since 1950 and categorizes them as to whether they achieved fleet use, one measure, albeit incomplete, of development success. A detailed examination of all these models is well beyond the scope of this presentation, but some lessons learned and overall conclusions can be given:



1. Of the 51 airplanes started, 14 did not reach the fleet. Of these 14, three were terminated almost immediately.



1. The F6D was the "Missileer", a subsonic, long endurance, fleet defense airplane, which carried six "eagle" missiles, the development of which had preceded the airplane by a year or two. The development was stopped by the incoming McNamara administration in favor of the TFX. The airplane would certainly have been a success, but would have required a complementary fighter to handle the jobs requiring airplane speed and agility. The concept won all Fleet Air Defense operational analyses then, and would today. When it became possible to do both fleet defense and the other fighter roles with a single airplane, the concept lost its attractiveness.
2. The VF/VTOL was a technically feasible design, with little operational capability. The Air Force took over the project, and continued it for about a year before dropping it.
3. The P6Y was an open ocean ASW seaplane designed to exploit a dunking sonar. The requirement was not firm enough to support the funds required, and it was dropped from the budget immediately after the competition.



2. Of the other 11 projects which failed to reach the fleet:



1. The F-111B failed because of the technical impossibility of meeting the diverse mission requirements of the Air Force and Navy. The failure was predicted.
2. The F8U-3 was a highly successful fighter started as a single place, single engine competitor to the F-4, cancelled by the Congress to eliminate duplication. It was carried to a "fly-off" stage, which it won in the normal sense. It had better performance and flying qualities and was cheaper. The decision for the F-4 was made because of the growing conviction that 2 men were needed for the all weather fighter job.
3. The TT-1 was an underpowered primary trainer which verified the predictions that it should not be procured. It was actually started as an afterthought to the well planned T-2 basic trainer project.
4. The X-22, FY, and FV were all VTOL research projects, although the latter were disguised as "convoy fighters" since pure research projects were not then popular enough to be financed.
5. The ROE and RON were one man, portable helicopters intended to investigate a concept of getting all Marines off the ground. There were few who believed in the practicality of the concept. The RON eventually became the base for the unmanned DASH helicopter.
6. The P6M was the final attempt of seaplane advocates to continue the type as a prime military weapon. The hydrodynamic development was successful, but two aircraft were lost in accidents, and the overall cost of the program eventually forced abandonment of the concept. The requirement was questionable at best, since at the time, 2 A3Ds could do the military missions from a carrier and would cost less.
7. The HCH and HRH were related reaction drive helicopter projects. The HRH was a secondary award from the competition which started the HR2S (H-37) and was recognized as a high risk project in order to achieve a 300 kt. Cruise speed for the Marine amphibious mission. The HCH was a crane version of the same design, with a very short range requirement. The power plants were dual
8. The remaining 37 airplanes all saw service use, though not always in the precise way intended. For example, the A3J had a short history as a supersonic attack airplane for high altitude nuclear missions, but a long history as the RA-5C for reconnaissance missions. All of the designs started by the Navy, and intended for service use, came close enough to their technical predictions that failure to reach the fleet was for other reasons. It can also be observed that no design characterized as a high risk project which would require prototyping, has achieved a production status.



7. The types of contract used in airplane developments have changed over the years. Prior to WW II, it was normal to procure prototype designs on a fixed price (and underpriced) basis. After a test period, production aircraft contracts would be awarded, again normally on a fixed price basis. With the changing economic conditions of WW II, cost reimbursement types became the accepted means of doing business. This type of contract allows looser specifications, and is easier and quicker to negotiate. Justification from the government viewpoint was normally that contractors would include unduly large contingency allowances in any fixed price arrangement so that the cost plus types would really be less expensive. This reasoning was debatable then and is still questionable. Because of the split in the government's budget process between R&D and Production funds, and with R&D funds tending to be the more restrictive, there was always a tendency to justify projects as "Production". In the early fifties, approval was obtained to fund new airplanes from R&D only through mock-up, or preliminary engineering. This strategy allowed many more starts than would have been possible under full funding rules, but of course, lead eventually to conflicts with other authorities on duplication and an excessive number of production models to be funded. Since the R&D budget failed to expand as rapidly as inflation in the economy did, this later led to very real problems in the 60s when purity in the R&D budget became fashionable again. Any new airplane required orders of magnitude more money than had been allocated previously. For example, the F-4 was built with no R&D funds, and the A3J with about 4 million. The current requirement for this type of airplane would be at least a half billion.



8. It was financial pressure that forced the Navy into fixed price contracting for development in the late 50s and early 60s. In the CPFF and CPIF developments of the A-6, E-2, and RA-5C, the yearly overruns were so sizable that many smaller programs had to be cancelled to fund the airplanes. The government just could not afford such overruns, and so logically undertook to get better cost quotations from its contractors. This led to getting fixed price bids as well as cost plus. Surprisingly, the fixed price bids, in general, were close enough to the others that they were more attractive. Fixed price contracts were then used for the CH-46, CH-53, A-7, and OV-10 contracts, and fixed price incentive (with a ceiling) contracts for the F-14 and S-3. Fixed price contracting is a good deal more demanding for the buyer, as well as the seller. Specifications must be solid not only for the airplane proper, but for all the test work and data required. This requires a discipline not usually appreciated. Because of the lack of flexibility provided in the A-7 FFP contract (when the contractor would not increase the quantity without an increase in unit price) variable lot prices were established in which the contractor provided firm prices for quantities 50% above and below the normal quantity. This is still the preferred method of contracting, in my opinion, but some pitfalls must be avoided:



1. The period of time, or the number of aircraft, should be limited. The A-7 and CH-46 at about 200 aircraft, and the CH-53 at 100 were reasonable.



2. The unknowns should be understood well enough to be assured that the contractor can produce a usable product.



3. Government estimates must be available to give reasonable assurance that any buy-in is within the capability of the contractor to absorb. (For example, the Hughes bid for the 700 -1000 OH-6s was ridiculously low, but he was able to absorb the loss).



4. Specifications must be firm. (The F-111 FPI contract for the R&D quantity failed to include about half the tasks. The flight test program for the F-111A was completely undefined and unpriced. Eventually, the R&D price doubled, but without loss to the contractor. (ed: showing the importance of specifications, this is what happened when they were insufficient)



5. Inflation corrections, which are beyond the control of the contractor and the Navy, should be permitted.



9. In the last two years, there has been a resurgence of interest in Prototyping, and the fly-before-buy approach. The interest was generated outside the Navy by some theorists who said that having prototypes available gave more options to the decision maker when force level budget decisions had to be made. Others felt the expenditures worthwhile to keep design teams intact and viable, while others apparently used the scheme as a way to stall and avoid production expenditures. The GAO has fostered the approach since at least 1959 while RAND has also issued a number of reports favoring the scheme. Few, if any, of the proponents ever price out alternative programs, or offer any proof of the purported advantages. The Navy abandoned the approach because it could not afford the time and dollar costs involved. Today engineers can accurately predict the success or failure of a design. Proof of the ability to fly has not been needed for many years. With a minimum of 2-3 years longer development period, even a noncompetitive prototype program will cost 10-15% more by virtue of inflation alone. When more than one design is prototyped the cost is increased. A Navy study in 1959 showed that a multiple design and prototype approach delayed fleet introduction by 2 ½ years and increased program cost by about 25%, and this without any inflationary penalties. Answers to other proposed benefits are noted below:



1. The OH-6 example noted previously proves that prototyping does not ensure realistic price quotes for production airplanes. Buy-ins are still possible.



2. Cost savings are often cited if contractors are allowed to ignore all MIL specification. An interesting proof to the contrary occurred in the mid 40s when the XF8B-1 and XF15C-1 projects were undertaken. The former was performed without specifications, while the latter was a standard development. Both designs failed to progress beyond the prototype stage, but the non-specification XF8B cost nearly twice the price of the XF15C.



3. The XF8B-1 design also proved the absurdity of giving the contractors a free rein in establishing design requirements. The resulting airplane was a technical success, but operationally unusable, since it was mediocre in each of its possible roles as fighter, bomber, or torpedo plane. Prototypes and research vehicles have served a useful purpose in the past and can in the future, but they have no place in the acquisition phase of a combat airplane where both time and cost are important.



10. It is abundantly clear that the most important problem facing naval aviation is obtaining enough money to procure a viable force. The magnitude of the problem can be seen by examining charts 6 and 7 which show annual deliveries again in comparison to "flyaway" dollars available in the budget. The dollars have actually decreased, while the purchasing power of the dollar has been cut in half even when using the OSSSD, Systems Analysis values for variation in purchasing power, inflation, or economic index. (A recent NavAir study shows a significantly worse position for that factor.)



11. The current OSD "solution" to the budget problem is to "design to cost". This is another of many programs with attractive titles, but of no real substance. Cost has long been one of the most important elements in the entire acquisition process. Alternative approaches must be priced in assessing how best to meet requirements. Once a project is defined, specifications prepared, competitions held, and a contractor selected, cost can hardly be treated as a design variable. In the design process, minimum acceptable levels are established for all features of an airplane. In theory then, trade offs are no longer possible to save cost. If lower performance or technical levels were acceptable, they should have been stated in the original type specifications at the beginning of the procurement cycle.



12. In an organization as large and as complex as DOD, a great problem exists in ensuring that the ultimate decision maker has correct facts on which to exercise his judgement. Most, if not all, of the poor decisions of the past can be traced directly to improper, inadequate, or misleading data. At the working level, invariably, the information is complete, but as the problem goes upward through the system there is always a condensation problem, and often a fallback position introduced by compromisers. Too often, "alternatives" are offered only because it is stated that some must be presented. Only a couple of examples need be given:



1. The TFX original decision to buy a single fighter for the Air Force and Navy was made on the basis that it would save a billion dollars. This was based on data included in a report issued by DDR&E and shown on chart 8, indicating that 1000 DOD compromise TFXs cost a billion dollars less than 500 Navy TFXs plus 500 Air Force TFXs. The individual unit estimates for the three airplanes involved are noted on chart 9. Note that there is an obvious discrepancy between their compromise prices. If the decision had been made to buy the Air Force airplane, as priced by the Air Force, a saving of two billion dollars could have been realized. The decision maker was given incorrect data.



2. In the HLH decision, Mr. Packard was presented with data, again from DDR&E, in the DCP which indicated a saving of a half billion dollars if the Marines were forced to use the Army's HLH design. Eventually, the decision was reversed when the true facts were presented, showing that the Marines could not operate the proposed design on available ships, and that even if they could, the cost was substantially greater than buying the small CH-53Es for the Marines and the very large HLHs for the Army. Incredibly, the same mistake was repeated with the new DepSecDef and again corrected. The fault in this case lay entirely with OSD staff personnel.



13. In summary, the naval aviation procurement record is markedly better than that of the other services. Most of the tactical airplanes and missiles used in Viet Nam were developed by the Navy. The system has worked well and should be retained and improved. Its essential ingredients have been:



1. Realistic requirements issued by OPNAV, after close coordination with NavAir to ensure technical feasibility.



2. Sound specification system SD-24, MIL-D-8706, MIL-D-8708, etc.



3. Honest design competitions.



4. Competent development monitoring and contract administration by NavAir's functional organization.




END