2015 F-35 DOT&E report: More of the same... With some fudging.

Mmmm...  Fudge.

The Pentagon's Director Operational Testing and Evaluation (DOT&E) report was released this week allowing us to take a peek at how well the JSF's testing has been going over the last year.

This has been an interesting year for the JSF, to say the least.  While it has been the brunt of much criticism (including mine), the program has seen quite a bit of progress over the last year.  The biggest sigh of relief must have been when two F-35Cs successfully completed carrier landing trials, putting that nasty tailhook problem to bed.

Of course, the elephant in the JSF's room for 2014 was the ill-timed engine fire that put a halt to the F-35's international debut at the Royal International Air Tattoo (RIAT) and the Farnborough Air Show after that.  A fix has been devised for engine issue, but flight restrictions were put in place, keeping F-35 test pilots from testing the aircraft's limits.

None of this is any surprise to those who have been following the program.

The real stand out in the 2015 DOT&E report is not so much in what is being reported, but how things are being reported.  It would seem as if some numbers were being fudged to make for a more positive image.  This was done three ways:

1. Reclassifying something as an "induced" failure (i.e. caused by wear-and-tear), rather than an "inherent" failure (i.e. design flaw).
2. Counting flight hours of all 100 aircraft, but only counting failures of the 30 aircraft with the most up-to-date parts.  (Thereby dramatically increasing the failure-free flight hour rate).
3. Fixes that require multiple attempts to find a solution are recorded as a single repair.  (Previously each attempt was recorded).

The report also brings to light an issue with the F-35's stealth coverings.  Repairs currently require a 48 hour "cure" time.  New materials may reduce this to as little as 12 hours, but require refrigeration, something that might be difficult to provide while operating from austere conditions.

Defense-aerospace.com is hosting a PDF of the report here:  http://www.defense-aerospace.com/dae/articles/communiques/DOT-E_2014_Annual_Report_Section_on_F-35.pdf

Here are some highlights:

• In spite of the focused effort, the program was not able to accomplish its goal of completing Block 2B flight testing by the end of October. 
• Based on test point accomplishment rates experienced since October 2013, the program will complete Block 2B development in February 2015. 
• As a result of the engine failure that occurred in an F-35A in late June, the program imposed aircraft operating limitations (AOL) on all variants of F-35 aircraft at the flight test centers and operational/training bases. These AOLs were:

          -  Maximum speed of 1.6 Mach (0.9 Mach for production aircraft at operational/training bases), 

          -  Maximum g-load of 3.2 g for test aircraft and 3.0 for production aircraft, 

          -  Maneuvers limited to half-stick roll rate and 18 degrees angle of attack 

          -  No rudder input, unless required for safe flight (production aircraft restriction only)

• Due to the AOL, numerous test points needed for the Block 2B fleet release and Marine Corps IOC were blocked and cannot be attempted until the restrictions are lifted. 
• Progress in weapons integration, in particular the completion of planned Block 2B weapon delivery accuracy (WDA) events, has been less in 2014 compared to that planned by the program. The program planned to complete all 15 Block 2B WDA events by the end of October, but completed only 7.
• Overall suitability continues to be less than desired by the Services, and relies heavily on contractor support and unacceptable workarounds, but has shown some improvement in CY14. 
• Inspections of the engines on all variants led to discoveries on nine production and test aircraft requiring engine replacement. 
• Restrictions imposed on the fleet from the June engine failure coupled with the focus on Block 2B mission systems testing hampered progress in F-35A flight sciences testing. 
• Progress in weapons integration, in particular the completion of planned weapon delivery accuracy (WDA) events, has been very limited in 2014 compared to that planned by the program. Multiple deficiencies in mission systems, aircraft grounding, and subsequent flight restrictions caused by the June engine failure all contributed to the limited progress.
• Overall suitability continues to be less than desired by the Services, and relies heavily on contractor support and unacceptable workarounds, but has shown some improvement in CY14.

-  Aircraft availability was flat over most of the past year, maintaining an average for the fleet of 37 percent for the 12-month rolling period ending in September – consistent with the availability reported in the FY13 DOT&Ereport of 37 percent for the 12-month period ending in October 2013. However, the program reported an improved availability in October 2014, reaching an average rate of 51 percent for the fleet of 90 aircraft and breaking 50 percent for the first time, but still short of the program objective of 60 percent set for the end of CY14. The
bump in availability in October brought the bump in availability in October brought the fleet 12-month average to 39 percent.-  Measures of reliability and maintainability that have ORD requirements have improved since last year, but all nine reliability measures (three for each variant) are still below program target values for the current stage of development. The reliability metric that has seen the most improvement since May 2013 is not an ORD requirement, but a contract specification metric, mean flight hour between failure scored as “design controllable” (which are equipment failures due to design flaws). For this metric, the F-35B and F-35C are currently above program target values,and F-35A is slightly below the target value, but has been above the target value for several months over the last year. 

•  Low availability rates, in part due to poor reliability, are preventing the fleet of fielded operational F-35 aircraft (all variants) from achieving planned, Service-funded flying hour goals. Original Service bed-down plans were based on F-35 squadrons ramping up to a steady state, fixed number of flight hours per tail per month, allowing for the projection of total fleet flight hours. 
• The most recent 90-day rolling averages for MFHBF_DC show more growth in this metric than for any other reliability metric for the period from May 2013 through August 2014. The following contributed to the reported growth in MFHBF_DC.

0. • -  In June 2013, the program re-categorized nut plate failures, one of the most common failures in the aircraft, as induced failures rather than inherent failures, removing them from the calculation of MFHBF_DC. Nut plates are bonded
     to an aircraft structure and receive bolt-type fasteners to hold removable surface panels in place. One way nut plates can fail, for example, is when torquing a bolt down while replacing a removed panel, the nut plate dis-bonds from the aircraft structure, preventing securing the surface panel.
     
   • -  Distinguishing between inherent design failures and induced failures can be subjective in certain cases. For example, if a maintainer working on the aircraft bumps a good component with a tool and breaks it while working on a different part nearby, it is a judgment call whether that is an inherent design failure because the component could not withstand “normal” wear and tear in operational service, or if it’s an induced failure because the maintainer was “too rough.”

• For example, as of September 2014, an improved horizontal tail actuator component had been introduced and installed on roughly 30 aircraft out of a fleet of nearly 100. Failures of the older component were not being counted in the metrics at all anymore, but flight hours from all 100 aircraft were counted. This calculation could result in the reported reliability of that component being increased by up to a factor of three compared to reliability if all of the horizontal tail actuator failures were counted. There are hundreds of components on the aircraft, so a single component’s increased estimate of reliability may have little influence on overall observed aircraft reliability. However, since multiple components are being upgraded simultaneously due to the unprecedented and highly concurrent nature of the F-35 program, the cumulative effect on the overall observed aircraft reliability of the increased estimate of reliability from all of these components may be significant. 

Maintenance as a whole remains to be a challenge for the F-35, with the report stating that "The amount of time spent on maintenance for all variants exceeds that required for mature aircraft".  Much of this time seems to be spent dealing with the finicky Automated Logistic Information System (ALIS). 

 "The program develops and fields ALIS in increments similar to the mission systems capability in the air vehicle. Overall, ALIS is behind schedule, has several capabilities delayed or deferred to later builds, and has been fielded with deficiencies. The program does not have a dedicated end-to-end developmental testing venue for ALIS and has relied on feedback from the field locations for identifying deficiencies. Though some of the early deficiencies have been addressed, ALIS continues to be cumbersome to use and inefficient, and requires the use of workarounds for deficiencies awaiting correction"

Despite all this, the USMC is still insisting that the F-35B will meet its initial operating capability (IOC) by July of this year.  This has skewed testing towards the STOVL version of the JSF while testing on the other two versions has lagged behind.

It should be noted here the "initial operating capability" is not the same as "full operating capability". All that is required for IOC is a handful of aircraft with a limited weapon capability.  It will not be ready to "go to war".  At best, it will be useful for training purposes and further testing.

Since the F-35B is capable of supersonic flight and BVR combat, it can be seen as a serious upgrade from the USMC's soon-to-be-retired fleet of AV-8B Harriers.  With its limited Block 2B software, the JSF does not yet meet the capability of the USMC's fleet of F/A-18C/D Hornets (not-Super).  That is probably why the USMC will keep its legacy Hornets until 2029.

While Lockheed Martin and the Pentagon often brag about the F-35's progress, one thing is very much certain:  The Joint Strike Fighter still has a way to go.