Boeing 717-200, VH-VQC

The crew of the Boeing 717-200 aircraft reported that during the climb from Launceston airport, while passing 7,000 ft above sea level, the right engine sustained an uncommanded in-flight shutdown. The R ENG RPM LO alert was observed followed by the RH SYS FAIL advisory. The crew reported that they did not see any caution advisories prior to the shutdown. The ENGINE FAIL/SHUTDOWN INFLIGHT checklist was actioned and the crew completed a single engine landing.

Following the event, the operator's maintenance personnel interrogated the multi-function control display unit (MCDU) and carried out a right engine electronic engine controller (EEC) fault review check. A return to service check, a dry motoring run and an engine idle run were carried out with no faults found. A further EEC fault review was carried out and several fault codes were noted in the memory. These related to electronic faults listed for a FADEC SYSTEM FAULT [full-authority digital engine control] and EEC BOX FAULT [electronic engine controller]. After conferring with the engine manufacturer, the EEC and the fuel-metering unit (FMU) were removed for further testing. After replacement of those units, an EEC return to service and an FMU leak check were carried out. At the request of the engine manufacturer, engine ground runs were carried out and the aircraft was deemed to be serviceable.

Flight recorder data

Data reviewed following the flight confirmed that approximately 3 minutes after full power application for takeoff, the fuel flow to the right engine dropped to zero, resulting in the subsequent in-flight shutdown of the engine.

Component history

Documentation provided by the operator annotated that the EEC part number 114E6112G119, serial number LHBR0141, had accumulated 426.7 hours time since new, 393 cycles since new, and had been installed on 2 August 2002. The unit had been modified to software version 6.1 (the most recent version).

Electronic engine controller

The EEC was a two-channel (A and B) electronic unit with system redundancy. It controlled, among other items, engine start sequencing, power requirements, operating temperature, turbine speeds, fuel flow, engine monitoring, and automatic relight. It contained fault detection, storage, and readout capabilities, all stored on an electrically erasable/programmable read-only memory (EEPROM) located on a computer board assembly. The EEPROM provided a history for troubleshooting purposes of any fault event within the EEC or associated control systems by logging a fault code of the event. Those fault codes were then stored until intentionally cleared during maintenance action. The distinct two channels in the unit ensured that should one channel fail, the other would assume control and monitoring of the engine. The transfer of control and monitoring of the engine to one EEC would not necessarily signify that other items controlled by the non-controlling channel would not function. The EEC also provided an electrical signal for opening the engine fuel-control metering valve (normally closed) upon engine starting, which was spring loaded to the closed position.

Component testing

The EEC and FMU units were shipped to the respective component manufacturer's facilities for testing under the supervision of the United States of America (USA) National Transportation Safety Board. Representatives from the aircraft, EEC and engine manufacturers were also present for the testing. Testing of the FMU revealed no anomalies.

EEC serial number LHBR0141 testing

During environmental stress screening of the EEC (a high-speed scan of the faults over a temperature cycle alternating from -55 to +74 degrees C), failures of the Channel A EEPROM were recorded when the internal temperature of the EEC was at -2 degrees C or colder. Test procedures used to test new units for acceptance, also revealed faults of the Channel A EEPROM at temperatures below -55 degrees C. However, initial testing of the EEC could not duplicate the dual channel failure (A and B) that would have been required to sustain the reported in-flight shutdown.

The Channel A EEPROM was sent to the manufacturer for detailed examination. Examination indicated that a phenomenon called a 'single bit flip' had occurred within the used memory section area of the input/output microprocessor of the unit. The input/output microprocessor memory was configured with positively charged hexadecimal binary 1s occupying unused memory sections. The 'single bit flip' phenomena was a result of unused sections of the microprocessor memory becoming negatively charged binary zeros, resulting in checksum failures and 'health lane' degradation of the EEC. Checksum failures are the result of discrepancies of the internal self-check program, which sums the values of all memory blocks.

Follow up vibratory testing of the EEC confirmed a failure of Channel B. Further examination indicated fracturing of solder joints at five resistors on the analog interface module circuit board of Channel B.

Other Australian fleet occurrences

On 24 November 2002, while on the downwind leg for landing at Hobart, another crew of the same aircraft reported that the left engine 'spooled down'. The crew reported that they did not see any caution advisories prior to the power decrease. The crew then completed a single engine landing. Following the event, the operator's maintenance personnel conducted troubleshooting of the left engine and noted one fault code related to the EEC (not listed in maintenance documentation) logged on the MCDU memory. The engine was inspected and ground run, both at idle and at a high power setting. The engine started and operated normally. The EEC and FMU were replaced after conferring with the engine manufacturer. Further engine ground runs were completed and the aircraft was deemed to be serviceable.

Documentation provided by the operator recorded that the EEC part number 114E6112G119, serial number LHBR0148, had accumulated 4,686.4 hours and 4,311 cycles since new. The unit had been modified to software version 6.1.

The engine manufacturer advised that a visual inspection of the unit revealed fracturing of the soldier joints of six resistors of Channel A and four resistors of the Channel B analog interface module circuit boards.

Other overseas occurrence

On 30 November 2002, a USA operator's Boeing 717-200 was on climb at FL 280 when it sustained an in-flight shutdown of the right engine. Following the event, EEC part number 114E6112G119, serial number LHBR0093, which had accumulated approximately 6,700 hours time since new was examined. That examination revealed fracturing of the solder joints at five resistors of Channel A and ten resistors of the Channel B analog interface module circuit board.

Solder joint fracturing

The engine manufacturer reported that the anomaly of fracturing or cracking of the resistor solder joints was believed to have resulted from thermal cycle induced stress due to differential thermal expansion between the printed circuit board and the resistor. They further reported that identical resistor packages were utilised on the installation of both channels within the EEC and that the solder joint fracturing anomaly could affect a total of seven resistors per channel of each EEC. Five of these resistors were assessed as being capable of contributing to the top-level failure events analysed in the unit system safety assessment.

Service bulletin history

On 20 December 2002, the engine manufacturer issued Service Bulletin SB-BR700-73-101401. That bulletin referenced compliance with the EEC manufacturer's Service Bulletin SB-BR715/73-009 also released 20 December 2002, which gave instructions for the repair of several specific resistors on the analog interface module circuit board. That repair would attach the resistors to the board and connect them to the original solder pad by 'flying leads'. This would then eliminate any mechanical stress on the resistors. Compliance time of that bulletin was at the next shop visit of the EEC for repair, or as arranged by the EEC manufacturer and was not mandatory.

On 17 January 2003, the engine manufacturer issued Service Bulletin SB-BR700-73-101404. That bulletin referenced compliance with the EEC manufacturer's Service Bulletin SB-BR715/73-010 also released on 17 January 2003, which gave instructions for a software modification of the processor communication's modules (A3, A4) with new input/output software to change the fill pattern of the unused areas of the EEPROM memory from hexadecimal binary 1s to binary 0s, thereby reducing the possibility of checksum failures. Compliance time of that bulletin was at the next shop visit of the EEC for repair, or as arranged by the EEC manufacturer.

On 20 February 2003, the engine manufacturer issued Service Bulletin SB-BR700-73-900316 advising the fleet operators of a numbers of inspections and modifications to improve the reliability rates of the EEC. The bulletin listed a total of ten service bulletins issued by either the engine or EEC manufacturer, which the engine manufacturer recommended be incorporated at the earliest opportunity without affecting flight schedule. Incorporation of these modifications required a return of the component to the engine manufacturer.

None of those service bulletins were mandated through the issuing of an airworthiness directive from either the USA Federal Aviation Administration or the German Airworthiness Authority.

During climb following take-off, the right engine electronic engine controller (EEC) removed electrical power from the engine fuel-control metering valve, which was spring loaded into the closed position. Following loss of the electrical signal, the valve closed resulting in fuel starvation and engine shutdown. The loss of electrical signal to the engine fuel-control metering valve was the result of a dual channel failure of the EEC. The dual channel failure was believed to be the result of:

the failure of channel A because of checksum anomalies of the electrically erasable/programmable read-only memory and

the failure of channel B as a result of electrical intermittences, caused by the loss of signal path resulting from solder joint fractures of the resistors of the analog interface module circuit board.

Examination of other fleet EECs has confirmed the fracturing anomaly of the solder joints of the resistors of the analog interface module circuit board. The effect of the solder joint fracturing on the function of the EEC appears to be loss of signal path on the circuit board and eventual 'health lane' degradation of the unit leading to a shutdown of that channel.

Fracturing of one or more resistor solder joints on both channels of the EEC simultaneously could lead to the loss of system redundancy in the EEC and a subsequent in-flight shutdown of the engine. The possibility of a successful engine restart could be difficult to predict. The crack propagation rate of the resistor solder joint fracture, and the amount required for signal loss is as yet unknown. Therefore, probability calculations for reliability rates of the units may be inaccurate. Compliance to Service Bulletins SB-BR700-73-101401 (SB-BR715/73-009), SB-BR700-73-101404 (SB-BR715/73-010) and SB-BR700-73-900316 was not mandatory.

The right engine EEC sustained a failure of both channels of an independent two-channel system, resulting in an in-flight engine shutdown with no prior indications to the flight crew.

Local safety action

Engine/EEC manufacturer

On 25 October 2002, the engine manufacturer issued worldwide communication WW/20032/1/25-10-002 informing operators and airframe and engine technical representatives of the in-flight shutdown event and the fault codes witnessed.

On 27 November 2002, the engine manufacturer issued worldwide communication WW/20032/2/27 Nov. 02 updating the operators and airframe and engine technical representatives of the in-flight shutdown event investigation. That communique informed of the testing of the Fuel Metering Unit and EEC.

On 20 December 2002, the engine manufacturer issued Notice to Operators (NTO) number 54 advising operators of the in-flight shutdown events and of engine restart procedures in the event of an engine shutdown without abnormal engine indications. That communique also advised of harness installation procedures, review of the fault codes and recommended interrogation of the multi-function control display unit (MCDU) at intervals of 50 flight hours.

The engine manufacturer further advised that they will be incorporating a software upgrade of the EEC to version 7.0, which will include an improvement to remove the possibility for certain intermittent failures to trigger a 'health lane' degradation without triggering the corresponding maintenance message.

The Operator

Following the return to service of the aircraft after the 4 October 2002 event, the operator implemented a MCDU interrogation procedure on the incident aircraft for a period of three days. The MCDU stored fault codes were reviewed at the end of each day of flying. The procedure was similar to that subsequently recommended by the engine manufacturer in NTO 54 issued on 20 December 2002. The maintenance history of the right engine was also reviewed as far back as its installation on the aircraft, which occurred on 3 August 2002.

Following the 24 November 2002 event, the operator carried out similar actions as completed after the 4 October 2002 event. The maintenance history of the aircraft was reviewed as far back as the last A check in September 2002. The MCDU review procedure was expanded to cover all aircraft in the fleet. The operator initiated review procedure was superseded on 27 November 2002 by an interim health check procedure developed by the engine manufacturer. This procedure required MCDU fault codes to be reviewed after every 25 to 30 sectors.

Australian Civil Aviation Safety Authority

On 29 November 2002, the Australian Civil Aviation Safety Authority (CASA) issued a directive to the operator to review all fault codes at the end of each day's flying for the occurrence aircraft until further notice.

On 6 December 2002, CASA relaxed the MCDU review requirement for VH-VQC to every port with engineering support available.

On 16 January 2003, CASA issued a request to the operator requiring reviews additional to NTO 54. That action included a review of fault codes at the end of each day's flying for all operator aircraft, with the incident aircraft logged fault codes being reviewed after each sector where engineering support was available. They further expanded the review of fault codes after each sector where engineering support was available to include all operator B717 aircraft.

CASA has subsequently advised the ATSB that the operator has commenced the following program to ensure continued airworthiness of the fleet:

- The MCDU is to be interrogated for EEC faults after each flight into a manned port. Recurring faults identified in NTO 54 should result in replacement of the EEC.

- Fault codes and corrective actions are to be reported to CASA.

- EECs are to be modified per RRD SB-BR700-73-900316.

- One modified EEC is to be installed, in turn, on each aircraft in the fleet.

- Once a modified EEC has been installed, its reliability is to be monitored by continuing the MCDU interrogation after each flight to a manned port for two weeks. Once the reliability is established, MCDU interrogation can be extended to service check intervals.

- Modification of both the EECs, and confirmation of their reliability through MCDU interrogation described above constitutes the corrective actions after which the MCDU interrogations can revert to service check intervals.

- All EECs returned to the manufacturer are to be upgraded and the entire fleet is scheduled to be modified not later than the first quarter of 2004.

RECOMMENDATIONS

Recommendation 20030032

As a result of this investigation, the Australian Transport Safety Bureau recommends that the German Airworthiness Authority, Luftfahrt-Bundesamt issue an airworthiness directive to mandate compliance with Rolls-Royce Deutschland Ltd and Co KG Service Bulletin SB-BR700-73-900316.

Recommendation 20030037

As a result of this investigation, the Australian Transport Safety Bureau recommends that the United States Federal Aviation Administration liaise with the German Airworthiness Authority, Luftfahrt-Bundesamt to develop and issue an airworthiness directive to mandate compliance with Rolls-Royce Deutschland Ltd and Co KG Service Bulletin SB-BR700-73-900316.