23 Oct 2018 - Final report published into JA85AN 737-800 Pressurisation failure 27 May 2016
On Friday, May 27, 2016, at around 08:22 Japan Standard Time (JST: UTC + 9 hours; all times are indicated in JST), Boeing 737-800, JA85AN, operated by All Nippon Airways Co., Ltd, took off from Tokyo International Airport as scheduled flight 561 but, as it was climbing, turned back at 08:27 because there was a warning indicating a drop in cabin pressure and landed at 09:11. Upon detailed inspection of the aircraft, no damage was observed; however, it was found that both valves for the intake of bleed air from the left and right engines into the respective air conditioning packs had temporarily malfunctioned and were closed. There were 170 people on board the aircraft, consisting of a Captain, five other crewmembers and 164 passengers. One passenger suffered minor injuries.
PROBABLE CAUSES: It is highly probable that this serious incident occurred when, as the aircraft was being continuously operated without a malfunction involving temporary shutdowns of the left air conditioning pack being perceived by the flight crewmembers or mechanics, the left air conditioning pack shut down during takeoff and then the right air conditioning pack, which had the same service hours and service environment, also shut down, and as a result pressurization was not maintained. It is probable that the left and right air conditioning packs shut down because, in both cases, the reference regulators inside the valves that control airflow to the air conditioning packs (eFCV) were stuck, and as a result the eFCVs closed from the rising Bleed Pressure and air was not supplied to the air conditioning packs.
*** Updated 20 Nov 2018 ***
An eFCV is a flow control valve that operates pneumatically and controls electronically. Switching a cockpit Pack Switch (Figure 4 ①) to AUTO or HIGH opens the eFCV’s butterfly valve (Figure 4 ②) and begins the supply of air to the air conditioning pack. Inside the eFCV is a reference regulator that adjusts air pressure used for control inside the eFCV (hereinafter referred to as “the Supply Pressure”) (Figure 4 ③). The reference regulator is in a fully open state when the Bleed Pressure is low, including when the aircraft is parked with the engines off. However, it maintains the Supply Pressure within a specified range by closing in response to increases in the Bleed Pressure. When the Supply Pressure is maintained within the specified range, the outflow from the vent is changed (Figure 4 ⑤) by electronic control by the PFTC (Figure 4 ④) and the opening position of the eFCV’s butterfly valve is controlled as a result. On the other hand, when the Supply Pressure exceeds the normal range, the outflow from the vent increases beyond the adjustment range of the PFTC’s electronic control, the eFCV’s butterfly valve closes, and the amount of air supplied to the air conditioning pack is decreased.
Investigation of Equipment Components
Relating to the Failure From the state of operation of the air conditioning packs described in 2.1.1, it was found that the left eFCV closed as the Aircraft was in its takeoff run and the right eFCV closed immediately after takeoff, that both eFCVs opened about eight minutes later, and that the left then repeatedly opened and closed intermittently. Additionally, as was described in 2.8, it was found that a phenomenon occurred whereby the left eFCV temporarily closed on one flight on each of the dates of May 18, May 20, and May 25 as well as all seven flights that took place that took place on May 26, the day prior to the serious incident. The self-diagnostic functions of each PFTC and CPC that control air-conditioning and pressurization were checked, but there were no records indicating a failure. Additionally, when the operations of the air-conditioning and pressurization systems were checked through engine ground tests and flight tests, the failure was not reproduced. To identify the cause, the eFCVs and equipment relating to the eFCVs operation were removed from the Aircraft, and a detailed investigation was conducted at the facilities of each designer/manufacturer with the cooperation of the National Transportation Safety Board (NTSB) of the United States and the Transportation Safety Board (TSB) of Canada. The results were as shown in Table 2. It deserves noting that all of the removed equipment items had been installed in the Aircraft since the time of the Aircraft’s manufacture and had no history of failure.
Investigation of the Aircraft’s eFCVs
The investigation of the left and right eFCVs resulted in the findings:
System-level test - Left eFCV - After the eFCV was exposed to a highhumidity environment, the five equipment items of the left air conditioning pack that were removed from the Aircraft (provided in Table 2) were mounted on a test rig for development testing of the designer/manufacturer and tested, but the failure was not reproduced.
Visual inspection - When the reference regulator was disassembled and visually inspected, black grime was found along the air passage. Additionally, a small number of abrasions were found on the movable part within the reference regulator.
Material (composition) analysis - When particles attached to the movable part within the reference regulator were analyzed, atmospheric dust and other constituents were detected.
3.3 Relationship with Meteorological Conditions At each time point during the times the Aircraft was parked between the Aircraft’s arrival at Tokushima Airport two days prior to the serious incident until the occurrence of the serious incident, the temperature was in a range of between 20 and 26℃ and the dew point was in a range of between 19 and 23℃; thus, the weather conditions were marked by high humidity. Accordingly, it is somewhat - 18 - likely that condensation occurred when the temperature temporarily dropped while the Aircraft was parked at night and that drops of condensed water flowed into the reference regulators positioned on the bottom of the eFCVs as described in 2.9.5.
3.4 Occurrence Time of the Serious Incident As was described in 2.1.1, the left air conditioning pack shut down at 8:21:47, the right air conditioning pack also shut down at 8:22:25, and an excessive cabin altitude warning device activated at 8:27:42, which was immediately after the cabin altitude passed 10,000 ft. Accordingly, it is highly probable that the time at which both air conditioning packs were shut down (8:22:25) was the time that the serious incident occurred.
3.5 Reason the Air Conditioning Packs Stopped (1) From the troubleshooting described in 2.10, it is probable that the air conditioning packs shut down because the eFCVs that control the flow of air to the air conditioning packs fully closed. (2) From the eFCV mechanism that was described in 2.9.4 as well as the eFCV investigation that was described in 2.10.2 and 2.10.3 and the case of a similar failure on another aircraft, it is probable that the eFCVs fully closed because the reference regulators were stuck open. If a reference regulator is stuck open, the function of the reference regulator, which adjusts the Supply Pressure to correspond to increases in the Bleed Pressure, is lost and the eFCV’s butterfly valve closes as a result of the excessive Supply Pressure. (3) It is probable that the air conditioning packs regained their function and began operating again because the high Supply Pressure mentioned in (2) above occurs when the Bleed Pressure increase due to engines high power operation, and because the eFCVs will subsequently return to their normal state when, as the Bleed Pressure decrease with aircraft’s descent or deceleration, the Supply Pressure also decrease. (4) It is somewhat likely that the attached atmospheric dust and other substances that were described in the teardown inspection mentioned in 2.10.2 and the flowing water drops mentioned in 3.3 were contributing factors to the reference regulators’ being stuck open.
3.6 Factors Contributing to the Shutdown of Both Air Conditioning Packs Because the possibility that the PFTC and other control equipment contributed to the shutdown of both air conditioning packs and the possibility that the failure of one pack had an effect on the other pack cannot be confirmed from the results of the troubleshooting described in 2.10, it is probable that the following factors contributed to the shutdown of both air conditioning packs at the time of takeoff. (1) Grime and abrasions similarly existed inside the reference regulators of both eFCVs. It is probable that this was because both eFCVs had been installed in the Aircraft since its manufacture and had been in operation in the same environment, and that as a result the reference regulators of both eFCVs were similarly prone to becoming stuck open. (2) It is probable that both air conditioning packs shut down during takeoff on the day of the serious incident because, under conditions in which the reference regulators of both eFCVs were similarly prone to becoming stuck open, the reference regulators became stuck due to high humidity resulting from weather conditions marked by showers and mist while the Aircraft was parked on the two previous days, said stuck condition continued until the first flight of the following day (specifically, the flight of the serious incident), and both engines power increased when the Aircraft was taking off with the reference regulators in a stuck condition, resulting in a situation in which the air conditioning packs shut down simultaneously for the reason described in 3.5 (2). (3) As was mentioned in 2.8, instances in which the left air conditioning pack had shut down were frequently recorded in the FDR in the time leading up to the serious incident in which both air conditioning packs shut down. The FDR contained a record for the most recent period of approximately ten days, and it is somewhat likely that the air conditioning pack had been repeatedly shutting down even before that period. However, it is probable that because, as was mentioned in 2.9.2, the Aircraft was not equipped with a function that notifies flight crewmembers or mechanics even if an eFCV closes and an air conditioning pack shuts down, the frequent failure of the left air conditioning pack was not noticed and the pack remained in operation without repair, and this led to the situation in which the two systems, left and right, malfunctioned simultaneously during the flight of the serious incident.
3.7 History of the Flight From the statements of the relevant parties mentioned in 2.1.2 and the eFCV malfunction factors mentioned in 3.5, the operational circumstances whereby the cabin altitude fell rapidly and rose rapidly after both air conditioning packs shut down are analyzed for each flight phase as follows. As a result, it is probable that the failure was limited to the left and right eFCVs and that the CPC and other equipment that control pressurization were functioning properly. (1) It is highly probable that the cabin altitude warning device activated when the cabin altitude reached approximately 10,000 ft because the left and right air conditioning packs sequentially shut down for the reason described in 3.5 (2) when the engines power increased during takeoff and pressure to the cabin increased under conditions in which it was not being maintained. (See Appendix Figure 2 ①) It is probable that, until the warning device activated, there were no warnings to notify the flight crewmembers of the abnormal pressure, and that flight crewmembers could not know about the abnormal pressure without reading the abnormal cabin altitude on the Cabin Altitude Panel (see Figure 3). (2) The flight crewmembers canceled ascent when the cabin altitude warning activated, transitioned to level flight at FL160, and slowed to 250 kt. It is probable that, as a result, the left and right air conditioning packs restarted for the reason described in 3.5 (3) because the Bleed Pressure decreased together with the decrease both engine power. (See Appendix Figure 2 ②) (3) It is probable that, although both air conditioning packs had restarted and pressurization began, the OFV was temporarily left open to prevent a rapid descent in cabin altitude because, as was mentioned in 2.11.3, a lower limit was set for the automatically controlled rate of descent of cabin altitude. (See Appendix Figure 2 ③) (4) It is probable that, after the Aircraft transitioned to level flight at FL160, the left air conditioning pack shut down again because the Bleed Pressure increased together with the engine power. (See Appendix Figure 2 ④) (5) Although the flight crewmembers executed the Cabin Altitude Warning checklist mentioned in 2.11.1, switched both PACK switches to HIGH, and operated the switch to close the OFV by switching the pressurization mode to MAN (manual), the OFV was already in a fully closed condition. (See Appendix Figure 2 ⑤) (6) It is probable that, as a result of the descent to 13,000 ft, the Bleed Pressure decreased together with the engine power and the left air conditioning pack restarted. (See Appendix Figure 2 ⑥) It is probable that the amount of air flowing into the cabin increased suddenly with the operation of both air conditioning packs because both PACK switches were operating at HIGH with the OFV fully closed. As a result, during the approximately two minutes that the Aircraft descended to 13,000 ft, pressurization progressed rapidly and there was a rapid descent in cabin altitude at rete of -3,500 fpm on average. During this time, the cabin altitude warning stopped, as the cabin altitude reached approximately 9,000 ft. (See Appendix Figure 2 ⑦) (7) It is probable that, when the Aircraft transitioned to level flight at 13,000 ft, the left air conditioning pack shut down again because the Bleed Pressure increased in the same manner described in (4) above. (See Appendix Figure 2 ⑧) (8) The FDR record showed that, during the Aircraft’s approximately ten minutes of level flight at 13,000 ft, the right air conditioning pack was working and operating at High Flow, but the cabin altitude was gradually rising and pressurization was not being maintained. Given that, as was described in 2.11.2, pressurization is ordinarily maintained with High Flow operation by one air conditioning pack, It is somewhat likely that, although the right eFCV was partially open but regulating incorrectly due to the reference regulator issues described earlier in the report. It is not likely that it was full closed as the limit switch was not faulty any other point during the event or during the subsequent component evaluations and because the cabin altitude was climbing slowly during this time and not rapidly as would be the case if there was no cabin air inflow. (See Appendix Figure 2 ⑨) (9) It is probable that, as a result of the descent to 9,000 ft, the Bleed Pressure decreased together with the engine power and the left air conditioning pack restarted again. (See Appendix Figure 2 ⑩) Similar to the situation described in (6) above, the operation of both air conditioning packs while the outflow valve was fully closed in manual mode resulted in a state of rapid pressurization. Moreover, it is probable that both air conditioning packs continued operating, even after the Aircraft transitioned to level flight at 9,000 ft, because the Bleed Pressure did not rise high enough to result in the left air conditioning pack’s shutting down in the manner described in (7) above, and thus the situation became one in which the most rapid pressurization on the flight was occurring. Consequently, for a period of approximately four minutes that lasted from after the Aircraft descended to 9,000 ft and transitioned to level flight until the pressurization mode was switched to AUTO and the OFV opened, pressurization continued rapidly and the cabin altitude rate of descent was below -4,000 fpm on average and for a moment fell below -8,000 ft （⊿Ｐ=8.967 psid）. (See Appendix Figure 2 ⑪) (10) As was mentioned in (6) and (9) above, during the Aircraft’s descended to 13,000 ft and then to 9,000 ft, the cabin altitude fell at a rapid rate of descent below -3,500 fpm on average for approximately two minutes in the case of the former and below -4,000 fpm on average for approximately four minutes in the case of the latter. This value falls far below the maximum cabin pressurization rate of change for pressurization when pressurization is automatically controlled of -350 slfpm that was provided in 2.11.3. It is probable that was the time when passengers felt pressure in their ears, and that some passengers suffered injury to their ears. (11) It is probable that, when the cabin altitude was descending rapidly, automatic control of pressurization by the CPC that was mentioned in 2.11.3 started when the Pressurization Mode Selector switch was switched from MAN to AUTO in a state which the cabin altitude was below -8,000 ft at a flight altitude of 9,000 ft, and the OFV opened because the cabin and outside differential pressure (delta P) raised to 8.967 psid. In addition, it is likely that one or both PPRV’s activated thus contributing to the rapid cabin altitude increase. (See Appendix Figure 2 ⑫) (12) For one minute around the time that the OFV opened to a maximum of 75%, the cabin altitude rapidly rose above 9,500 fpm on average. (See Appendix Figure 2 ⑬) It is probable that the “white mist” that appeared in the statement of the Chief Purser, mentioned in 2.1.2 (3), was caused when water vapor in the air condensed because, when the OFV opened to a maximum of 75% along with the likely activation of the Positive Pressure Relief Valves released air from cabin outside, the air pressure in the cabin fell rapidly and the temperature dropped under conditions in which a large amount of air was flowing in from the top of the cabin due to pressurization resulting from the restoration of air conditioning pack functions. (13) AUTO FAIL was issued when the operating CPC detected a failure from an excessive rise in the cabin altitude that greatly exceeded 2,000 fpm and the outflow valve has not responded correctly, which is one of the conditions for illumination of the AUTO FAIL light that was mentioned in 2.11.4. Subsequently, the pressurization mode switched to ALTN (alternate) and the green ALTN light came on with the switch to the backup CPC that was standing by. (See Appendix Figure 2 ⑭) It should be noted that, as was mentioned in (6) and (9) above, excessive changes in the cabin altitude above 2,000 fpm were observed at the times of descent to 13,000 ft and descent to and level flight at 9,000 ft; however, no warning was issued at those times because the pressurization mode was set to MAN (manual). Additionally, although an excessive rise in the cabin altitude was also observed when the Aircraft was taking off and climbing while in the AUTO pressurization mode, without pressurization being maintained, AUTO FAIL was not issued because, even at its largest, the rise in cabin altitude was about 1,900 fpm and did not exceed 2,000 fpm.
4. PROBABLE CAUSES
It is highly probable that this serious incident occurred when, as the aircraft was being continuously operated without a malfunction involving temporary shutdowns of the left air conditioning pack being perceived by the flight crewmembers or mechanics, the left air conditioning pack shut down during takeoff and then the right air conditioning pack, which had the same service hours and service environment, also shut down, and as a result pressurization was not maintained. It is probable that the left and right air conditioning packs shut down because, in both cases, the reference regulators inside the valves that control airflow to the air conditioning packs (eFCV) were stuck, and as a result the eFCVs closed from the rising Bleed Pressure and air was not supplied to the air conditioning packs.
5. SAFETY ACTIONS
5.1 Safety Actions Taken by the Aircraft and Equipment Manufacturers The Aircraft’s manufacturer (The Boeing Company) and equipment manufacturer for the eFCV (Honeywell) implemented the following measures. (1) Upgrade of ACMS software The Company upgraded the ACMS’s software so that it can monitor when an eFCV is closed for three seconds or longer, either the left side or the right side, after takeoff, and The Boeing Company informed other operators about a similar method. (2) Introduction of double packings on the reference regulator poppet and increased clearance between the reference regulator poppet and guide. Honeywell recommended that this revised reference regulator (eFCV P/N: 63396754-1 Series 3) be implemented as an interim measure only to reduce the probability of reference regulator failures. (3) The eFCV (P/N: 63396754-2) developed for the Boeing 737 MAX aircraft has a fail-safe design in which the eFCV will open even if there is a failure caused by a reference regulator’s becoming stuck open and Supply Pressure rises excessively. Because using this eFCV will eliminate failures occurring when an eFCV accidentally closes, Honeywell recommended upgrading to this eFCV and The Boeing Company proposed this improvement to operators.
5.2 Safety Actions Taken by the Company The Company implemented the following measures to prevent recurrence of similar incidents. (1) Addition of a function to notify flight crewmembers of an eFCV in a closed condition (provisional measure) With the upgrade to the ACMS software mentioned in 5.1 (1), an eFCV in a closed condition can be detected by the ACMS. The Company added a function that uses this signal to notify the flight crewmembers when an eFCV is closed as follows. (a) Notice is outputted directly from the ACMS by the cockpit printer. (b) The open/close condition of the eFCVs can be observed at any time on the cockpit’s Multi- Purpose Control Display Unit (MCDU). (c) When information on an eFCV’s closing is communicated from the ACMS through the ACARS*15 to the ground maintenance support department, the information is automatically uplinked from the ground and printed out in the cockpit together with a chime. (2) The Company prohibited use of the Minimum Equipment List*16, which permits operating on only one air conditioning pack, until permanent measures are implemented. (Provisional measure) (3) The Company prohibited use of the MEL for the ACMS, ACARS, and VHF communications in order to guarantee monitoring of eFCV closings by the ACMS and notification of flight crewmembers as described in (1) above. (Provisional measure) (4) The Company installed double-packings eFCV (P/N：63396754-1 Series 3) with O-rings on the reference regulator poppet and increased clearance between the reference regulator poppet and guide that was mentioned in 5.1 (2) and confirmed that no abnormalities occurred in the reference regulators. (5) Following the measure described in (4), it became possible to equip its aircraft with the eFCV (P/N: 63396754-2) mentioned in 5.1 (3), and the Company installed them. It should be noted that, with the introduction of this eFCV, the eFCV will not close even if the reference regulator is stuck open, and therefore the provisional measures mentioned in (1) to (3) above were cancelled.