Investigation number
RO-2020-005
Occurrence date
Location
Randwick Depot
State
New South Wales
Report release date
Report status
Final
Investigation level
Defined
Investigation type
Occurrence Investigation
Investigation phase
Final report: Dissemination
Investigation status
Completed
Occurrence category
Incident
Highest injury level
None

Safety summary

What happened

On 3 April 2020, light rail vehicles (LRV) 053/054 were stabled at the light rail depot in Randwick, New South Wales. At approximately 0249, workers heard a loud noise and on investigating found a battery enclosure cover on the ground between two other LRVs. The cover was identified as originating from LRV 053.

Closed-circuit television cameras within the light rail depot captured the ejection of the cover from LRV 053. The cover was ejected with a flash visible followed by a plume of smoke or vapour and the cover landed approximately 6 m away.

There was significant damage to the battery compartment and batteries on LRV 053 and also minor damage to two other vehicles. There were no reported injuries.

What the ATSB found

The software controlling the battery charging likely corrupted during the uploading process which went undetected. The fault with the software resulted in overcharging the batteries on multiple LRVs, including LRV 053. The overcharging of batteries was not detected prior to the occurrence through Alstom’s validation or fault monitoring processes.

The overcharging of LRV 053 batteries generated excessive hydrogen within the batteries. The battery enclosure on LRV 053 ruptured when flammable gases were released into the enclosure in the presence of an ignition source. The risk of explosion had been identified although relied on risk controls that were ineffective at preventing the escalation in this instance.

What has been done as a result

Alstom made changes to the management of train fault data to generate automatic alerts for battery over temperature faults. A new fault code was created within the train control monitoring system to monitor the battery charging temperature and alert the driver if any of the defined thresholds were exceeded.

Additionally, further software validation and testing will be conducted by the battery charger software supplier, as well as during software acceptance testing and following uploading of revised software by the maintainer.

Safety message

The introduction and commissioning of new assets must ensure that design requirements and risk controls are tested and validated as functional. Additionally, fault monitoring and maintenance regimes must monitor asset condition, so as to avoid conditions that might escalate and contribute to accidents.

The occurrence

On the evening of 2 April 2020, light rail vehicles (LRV) 053/054 arrived at the light rail Randwick Depot, New South Wales. The vehicles were stabled towards the back of road 3 along with other vehicles. A worker completed a pre-service inspection of LRV 053/054 around midnight with no abnormalities identified.

On 3 April 2020 at approximately 0249,[1] workers in the stabling yard heard a loud noise. On investigating the source of the noise, they located a roof-mounted battery enclosure cover on the ground between two other vehicles on road 1 and 2. Further investigation identified LRV 053 as missing a battery enclosure cover.

Closed-circuit television (CCTV) cameras in the stabling yard showed a flash coming from the battery enclosure of LRV 053 as the enclosure cover was ejected from the LRV (Figure 1). The cover struck the overhead contact wire, then struck LRV 005 and 058 before falling between the two vehicles (Figure 2). The cover was airborne for approximately 4.12 seconds[2] from being ejected to striking LRV 005. There was substantial damage to the battery enclosure on LRV 053 and minor damage to LRV 005 and 058. There were no reported injuries.

Figure 1: Randwick yard CCTV footage

Figure 1: Randwick yard CCTV footage.<br />
The figure shows stills taken from the CCTV at 0248:50 with a flash visible and at 0248:57 showing smoke or vapour emanating from the battery enclosure of LRV 053. The ejected panel struck the overhead contact wire and landed to the left of LRV 053 out of view of camera CM03DR.<br />
Source: Transdev, modified and annotated by OTSI

The figure shows stills taken from the CCTV at 0248:50 with a flash visible and at 0248:57 showing smoke or vapour emanating from the battery enclosure of LRV 053. The ejected panel struck the overhead contact wire and landed to the left of LRV 053 out of view of camera CM03DR.

Source: Transdev, modified and annotated by OTSI

Figure 2: Battery enclosure cover landing and on the ground

Figure 2: Battery enclosure cover landing and on the ground.<br />
The figure shows stills from the external CCTV from LRV 005. The ejected cover from LRV 053 struck LRV 005 and 058 and fell between the two vehicles.<br />
Source: Transdev and OTSI, modified and annotated by OTSI

The figure shows stills from the external CCTV from LRV 005. The ejected cover from LRV 053 struck LRV 005 and 058 and fell between the two vehicles.

Source: Transdev and OTSI, modified and annotated by OTSI

Post-occurrence events

Immediately following the occurrence, Alstom reviewed the available data and identified numerous battery over temperature faults had been recorded on LRV 053 and 054.

In addition, LRV 023 was identified as recording battery over temperature faults. LRV 023 was in service and was terminated, before returning to Randwick depot for inspection at approximately 0920 on 3 April. An initial visual inspection did not detect any obvious abnormalities with the batteries on LRV 023.

__________

  1. Times shown in 24 hour time as Australian Eastern Daylight Savings Time (AEDT).
  2. The CCTV frame rate was 25 frames per seconds, time + 0.04 seconds.
Context

Environment

The Bureau of Meteorology (BOM) automatic weather station at Observation Hill,[3] recorded the temperature as 18.7 °C at 0300 on 3 April 2020.

The temperature was recorded between 17.8 °C and 24.8 °C in the days leading up to 3 April. The mean temperature for March 2020 was 25.3 °C with a maximum of 37.6 °C.

Location

The light rail depot was located in Randwick, New South Wales (NSW). The depot was built as part of the Sydney Light Rail project and operates as a light maintenance and stabling yard.

There were a total of 13 stabling roads and the yard had the capacity to store the entire fleet of 60 LRVs. LRV 053/054 were stabled on road 3 at the time with LRV 005 on road 2 and LRV 058 on road 1.

Sydney Light Rail

The Sydney Light Rail (SLR) project was awarded to Altrac Light Rail Partnership as a consortium including Acciona, Alstom and Transdev (Figure 3).

Figure 3: Sydney Light Rail structure

Figure 3: Sydney Light Rail structure.<br />
The operations and maintenance sub-structure is made up of Alstom, Transdev Maintenance Services (TDMS) and International Cleaning Services (ICS).<br />
Source: Altrac, modified and annotated OTSI

Source: Altrac, modified and annotated OTSI

The SLR project included:

  • design and construction of the CBD South East Light Rail (CSELR)
  • design, construction and commissioning of the LRVs for the CSELR
  • operation and maintenance of the CSELR and operation of the existing Inner West Light Rail (IWLR) Line 1.

The CSELR consists of two new rail lines, with Line 2 running between Circular Quay and Randwick and Line 3 between Circular Quay and Juniors Kingsford.

Testing and commissioning for the Line 2 LRVs commenced in Sydney in late 2018 and continued up until December 2019. Public passenger services commenced on Line 2 on 14 December 2019 and Line 3 on 3 April 2020.

Light rail vehicle information

Vehicle 053

LRV 053 was manufactured in Spain and was delivered to Sydney in October 2019. The LRV underwent final commissioning and entered service in November 2019.

As of 2 April 2020, LRV 053 had travelled 18297 km and there were no known faults with the LRV.

General

The SLR vehicles are variant of Alstom Citadis X05 (305)[4] range designed for city environments. The LRVs operate with either a conventional pantograph or on catenary-free sections utilising the Aesthetic Power Supply (APS) system. Both systems provide 750 V DC for traction power and to the auxiliary converter for auxiliary loads and battery charging. Within the stabling yard, the LRV is powered by the pantograph and overhead contact wire.

Each LRV set consist of five vehicles with drivers cabs at each end (Figure 4). In service the LRVs operate with two sets coupled together, in this case LRV 053 and 054.

Figure 4: Light rail vehicle details

Figure 4: Light rail vehicle details.<br />
Source: Altrac, annotated by OTS

Source: Altrac, annotated by OTSI

Battery system

The LRVs are fitted with batteries to provide power to the LRVs auxiliary equipment. The battery enclosure and batteries were supplied by Hoppecke to meet Alstom’s functional requirements. The battery enclosure was mounted on the roof of the M1 vehicle and the batteries were charged by the auxiliary converter fitted to the M2 vehicle.

Battery cells

The battery cells were vented nickel cadmium (NiCd) designed for rail applications. The battery cell consists of two stacks of positive and negative fibre nickel cadmium (FNC) plates submerged in an electrolyte (Figure 5). The electrolyte used was a mixture potassium hydroxide (KOH) with an addition of lithium hydroxide (LiOH) and distilled water.

The battery model was a Hoppecke FNC 235 R3 (235AH) and the cell casings were manufactured from flame retardant polypropylene (PP-VO).

The nominal cell weight when filled with electrolyte was 11.1 kg + 3% (10.77 to 11.43 kg). The volume of electrolyte between the minimum and maximum level was 810 mL.

Figure 5: Battery cell structure

Figure 5: Battery cell structure.<br />
Source: Hoppecke, 2019. D62109-300-en08_Manual_Alstom_X05_NAT, modified by OTSI

Source: Hoppecke, 2019. D62109-300-en08_Manual_Alstom_X05_NAT, modified by OTSI

Battery enclosure

The battery enclosure consists of two compartments to house the battery cells and electrical compartment for electrical connections and to facilitate current, voltage and temperature monitoring. The battery enclosure design is a standard product and can house a number of different battery cell configurations. The SLR LRVs used 19 battery cells to form the battery bank.

The battery enclosure was manufactured from stainless steel and the covers for both compartments were secured with four latches each. The battery compartment had two vents positioned on the cover to provide ventilation. The covers for both compartments were also fitted with sun shields to provide some thermal protection.

The battery cells were numbered from 1 to 19, starting with battery 1 at the positive battery terminal. Each cell was connected with a metal cell connector strap and a vent / water filling hose. The battery vent / water filling hose connected to each battery cell and vented to atmosphere through a flame arrestor for backfire protection (Figure 6).

The temperature probe was fitted on top of the battery cells between cells 5, 6, 9 and 10 and reported the temperature to the auxiliary converter.

Figure 6: Battery compartment components

Figure 6: Battery compartment components.<br />
Source: Hoppecke (2019). D62109-300-en08_Manual_Alstom_X05_NAT, modified and annotated by OTSI

Source: Hoppecke (2019). D62109-300-en08_Manual_Alstom_X05_NAT, modified and annotated by OTSI

Maintenance requirements

The technical maintenance plans (TMP) specified a number of tasks to be completed relating to the batteries when the LRV reached 75,000 km (annual inspection). These tasks required the following:

  • inspection and cleaning of the battery enclosure
  • checking electrolyte level and topping-up as required
  • testing the battery insulation resistance
  • measuring charging voltage of the battery.

LRV 053 had not travelled 75,000 km, neither had any other LRV and was not due these inspections.

Auxiliary converter

The auxiliary converter which contained the battery charger was supplied by Centum Adetel Transportation Solution (Adetel). Alstom as the design integrator communicated the requirements from Hoppecke and Alstom to Adetel to provide a suitable auxiliary converter.

The auxiliary converter functions to invert the 750 V DC power supplied by the overhead contact wire or APS to lower voltages:

  • 400 V AC heating and ventilation air conditioning (HVAC)
  • 230 V AC general
  • 24 V DC low voltage auxiliary equipment and battery charging.

Battery charging

The battery charger was designed to supply voltage and current to the batteries. Hoppecke specified that the battery charging must supply a float voltage of 27.93 V DC at 20 °C and be temperature compensated by -3 mV/°C per cell.[5] The charging voltage should increase for temperatures below 20 °C and decrease for temperatures above 20 °C (Figure 9). The auxiliary converter software controlled the battery charging voltage and temperature compensation.

Software version

The SLR fleet was supplied with auxiliary converter software version B09. This version of software had been designed and validated as meeting the Hoppecke and Alstom’s requirements. This software version was also supplied to Alstom’s X05 fleet of LRVs in Nice, France.

In November 2019, Adetel released a revised software version B11. The change was deemed minimal and was approved for use by Alstom.

The software update to B11 for the SLR fleet commenced in December 2019. There were six LRVs outstanding as of 3 April 2020, these were LRV 023/024, 045/046 and 053/054.

Fault monitoring and management

When the battery temperature reached ≥ 60 °C for more than five seconds a battery over temperature fault (F_MVS_BatOverTempFail) was recorded within the train control monitoring system (TCMS). The maximum charging voltage was also limited to 24.3 V when the battery temperature reached 60 °C.

This fault was available for review within the maintenance menu on the drivers display unit (DDU), although would not generate an alert to the driver. Remote monitoring of the LRV fault data was also possible through the use of Alstom’s HealthHub.[6]

Battery temperature faults

Review of the battery temperature faults for the 30 days prior to the occurrence showed a number of LRVs had recorded over temperature faults. LRV 053 and 054 accounted for most of the detected faults with 971 and 74 faults respectively, closely followed by LRV 023 with 68 faults (Figure 7).

Figure 7: Battery temperature faults

Figure 7: Battery temperature faults.<br />
Source: Alstom, modified by OTSI

Source: Alstom, modified by OTSI

Hazards associated with batteries

There are a number of hazards associated with batteries that must be managed to ensure batteries perform their intended function safely.

Flammable gases

These vented battery cells produce hydrogen and oxygen during charging. The production of hydrogen gas mostly occurs once the battery has achieved 95 per cent charge, or during any boost charging or overcharging the battery.[7],[8] The hydrogen and oxygen are the result of the decomposition of water in the electrolyte, with hydrogen forming on the negative plates and oxygen on the positive plates.

Hydrogen forms flammable mixtures in air at concentrations between 4 per cent and 75 per cent. These limits are referred to as the lower flammable limit (LFL) and upper flammable limit (UFL) respectively and represent the concentrations where vapours will ignite when exposed to an ignitions source of sufficient energy.[9]

Hydrogen requires minimal energy for ignition (0.02mJ)[10] and must be kept away from all potential sources of ignition such as:[11]

  • open flames or fire
  • sparks, electrical (fuses or switches) or mechanical (sparks from grinding)
  • hot surfaces were the temperature is above 300 °C
  • electrostatic discharge.

Combustion of flammable mixtures are defined as either deflagrations or detonations, depending on the velocity of the flame front propagation through the fuel-air mixture.[12]

Combustion of hydrogen and air mixtures will typically result in a deflagration but can transition to a detonation. There are a number of factors that can influence a deflagration or detonations including, hydrogen percentage, enclosure design and strength of ignition source. Detonation of hydrogen and air mixtures is possible at a narrower concentration between approximately 18 per cent and 59 per cent.[13]

Deflagrations usually produce fairly low pressure and generally do not pulverize, but can cause serious structural damage. Detonations are more destructive than deflagrations and produce shock waves and very high pressures.[14]

Chemical

The electrolyte within batteries is caustic and can cause severe burns and eye damage in the event of exposure. The battery cells used on the SLR also contained cadmium which is a toxic substance and a carcinogen.

Vented battery cells are typically safe providing they are handled, stored, maintained and charged in accordance with the manufactures guidelines. In the event of a cell casing failure, mishandling or uncontained battery failure, the battery contents can be released or exposed.

Electrical

Battery cells store energy and terminals are always live. Care must be taken when working on batteries to prevent electrical shocks or short circuits. High currents can be delivered by a single cell or battery bank if there is a short circuit.

The batteries must also be installed with the correct polarity.

Similar occurrences

A review of available records was unable to identify similar occurrences relating to batteries on light rail vehicles, passenger, or freight trains locally or internationally.

There have been two previous incidents investigated where a train component was ejected from a train while in service. Both incidents involved the failure of capacitors associated with traction systems.

  • The Office of Transport Safety Investigations (OTSI NSW) investigated the partial volume deflagration on Waratah carriage N5508 on 20 March 2017. A number of factors contributed to production of flammable gases within the roof mounted traction inverter module. The force of the deflagration ejected a total of four hatches across platforms 3 and 5 at Burwood. (OTSI Investigation - 04770).
  • The Rail Accident Investigations Branch (RAIB UK) investigated the explosions inside an underframe equipment case at Guilford on 7 July 2017. It was found that a manufacturing defect resulted in the capacitor producing flammable gases. The gas ignited with the force ejecting debris across the adjacent platform and up to 70 m away. (RAIB Investigation - 052018).

__________

  1. Bureau of Meteorology recordings were taken from the weather station at Observation Hill at Millers Point. This is approximately 5.8 km north north west from Randwick.
  2. The Citadis X05 range was Alstom’s fifth generation vehicle and were available in different configurations. The X05 fleet consisted of the 205, 305 and 405.
  3. Hoppecke, 2014. FNC R, Railway cells, TD-R-Cells_300, V3.0
  4. HealthHub – an Alstom remote condition based monitoring product.
  5. Australian Standard (2020). AS 2676.1:2020 Installation, maintenance, testing and replacement of secondary batteries in buildings Vented cells.
  6. Australian Standard (2019). AS 3011.1:2019 Electrical installations - Secondary batteries installed in buildings Vented cells.
  7. Brown, William J., et al., (1997). Safety Standard for Hydrogen and Hydrogen Systems: Guidelines for Hydrogen System Design, Materials Selection, Operations, Storage, and Transportation. Office of Safety and Mission Assurance, National Aeronautics and Space Administration.
  8. Hydrogen requires considerably less energy to ignite than other flammable gases or liquids such as petrol (0.24mJ) or methane (0.29mJ).
  9. Hoppecke (2019). Alstom Citadis X05, FNC rail battery system operating and installation manual, D62109-300-en08_Manual_Alstom_X05_NAT.
  10. National Fire Protection Agency (2020). NFPA 921 Guide for Fire and Explosion Investigations 2021.
  11. Brown, William J., et al., (1997).
  12. DeHaan, John D., et al., (2012). Kirk’s fire investigation, 7th ed.
Safety analysis

Battery charging

Design and validation

Alstom as the design integrator used DOORS[15] to manage and validate the various engineering requirements for the design and construction of the SLR fleet.

Alstom provided evidence demonstrating that the requirements from Hoppecke and Alstom had been communicated to Adetel during the design phase. Adetel in turn provided validation test results demonstrating that the auxiliary converter and software met the requirements and functioned correctly within the design environment.

Auxiliary converter software

Prior to acceptance of software version B09 for installation across the SLR fleet, high level functional testing was conducted on LRVs 027/028. Given all the parameters of software version B09 had been validated by Adetel (charging management, temperature curve, etc), Alstom’s software change processes did not require full functional testing. This high level functional testing did not detect any faults and was approved for installation across the entire SLR fleet.

Software version B09 consisted of a number of separate files that needed to be uploaded. The parameters for the temperature compensation and charger management were contained within the configuration script file. Post incident analysis detected that the temperature compensation law on some LRV’s with software B09 was incorrect. For reasons that could not be determined, it was believed that the configuration file was likely corrupted during the uploading process on some LRVs. This was not detected at the time of uploading the software, during the validation testing or prior to the occurrence. Later it was found that the software on LRV 027 had corrupted during the uploading prior to the acceptance testing.

Adetel provided a revised auxiliary converter software version B11 in November 2019. The release notes for this version of software did not detail any known issue with the temperature compensation for software B09. It was communicated that the only known changes to software B11, was the removal of files associated with B10[16] and changes to the uploading process.[17] The process for uploading software version B11 was different to B09 and did not require separate files to be uploaded. The parameters for the charging management and temperature compensation were now imbedded within software B11 and appeared to upload correctly.

At the time of the software revision (B11), the project was transitioning from design and construction to operate and maintain. The change was deemed minor and approved for use by Alstom with B11 forming the standard configuration for the SLR fleet. This change occurred after the transition and was not subjected to the full configuration control board (CCB) review as required.

Temperature compensation

Recorded data from LRV 053 showed that the batteries were charging at approximately 28 V for most of 2 April 2020 (Figure 8). During the same period the battery temperature fluctuated between 50 °C and 60 °C and the charging voltage of 28.11 V was higher than the specified temperature compensated value (Figure 9). At these temperatures the charging voltage should have reduced to between 26.22 V and 25.65 V which did not occur. The batteries were also charging at a high current (17 A) which increased from 2100 and was recorded at 33 A at the time of the occurrence.

Data from LRV 054 showed the batteries were charging at around 28 V on 2 April 2020. The charging voltage of 28.43 V at the time of the occurrence was also higher than that specified by the temperature compensation. The charging current (3 A) and battery temperature (39 °C) were however lower than that of LRV 053.

The batteries on both LRV 053/054 were overcharging in the lead up to the occurrence, while in service and within the stabling yard. The overcharging likely increased the production of hydrogen and oxygen within the battery cells and depleted the electrolyte. The high current recorded on LRV 053 was very likely a result of the battery condition and effects of overcharging.

The recorded battery temperature on LRV 053 was more than 40 °C above the ambient temperature, so ambient temperature was not a factor in the elevated battery temperature.

Figure 8: LRV 053/054 battery charging data from 2 and 3 April 2020

Figure 8: LRV 053/054 battery charging data from 2 and 3 April 2020.<br />
The image shows the graphical representation of the recorded battery current, temperature and voltage measurements for LRV 053/054 in the 24 hours leading up to the occurrence. Battery temperature was recorded as VI_MVS_BattTemp. The value shown for LRV 053 varied between 178 and 188 on 2 April 2020. The actual temperature is derived by subtracting 128 from the values based on the LRV software requirements.<br />
Source: Alstom, modified and annotated by OTSI

Source: Alstom, modified and annotated by OTSI

Figure 9: LRV 053/054 charging voltage vs electrolyte temperature

Figure 9: LRV 053/054 charging voltage vs electrolyte temperature.<br />
The required temperature compensations is shown in the orange with the corresponding voltage shown above the line. The actual recorded voltage and temperature from LRV 053/054 are shown and compared with the required temperature compensated voltage.<br />
Source: Hoppecke (2019). D62109-300-en08_Manual_Alstom_X05_NAT and Alstom, modified and annotated by OTSI

Source: Hoppecke (2019). D62109-300-en08_Manual_Alstom_X05_NAT and Alstom, modified and annotated by OTSI

Temperature probe

The temperature probe was mounted to a post and positioned above batteries 5, 6, 9 and 10. The probe on LRV 053 was found to be sitting up and was not in contact with the top of the battery cells (Figure 11). This was probably the result of the forces created during the occurrence as the temperature probe post allowed the probe to sit on top of the batteries post incident.

The Hoppecke temperature compensation refers to compensating the charging voltage based on the electrolyte temperature (Figure 9). The location of the probe would not provide a direct temperature of the electrolyte but rather the case temperature at the top of the battery cells. Testing conducted post incident determined that the probe measured within approximately 1 °C of the electrolyte temperature.

The difference between the electrolyte temperature and case temperature under normal circumstances would probably be negligible. However, low electrolyte levels would decrease the effectiveness of the temperature probe as the distance between the electrolyte and probe increases.

Survey of battery electrolyte

Post incident Alstom undertook a survey to inspect the battery compartments on the SLR fleet including checking the battery electrolyte levels.

Low electrolyte levels were identified on a total 28 LRVs, including 023, 041, 046 and 54 which had reported battery over temperature faults. The battery banks on the 28 LRVs required between 0.2 and 26 L of distilled water to restore the electrolyte to the correct level. Refer to Appendix A – Electrolyte survey results.

The low electrolyte had not been detected prior to the fleet survey as the LRVs had not reached the 75,000 km maintenance interval and the over temperature faults had not been monitored.

Alstom also undertook the same survey across the global X05 fleets and advised that low electrolyte levels were also identified in the fleet of LRVs in Nice, France.

Uncontained battery failure

The initial inspection conducted on 3 April 2020 identified the battery compartment had ruptured and there was evidence of heat (browning) between a number of cells. Most the battery filler plugs were lifted and a number of battery cell casing had cracked outwards (Figure 10 and Figure 11). The vents on the ejected cover appeared to be unobstructed.

Figure 10: LRV 053 ruptured battery compartment

Figure 10: LRV 053 ruptured battery compartment.<br />
Source: OTSI

Source: OTSI

Figure 11: LRV 053 battery cells and temperature probe

Figure 11: LRV 053 battery cells and temperature probe.<br />
The image shows a close up view of signs of heat between battery cells 9 and 10 and between cells 10, 11, 12 and 13. The temperature probe can also be seen to be sitting up and not in direct contact with the top of the battery cells.<br />
Source: OTSI

The image shows a close up view of signs of heat between battery cells 9 and 10 and between cells 10, 11, 12 and 13. The temperature probe can also be seen to be sitting up and not in direct contact with the top of the battery cells.

Source: OTSI

Two further inspections of the battery compartment were conducted at the Randwick light rail depot on 15 April and 30 April 2020. These inspections were undertaken with representatives from Alstom, ATSB and representatives on behalf of Hoppecke. Observations from these inspections are recorded in Appendix B – Battery cell assessment.

There was no evidence of high resistance joints within the battery or electrical compartments and all electrical connections were secure with witness markings intact. The temperature probe was tested and functioned as expected.

All cells except cell 15 showed signs of damage or the filler plug was lifted. Battery cells 9, 10, 11, 12 and 13 all showed signs of heat with the cell casings melted (Figure 12). Cell 12 had also short circuited. The damage to the cell casings indicates that the actual temperature far exceeded the recorded temperature of 60 °C as the cell casing begin to melt at 120 °C.

There was no visible electrolyte within the cells when inspected and all battery cells were below the minimum electrolyte level. Cell 19 which was missing a portion of the cell casing appeared to have approximately 20 to 30 mm[18] of electrolyte in the bottom of the cell.

The battery cells weighed between 1.5 and 1.9 kg less than the nominal weight of 11.1 kg. Some cells were missing sections of the cell casing although the low electrolyte would be the greatest contributor to the reduced cell weight.

Figure 12: Battery cells 9 to 13

Figure 12: Battery cells 9 to 13.<br />
The image shows the damage to the cell casings for batteries 9 to 13. Cell casings for batteries 11 and 12 were fused and required mechanical separation to remove from the battery compartment.<br />
Source: OTSI

The image shows the damage to the cell casings for batteries 9 to 13. Cell casings for batteries 11 and 12 were fused and required mechanical separation to remove from the battery compartment.

Source: OTSI

The batteries on LRV 053 recorded frequent over temperature alerts in the 30 days leading up to the occurrence. During this time the batteries were overcharging. Most of the electrolyte probably evolved into hydrogen and oxygen due to overcharging and vented to atmosphere via the filler plugs.

The battery compartment ruptured due to pressure caused by the deflagration of flammable gases containing hydrogen. The flammable gases were probably released into the enclosure by a combination of the melted battery cells and an internal deflagration within the battery cells.

The exact source of ignition was not able to be determined. Hydrogen requires minimal energy for ignition and it is possible that the gases could have been ignited by one of the following:

  • a short circuit in cell 12
  • heat produced as a result of increased current and cell degradation.

Management of light rail vehicle faults

The battery temperature fault management characteristic for the X05 fleet were based on the similar logic used for Alstom’s Citadis X02 fleet. The over temperature fault had not been determined to be critical providing the temperature compensation and charging logic functioned as designed.

On each occasion the batteries on LRV 053 reached 60 °C a battery over temperature fault would be recorded. Charging continued in excess of the designed maximum charging voltage of 24.3 V at 60 °C.

The over temperature faults, charging voltage and current were all recorded, however, there were no systems in place to monitor fault conditions and alerts. Consequently, the ability to identify and monitor potential defects that might escalate and contribute to accidents was limited.

Enclosure design

The risk of explosion was known to both Alstom and Hoppecke and had been assessed with the following risk controls and recommendations:[19],[20],[21],[22]

  • temperature compensated charging
  • appropriate ventilation
  • maintenance inspection (including electrolyte level, charging voltage and vents for obstruction).

Projection of materials had also been assessed as a known risk in the event of an explosion.

In order to prevent the build-up of flammable gases the battery enclosure was vented. Ventilation had been calculated in accordance with EN 50272 – 2, Safety requirements for secondary batteries and battery installations. Stationary batteries. This specified that the production of hydrogen within the battery enclosure must not exceed 4% (LFL). The following boundary conditions were used to calculate the required ventilation:[23]

  • Temperature: 20 °C
  • Cell type: FNC 235 R3 (235AH)
  • Batteries are fully charged
  • Batteries are charged with a constant voltage charger with current limit
  • Train [LRV] is not moving
  • The ventilation will be calculated for float charging conditions of the battery
  • We also assume that electrical precaution against charger malfunction will be provided
  • Factor of safety of 5, to accommodate faulty cells in a battery string and an aged battery this safety factor also compensates for a temperature increase up to approximately 44 °C.

The calculated minimum ventilation requirement was 43 cm2. The battery enclosure cover for the SLR fleet was supplied with two 75 cm2 mushroom style vents (inlet and outlet). The ventilation exceeded the requirements of EN 50272-2 and permitted the use of a standard cover across the various X05 fleets.

Under normal circumstances hydrogen and oxygen produced during charging would be vented externally to the enclosure through the vent / water filler hose and flame arrestor.

The battery enclosure on LRV 053 ruptured as a result of forces created by internal pressure when a flammable concentration above the LFL was ignited. The cover and debris were ejected from the LRV as there was no means of either venting the forces or containing projectiles. The four latches that secured the cover all failed in a similar manner with the lock pin failing.

The ejected cover weighed 20 kg and was airborne for approximately 4.12 seconds in which time it struck the overhead contact wire likely slowing the ascent. The cover landed approximately 6 m away from LRV 053 and posed a significant risk to any persons in the area. The chemical hazards could also have caused burns or eye damage if released when the enclosure ruptured.

The addition of deflagration venting such as frangible panels and/or secondary restraints to prevent projections from the roof of the LRV, may have reduced the potential consequences. These measures could however introduce new or different risks and would require assessment.

__________

  1. DOORS - an IBM proprietary software program used for the management of engineering requirements to track the communication, documentation and validation of the requirements.
  2. Software version B10 was not supplied to the SLR fleet.
  3. Alstom (2020). X05-SNY-LRV53-LV BATTERY Static converter SW.
  4. Euro Power Australia (2020). Battery Inspection Report, V1.1
  5. Alstom (2017). Sydney Light Rail - Preliminary Hazard Analysis, RS-PHA-SLR-ALS-D80-RST-SPE-000020_PHA_E_Ap1.
  6. Hoppecke (2016). Citadis X05, LVPS FNC, Sub System Hazard Analysis, D62109-210 ALSTOM Citadis X05_Hazard Analysis.
  7. Hoppecke (2016). Fault tree analysis, LVPS FNC 22,8V / 24V CITADIS X05, D63041-220 ALSTOM X05 FTA_global_NAT_R002.
  8. Hoppecke (2016). Safety recommendations, LVPS FNC 22,8V / 24V CITADIS X05, 62109 ALSTOM X05 Safety recommendations_R003.
  9. Hoppecke (2015). Citadis X05, Technical Offer, spsh-f104514-alstom-citadis-xo5-305aps-rev007.
Findings

ATSB investigation report findings focus on safety factors (that is, events and conditions that increase risk). Safety factors include ‘contributing factors’ and ‘other factors that increased risk’ (that is, factors that did not meet the definition of a contributing factor for this occurrence but were still considered important to include in the report for the purpose of increasing awareness and enhancing safety). In addition ‘other findings’ may be included to provide important information about topics other than safety factors.

Safety issues are highlighted in bold to emphasise their importance. A safety issue is a safety factor that (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.

These findings should not be read as apportioning blame or liability to any particular organisation or individual.

From the evidence available, the following findings are made with respect to the uncontained battery failure and ejection of the battery compartment cover on LRV 053 on 3 April 2020.

Contributing factors

  • Data corruption likely occurred during the uploading of the auxiliary converter software configuration file (version B09) on some light rail vehicles, including for LRV 053. The fault rendered the battery temperature compensation ineffective, resulting in overcharging of the battery system.
  • Overcharging of the battery system depleted the battery cell electrolyte levels and generated excessive hydrogen within the batteries. Overcharging also generated excessive heat within the cells, resulting in the failure of some cell casing.
  • Flammable gases consisting of hydrogen, were released into battery enclosure in the presence of an ignition source. The gases ignited with the force of the expanding gases rupturing the battery enclosure and ejecting the cover from the roof of LRV 053.
  • Neither Alstom’s validation processes nor fault monitoring processes were sufficient to detect the overcharging of batteries prior to the event. [Safety issue]

Other factors that increased risk

  • The battery enclosure while vented was not designed to vent the forces of a deflagration or contain projectiles, resulting in the ejection of the cover.
Safety issues and actions

Central to the ATSB’s investigation of transport safety matters is the early identification of safety issues. The ATSB expects relevant organisations will address all safety issues an investigation identifies.

Depending on the level of risk of a safety issue, the extent of corrective action taken by the relevant organisation(s), or the desirability of directing a broad safety message to the rail industry, the ATSB may issue a formal safety recommendation or safety advisory notice as part of the final report.

All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.

Descriptions of each safety issue, and any associated safety recommendations, are detailed below. Click the link to read the full safety issue description, including the issue status and any safety action/s taken. Safety issues and actions are updated on this website when safety issue owners provide further information concerning the implementation of safety action.

Alstom's validation and fault monitoring processes

Safety issue number: RO-2020-005-SI-01

Safety issue description: Neither Alstom’s validation processes nor fault monitoring processes were sufficient to detect the overcharging of batteries prior to the event.

Sources and submissions

Sources of information

The sources of information during the investigation included the:

  • Alstom Limited Australia
  • Altrac Light Rail
  • Centum Adetel Transportation Solution
  • Euro Power Australia
  • Hoppecke
  • Office of the National Rail Safety Regulator
  • Transdev
  • Transport for NSW.

References

Alstom (2017). Sydney Light Rail - Preliminary Hazard Analysis, RS-PHA-SLR-ALS-D80-RST-SPE-000020_PHA_E_Ap1.

Alstom (2020). X05-SNY-LRV53-LV BATTERY Static converter SW, 10 June 2020.

Australian Standard (2019). AS 3011.1:2019 Electrical installations - Secondary batteries installed in buildings Vented cells.

Australian Standard (2020). AS 2676.1:2020 Installation, maintenance, testing and replacement of secondary batteries in buildings Vented cells.

Brown, William J., et al., (1997). Safety Standard for Hydrogen and Hydrogen Systems: Guidelines for Hydrogen System Design, Materials Selection, Operations, Storage, and Transportation. Office of Safety and Mission Assurance, National Aeronautics and Space Administration. Accessed at: https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19970033338.pdf

DeHaan, John D., et al., (2012). Kirk’s fire investigation, 7th ed, p.28, 525 529.

European Standards. EN 50272 – 2, Safety requirements for secondary batteries and battery installations. Stationary batteries.

Euro Power Australia (2020). Battery Inspection Report, V1.1.

Hoppecke (2014). FNC R, Railway cells, TD-R-Cells_300, V3.0.

Hoppecke (2015). Citadis X05, Technical Offer, spsh-f104514-alstom-citadis-xo5-305aps-rev007.

Hoppecke (2016). Citadis X05, LVPS FNC, Sub System Hazard Analysis, D62109-210 ALSTOM Citadis X05_Hazard Analysis Battery_global_NAT_R002.

Hoppecke (2016). Fault tree analysis, LVPS FNC 22,8V / 24V CITADIS X05, D63041-220 ALSTOM X05 FTA_global_NAT_R002.

Hoppecke (2016). Safety recommendations, LVPS FNC 22,8V / 24V CITADIS X05, 62109 ALSTOM X05 Safety recommendations_R003.

Hoppecke (2019). Alstom Citadis X05, FNC rail battery system operating and installation manual, D62109-300-en08_Manual_Alstom_X05_NAT.

National Fire Protection Agency (2019). NFPA Glossary of terms. Accessed at: www.nfpa.org/Codes-and-Standards/Resources/Glossary-of-Terms

National Fire Protection Agency (2020). NFPA 921 Guide for Fire and Explosion Investigations 2021, p.247 – 270.

Rail Industry Safety and Standards Board (2020). Glossary of Terms. Accessed at: www.rissb.com.au/glossary/

Submissions

Under section 26 of the Transport Safety Investigation Act 2003, the ATSB may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. That section allows a person receiving a draft report to make submissions to the ATSB about the draft report.

A draft of this report was provided to the following directly involved parties:

  • Altrac Light Rail
  • Office of the National Rail Safety Regulator
  • Transport for NSW.

Submissions were received from:

  • Altrac Light Rail
  • Office of the National Rail Safety Regulator.

The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.

Appendices

Appendix A – Electrolyte survey results

LRV No.

Software

Water added (L)

LRV No.

Software

Water added (L)

001

B11

5

031

B11

0

002

B11

9

032

B11

0

003

B11

0

033

B11

0

004

B11

0

034

B11

0

005

B11

0

035

B11

12

006

B11

0

036

B11

24

007

B11

0

037

B11

24

008

B11

0

038

B11

14

009

B11

0

039

B11

0

010

B11

0

040

B11

17

011

B11

0

041

B11

17

012

B11

0

042

B11

17.5

013

B11

20

043

B11

26

014

B11

0

044

B11

1

015

B11

0

045

B11[1]

0

016

B11

0

046

B11[1]

6

017

B11

25

047

B11

0

018

B11

0

048

B11

0

019

B11

0

049

B11

20

020

B11

0.2

050

B11

0

021

B11

8

051

B11

19

022

B11

0

052

B11

20

023

B09

15

053

B09

Failed

024

B09

15

054

B09

25

025

B11

0

055

B11

20

026

B11

0

056

B11

21

027

B11

8

057

B11

20

028

B11

0

058

B11

3.5

029

B11

5

059

B11

0

030

B11

0

060

B11

0

[1] Auxiliary converter software was updated from B09 to B11 prior the completion of the survey for this vehicle.

Note: The addition of electrolyte greater than 15.4 L indicates that the battery cells on average would have been below the minimum electrolyte level.

Source: Alstom, modified by OTSI

Appendix B – Battery cell assessment

Cell No.

Voltage (V)[1]

Voltage (V)[2]

Weight (kg)[2]

Observation

1

1.230

1.236

9.25

Filler plug lifted and cell casing cracked.

2

1.234

1.228

9.30

Filler plug lifted and cell casing cracked.

3

1.235

1.231

9.20

Filler plug lifted and cell casing cracked.

4

1.244

1.240

9.35

Filler plug lifted.

5

1.250

1.246

9.55

Filler plug lifted and cell casing cracked.

6

1.250

1.246

9.40

Filler plug lifted.

7

1.238

1.236

9.40

Filler plug lifted and cell casing cracked.

8

1.245

1.241

9.45

Filler plug lifted.

9

1.247

1.052

9.20

Filler plug lifted.

Cell casing melted and charred between cell 9 and 10.

10

1.245

1.242

9.55

Filler plug lifted.

Cell casing melted and charred between cell 9 and 10 and at the corner of cells 11, 12 and 13.

11

1.245[3]

1.240

9.40

Filler plug lifted.

Cell casing melted and charred between cells 10, 11, 12 and 13.

Casings for cell 11 and 12 were fused and required mechanical separation for inspection.

12

.036

0

9.40

Battery cell short circuit.

Filler plug lifted and melted internally.

Cell casing melted and charred between cells 10, 11, 12 and 13.

Casings for cell 11 and 12 were fused and required mechanical separation for inspection.

13

1.248

1.242

9.50

Filler plug lifted.

Cell casing melted and charred between cells 10, 11, 12 and 13.

Cell 12 was fused to cell 13 and required mechanical separation for inspection.

14

1.253

1.247

9.45

Filler plug lifted.

15

1.245

1.241

9.45

No visible damage and filler plug in situ.

16

1.234

1.229

9.15

Filler plug lifted and cell casing cracked.

17

1.245

1.241

9.45

Filler plug lifted and cell casing cracked.

18

1.230

1.225

9.25

Filler plug lifted and cell casing cracked.

19

1.253

1.240

9.30

Filler plug lifted and cell casing cracked with battery internals exposed.

[1] Voltage recorded on 15 April 2020.

[2] Voltage and weight recorded on 30 April 2020.

[3] Cell 11 was measured but the value not recorded, voltage estimated.

Source: Euro Power Australia and OTSI

Purpose of safety investigations & publishing information

Purpose of safety investigations

The objective of a safety investigation is to enhance transport safety. This is done through:

  • identifying safety issues and facilitating safety action to address those issues
  • providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.

It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.

Terminology

An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.

Publishing information 

Released in accordance with section 25 of the Transport Safety Investigation Act 2003

Published by: Australian Transport Safety Bureau

© Commonwealth of Australia 2020

Ownership of intellectual property rights in this publication

Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.

Creative Commons licence

With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.

Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.

The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau

Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.

Train details

Train 1 details

Train operator:

Transdev

Train number:

LRV 053/054

Type of operation:

Light Rail Vehicle

Persons on board:

Crew – 0

Passengers – 0

Injuries:

Crew – 0

Passengers – 0

Damage:

Substantial damage to roof mounted battery enclosure and battery cells

Train 2 details

Train operator:

Transdev

Train number:

LRV 005/006

Type of operation:

Light Rail Vehicle

Persons on board:

Crew – 0

Passengers – 0

Injuries:

Crew – 0

Passengers – 0

Damage:

Minor panel damage to LRV 005

Train 3 details

Train operator:

Transdev

Train number:

LRV 057/058

Type of operation:

Light Rail Vehicle

Persons on board:

Crew – 0

Passengers – 0

Injuries:

Crew – 0

Passengers – 0

Damage:

Minor panel damage to LRV 058

Glossary of terms

Aesthetic Power Supply (APS) system – The APS system provides power through a ground based system with contact shoes under the intermediate car (IC) that transfers power to the vehicle. The system functions similarly to a third rail, however, the sections imbedded within the ground are only live when the vehicle crosses over that section.

Alternating current (AC) – Electrical current that reverses direction periodically.

Auxiliary converter – A auxiliary converter functions to invert direct current to alternating current. In this case the direct current supply from the overhead contact wire or APS to alternating current for use on the vehicle, as well as a DC output for battery charging.

Battery cell – A functional battery containing an assembly of electrodes, electrolyte, container, terminals and usually separators, that is a source of electric energy obtained by direct conversation of chemical energy.

Battery enclosure – a cabinet or box that provides protection against electrical contact or damage to the battery.

Catenary – In overhead electrification, the uppermost of the two overhead wires mounted above the track and supporting the contact wire.

Contact wire – A bare solid conductor being the lowest of the two overhead wires mounted directly above the track centreline. The pantographs of electric trains press against the underside of this wire and collect the current required by the train.

Combustion – A chemical process of oxidation that occurs at a rate fast enough to produce heat and usually light in the form of either a glow or a flame.

Decomposition – The transformation of a substance into simpler substances or basic elements brought about by exposure to heat, light, or chemical or biological activity.

Deflagration – Propagation of a combustion zone at a velocity that is less than the speed of sound in the unreacted medium.

Detonation - Propagation of a combustion zone at a velocity that is greater than the speed of sound in the unreacted medium.

Direct current (DC) – Electrical current that flows in one direction only.

Drivers display unit (DDU) – The display unit within the drivers compartment to display critical information to the driver.

Explosion – The sudden conversion of potential energy (chemical or mechanical) into kinetic energy with the production and release of gases under pressure, or the release of gas under pressure. These high-pressure gases then do mechanical work such as moving, changing, or shattering nearby materials.

Flammable – A combustible that is capable of easily being ignited and rapidly consumed by fire. Flammables may be solids, liquids, or gases exhibiting these qualities.

Float charging – The period of charging after a battery has fully charged and designed to maintain the battery charge.

Fibre nickel cadmium (FNC) – Hoppecke proprietary technology that refers to the metallised fibre structure within the battery cell electrode.

Frangible panels – A sacrificial panel designed to rupture or open to vent forces created through the combustion process to prevent damage to an enclosure or structure.

Light Rail Vehicle (LRV) – A vehicle used on a light rail system.

Lower flammable limit (LFL) – The lowest concentration of a combustible substance in an oxidizing medium that will propagate a flame.

Joule – The preferred international standard (SI) unit of heat, energy, or work. A joule is the heat produced when one ampere is passed through a resistance of one ohm for one second, or it is the work required to move a distance of one meter against a force of one newton.

Nickel cadmium battery (NiCd) – Storage battery which has an alkaline electrolyte, with nickel oxide as the positive element and cadmium as the negative, this type being used especially as a rechargeable battery.

Pantograph – An apparatus fixed to the roof of electric traction vehicles to draw current from the overhead supply.

Polypropylene (PP-VO) – A plastic polymer of propylene, in this instance with flame retardant characteristics.

Upper flammable limit (UFL) - The highest concentration of a combustible substance in a gaseous oxidizer that will propagate a flame.

Volts (V) – The derived SI unit of electric potential or electromotive force, defined as the difference of electric potential between two points of a conducting wire carrying a constant current of one ampere, when the power dissipated between these points is one watt.

Train Details
Train number
LRV053/054
Train damage
Substantial
Rail occurrence type
Explosion
Rail Operator
Sydney Light Rail
Rail Operation Type
Light Rail Vehicle