***********************************
Following is a question by the Hon Charles Mok and a written reply by the Secretary for Transport and Housing, Professor Anthony Cheung Bing-leung, in the Legislative Council today (May 28):
Question:
It has been reported that on the 27th of last month, a router in the data network transmission system (transmission system) of the monitoring and communications system (signalling system) of the MTR East Rail Line (ERL) malfunctioned, causing a 36-minute suspension of the service of the entire ERL. The MTR Corporation Limited (MTRCL) stated on the next day that replacement of the router in question had been arranged. Separately, the backup transmission system failed to activate automatically when the incident occurred and the trains had to switch to manual operation mode before train service resumed normal. There are views that the failure of MTRCL to explain in detail the causes of the system malfunctioning has caused people's concern about the reliability of MTR's signalling system. Also, the contingency measures (including the emergency notification mechanism and the connecting transport arrangements) taken by MTRCL during the incident have aroused dissatisfaction among passengers. In this connection, will the Government inform this Council:
(1) whether it knows the progress of MTRCL's investigation into the aforesaid incident, the causes of the malfunctioning of the system and components concerned, if the causes of the aforesaid incident involved components other than the router, and the reasons for the failure of the backup system to activate automatically (and whether system software errors were involved);
(2) as it has been reported that MTRCL requested the Australian supplier of the router in question to send personnel to Hong Kong to investigate the aforesaid incident, whether it knows the scope, timetable and progress of the investigation;
(3) whether it knows the possible factors that might have led to the double failure of both the transmission and backup systems of MTR; as MTRCL indicated on the 28th of last month that replacement of the routers in question had been arranged, the reasons why failures of the signalling systems still occurred on the 29th of last month and also on the 2nd and 3rd of this month; whether such failures were related to the two primary routers or other components of the integrated communications and control system; whether software and hardware updates have recently been performed for the system concerned or its sub-systems to align with the development needs of other railway lines (eg the Shatin to Central Link); if such updates have been performed, of the details, and whether MTRCL has assessed the additional risks thus caused to the system, and has any plan to restore the software to its original version;
(4) whether it knows if MTRCL has assessed the risks of the signalling systems of its railway lines being attacked, including the possibility of sabotage or online attacks against the relevant components; if MTRCL has assessed, of the outcome; whether MTRCL has put in place a contingency plan to ensure the continued provision of safe train service in the event of an attack on its signalling systems;
(5) as it has been reported that, for urban railway lines, the signalling units are configured in a "de-centralised" way that the signalling units of various stations are connected to form a network, which allows individual station units to take control when the Operations Control Centre is unable to regulate the train service and ultimately allows the integrated backup panels in individual stations' control rooms to alter the track directions through the circuit switch in order to let trains in and out one by one in the event that the signalling units suffer a network connection failure due to a communication breakdown, whether it knows if MTRCL adopted, in the past five years, such an approach in maintaining the train service; if MTRCL did, of the details;
(6) as it has been reported that, for non-urban railway lines, the signalling units adopt a "centralised" configuration under which MTRCL can only activate the backup computer system as remedy when the Operations Control Centre is unable to regulate the trains, whether it knows if MTRCL has reviewed the impacts of the "centralised" configuration on the reliability of the existing railway transport system; if MTRCL has reviewed, of the details;
(7) as I have learned that subsequent to a number of signalling failures which occurred on ERL, additional backup systems were installed for ERL but the one commissioned in 2010 failed during the aforesaid incident, whether it knows if MTRCL has any measure in place to ensure that the additional backup systems can operate when needed;
(8) as I have learned that, before the rail merger, the Kowloon-Canton Railway Corporation could shift to the "temporary block post working" by using aids eg flag signals, light signals and direct telephone lines etc. to help identify the locations of trains in order to maintain railway service when ERL's signalling system failed, whether it knows if MTRCL can use this method to provide railway service at present; if it can, whether this method was used during the recent railway incidents; if MTRCL no longer uses this method, of the reasons and justifications for that; and
(9) whether it has plans to engage independent experts and academics to conduct thorough investigation into the aforesaid incident and publish the results in order to allay public concerns; if so, of the details?
Reply:
President,
The operation of MTR train service relies on three major systems. They are (i) signalling system; (ii) central monitoring and communications system; and (iii) console of the monitoring and communications system for various railway lines.
Signalling system
-----------------
Installed along railway tracks and inside equipment rooms along the railway lines, the signalling system controls train operations. Its fail-safe design will bring trains to an automatic halt once irregularities are detected, ensuring that trains are kept at a safe distance apart.
Central monitoring and communications system
--------------------------------------------
The Tsing Yi Operations Control Centre (OCC) captures the operation status of the signalling system via the data network transmission of the central monitoring and communications system. The functions of the data network transmission are mainly performed vide the routers and the concentrators.
Console of the monitoring and communications system
----------------------------------------------------
The monitoring and communications system of various railway lines is installed inside equipment rooms along the railway lines; while the console of the system is set up at the Tsing Yi OCC. The console is responsible for monitoring train operations, communicating with train captains or stations, etc, and adjusting train service as and when necessary.
Incidents occurred on April 27 and 28 as well as May 1 and 2 this year involved the operational instability of a router and a concentrator of the data network transmission of the monitoring and communications system, harddisk of the train control system of the signalling system, and a trackside encoder of the East Rail Line (EAL) respectively. These incidents had an impact on the railway service, causing delays of 36 minutes at most. Although railway safety was not affected, the Government has expressed grave concern over the recent spate of malfunctioning of these systems and requested the MTR Corporation Limited (MTRCL) to conduct thorough investigation into the causes of the incidents and take proper follow-up actions accordingly. MTRCL has responded proactively and engaged a number of independent experts to provide assistance. The Electrical and Mechanical Services Department (EMSD) together with the Transport Department (TD) will also actively participate to ensure safety and reliability of the railway service.
My reply to Hon Charles Mok's question is as follows:
(1)&(3) The service disruption of EAL on April 27, 2014 was caused by the unstable performance of one of the routers of the data network transmission, which forms part of the monitoring and communications system of EAL at the Fo Tan Railway House. As the standby router is designed to be activated automatically only at the complete malfunctioning of the original router, automatic activation was not successful as the original router was only unstable instead of malfunctioning completely on the day of the incident. As a result, the operation of the data network transmission was affected. After switching off the original router and activating the standby router manually, train service resumed normal. The original router was replaced that night after train service. Following the incident, MTRCL installed a data analyser to monitor the stability of data network transmission. Should there be any instability of the data network transmission, the problem can be detected and rectified as early as possible.
Another incident on April 28, 2014 only affected railway service for 12 minutes. Unlike the case involving an unstable router on April 27, this incident was caused by the malfunctioning of one of the concentrators of the data network transmission. Subsequent to the replacement of the malfunctioned concentrator on the day of the incident, the data network transmission resumed normal operation. The two incidents had no adverse safety impact on train service.
The cause of the EAL incidents on May 1 and 2 was different from that of the cases on April 27 and 28. The May 1 incident was caused by the failure of the harddisk of the major train control system of the signalling system of EAL. As soon as the standby train control system was activated automatically, the signalling system resumed normal operation. The incident on May 2 was caused by the failure of the trackside encoder (a component of the signalling system). After the relevant component was replaced, the signalling system resumed normal operation. The two incidents caused service disruptions of 10 and 33 minutes respectively.
(2) Upon MTRCL's request, the supplier of the data network transmission has sent its staff to Hong Kong to investigate into the incidents on April 27 and 28, 2014. Relevant data and records have been collected for further analysis. The investigation has been completed which confirms that the two incidents were caused by the malfunctioning of one of the routers and one of the concentrators of the data network transmission respectively. In collaboration with the supplier, MTRCL will conduct further inspection and formulate necessary follow-up measures based on the findings, with a view to ensuring stability and reliability of the data network transmission. EMSD together with TD will also participate in the inspection to ensure that it is in order and the findings are followed up properly. The outcome of the review and implementation of follow-ups will be made public.
(4) & (9) The signalling system of various railway lines (including EAL) is an independent system. It is believed that outside parties are unable to enter the system to launch an attack. In fact, MTRCL conducts risk assessment on the security of the signalling system from time to time and formulates corresponding measures to prevent malicious attacks. MTRCL has engaged an independent expert to carry out comprehensive analysis and assessment on the signalling system of EAL and make recommendations for improvement. Furthermore, MTRCL has engaged an independent consultant to evaluate the protection of the interface between the data network transmission and the signalling system. The evaluation is scheduled for completion in mid-2014. Once completed, the evaluation report will be submitted to EMSD. If recommendations for improvement are put forward in the report, EMSD together with TD will review the appropriateness of the proposed measures and oversee the implementation of necessary measures. Result of the evaluation and implementation of the measures will be made public.
(5) Tsuen Wan Line, Kwun Tong Line, Island Line, Airport Express, Tung Chung Line and Tseung Kwan O Line (ie the MTR network before rail merger) currently make use of the "Integrated Backup Panel" for "station control". In case of failure in central control and monitoring by OCC, train operations can still be monitored via the local control mode by stations as backup. As for Disneyland Resort Line, both the control for train operations and its backup control are situated at Sunny Bay Station (Tsing Yi OCC as central monitor). In the past five years, MTRCL monitored train operations via the local control mode on several occasions and none of them hampered the normal and safe operation of train service.
(6) Ma On Shan Line and West Rail Line use "station control" as backup operation, same as the railway lines of the pre-merger MTR network. But for EAL, a central-controlled backup system is used for backup operation. The two modes are of different design. The signalling system of EAL will be upgraded in tandem with the Shatin to Central Link project, providing "station control" as the backup operation mode.
(7) Currently, EAL has in place another backup console activated manually as support. Installed in 2010, this backup console enables OCC to have direct access to the signalling system without going through the data network transmission if data cannot be received or transmitted from the signalling system due to failure of the data network transmission. On April 27, 2014 when the data network transmission failed, the backup console in OCC was not activated because the data network transmission was rebooted manually and data transmission resumed before its activation. The failure on the day was not related to the signalling system. In fact, MTRCL tests the backup console regularly to ensure its normal operation.
(8) At present, if the signalling system of EAL fails, the "Pilotman" working mode (similar to the "temporary block post working" mode as mentioned in the question) can be used to maintain train service. During the period, OCC will first deploy personnel to the railway section not receiving signals to secure the points along the tracks for trains to pass through the affected section under the manual command of a pilotman. During the incident on April 27, 2014, MTRCL considered switching to the above working mode before the failure of the signalling system could be ascertained. Since data transmission restored after rebooting the data network transmission, the "Pilotman" working mode was not used eventually.
Ends/Wednesday, May 28, 2014
Issued at HKT 12:30
NNNN