LCQ14: Examinations Support System of Hong Kong Examinations and Assessment Authority
*************************************************************************************
Question:
It is learnt that the Second Generation Public Examinations Support System (PESS2) designed to support the conduct of the Hong Kong Diploma of Secondary Education Examination (HKDSE), as well as the mobile applications "i-Invigilation (HKDSE)" and "Check-in Smart (HKDSE)" did not run smoothly during the HKDSE, and the use of i-Invigilation (HKDSE) has even been suspended since the 13th of last month. On the other hand, it is learnt that unlike government departments, the Hong Kong Examinations and Assessment Authority (HKEAA) is exempt from the requirement of the Office of the Government Chief Information Officer (OGCIO) that before the launch of large-scale digital services, the relevant systems be subject to additional load tests and stress tests conducted by an independent third party arranged by the OGCIO. In this connection, will the Government inform this Council:
(1) whether the HKEAA will submit a report on the aforesaid incident to the Government;
(2) as it has been reported that the HKEAA has earlier conducted a stress test on PESS2, whether it knows the details of the test, including the test frequency, duration and method, as well as the test result;
(3) whether it knows if the HKEAA has formulated fallback plans for the full implementation of PESS2, such as deploying additional manpower to cope with contingencies; if so, of the details; if not, the reasons for that; and
(4) given that in reply to a question raised by a Member of this Council at a special meeting of the Finance Committee of this Council held on the 19th of last month to discuss the Estimates of Expenditure 2024-2025, the Secretary for Innovation, Technology and Industry advised that the Government was considering extending the aforesaid requirement of the OGCIO to legal entities and public organisations, of the relevant measures being formulated by the authorities and the implementation timetable?
Reply:
President,
The Hong Kong Examinations and Assessment Authority (HKEAA) launched the Second-Generation Public Examinations Support System (PESS2) in normal examination centres for the written examinations of Category A subjects of the 2024 Hong Kong Diploma of Secondary Education Examination (HKDSE) with the aim of streamlining the examination operations and improving the efficiency of the tasks related to the HKDSE. The PESS2, which was developed by a contractor commissioned by the HKEAA, consists of two major mobile applications and one online portal:
(1) "Check-in Smart (HKDSE)": to allow candidates to check examination information and perform self check-in before the examination;
(2) "i-Invigilation (HKDSE)": for invigilators to verify the candidates' identities, confirm attendance, and count the answer scripts collected; and
(3) "Centre Supervisor Control Panel": to enable centre supervisors to monitor the examination operations, confirm the candidates' attendance and answer scripts collected, handle irregularities and communicate with the HKEAA Command Centre.
My reply to the question raised by Dr the Hon Hoey Simon Lee is as follows:
(1) to (3)
Stress test
After completing the system development work for the PESS2 and related applications in December 2022, the HKEAA conducted two rounds of load tests for the system having regard to the operational needs of the 2024 HKDSE that would involve over 50 000 candidates, 2 000 invigilators and some 500 examination centres.
The first load test, which took place in January 2023, aimed to verify the system's capacity for its phased deployment in the HKDSE normal examination centres. During the load test, the contractor used computer simulations to generate a higher workload than would normally be required. This included operating the system in up to 600 centre schools at the same time, with approximately 70 000 candidates expected to use the "Check-in Smart (HKDSE)" application for self-check-in, while around 3 000 invigilators using the "i-Invigilation (HKDSE)" application to perform invigilation duties required. Before conducting the load test, the HKEAA Secretariat (the Secretariat) reviewed the load test plan and relevant design flowcharts submitted by the contractor and provided feedback. Upon completion of the load test, the Secretariat examined the test performance report submitted by the contractor and accepted that the test was completed.
Regarding the HKDSE in April 2023, the HKEAA deployed the PESS2 for the Mathematics (Compulsory Part) and Chinese Language subjects in 48 centre schools for pilot implementation. The system operated smoothly throughout the examination period, without any anomalies being detected.
Thereafter, to fully deploy the PESS2 for all normal examination centres in the 2024 HKDSE, the HKEAA requested the contractor to conduct the second load test in January 2024. As with the first load test, this test plan also used computer simulations to generate a higher workload than would normally be required to verify that the PESS2's capacity could meet the various operational needs of the 2024 HKDSE. This included operating the system in up to 600 centre schools at the same time, with approximately 70 000 candidates expected to use the "Check-in Smart (HKDSE)" application for self-check-in, while around 3 000 invigilators using the "i-Invigilation (HKDSE)" application to perform invigilation duties required. According to the contractor's report, the system performed as expected during the second load test, without any anomalies being detected.
Contingency plan
Prior to the commencement of the 2024 HKDSE, the HKEAA has developed contingency measures in the event of any issues with the PESS2. If the system is unable to function properly, examination personnel should handle the candidates' attendance and the checking of answer scripts manually using paper forms. These paper forms were delivered to centre schools with other examination stationery in mid-March. The details of these contingency measures were outlined in the Centre Supervisor's Handbook and explained to examination personnel during the PESS2 Briefing Session and the Centre Supervisors' Conferences held in February and March 2024 respectively.
Incident report
In the wake of the PESS2 incident, the HKEAA and the contractor immediately conducted initial investigations and in-depth discussions. To ensure the smooth conduct of the subsequent examinations of the 2024 HKDSE, the HKEAA convened a press conference on the evening of April 12, 2024 to announce the suspension of the "i-Invigilation (HKDSE)" application with the tasks related to the examination being handled manually until further notice and account for the follow-up actions for this incident. The usage of the "Check-in Smart (HKDSE)" application by the candidates would not be affected. The HKEAA also submitted a preliminary report to the Government. In addition, the HKEAA has set up a Task Force to investigate in detail the causes of the incident, including the methodology of the load tests, their results and the follow-up work, and devise improvement measures. The Task Force comprises members from the relevant committees of the HKEAA Council, university professors, representatives from the school sector, the Education Bureau and the Office of the Government Chief Information Officer (OGCIO). The Task Force expected to finish the initial investigation in June 2024.
(4) The OGCIO is actively examining appropriate guidelines and technical support to Bureaux/Departments to strengthen their governance of new information technology systems launched by public bodies within their purview, including adequate testing before system rollout. The OGCIO aims to introduce relevant measures as soon as possible.
Ends/Wednesday, May 22, 2024
Issued at HKT 14:30
Issued at HKT 14:30
NNNN