LCQ14: Improving the implementation of the Territory-wide System Assessment
**********************************************************

     Following is a question by Dr Hon Lam Tai-fai and a written reply by the Secretary for Education, Mr Eddie Ng Hak-kim, in the Legislative Council today (November 18):

Question:

     The Territory-wide System Assessment (TSA) is designed to gauge students' attainment of the basic competence in the three subjects of Chinese Language, English Language and Mathematics at the end of the three key learning stages (i.e. Primary 3, Primary 6 and Secondary 3). The Government has advised that TSA data help the Government review policies and provide appropriate support to schools, and schools may make use of the school-level report to devise plans for enhancing learning and teaching. In recent years, some stakeholders such as education groups and parents have been greatly concerned about over-drilling students by schools for the purpose of TSA and have requested the Education Bureau (EDB) to abolish the Primary 3 TSA. Moreover, some private schools have decided not to participate in TSA any longer. Some school sponsoring bodies have relayed to me that as the academic standards of some students allocated to the schools are unsatisfactory and TSA questions have become increasingly difficult, it is quite normal that the performance of schools in TSA varies greatly. However, as EDB often takes the TSA performance of a school as an indicator of the school's competence in and quality of education provision without taking into consideration the school's actual circumstances, schools are put under immense pressure. On the other hand, the Secretary for Education has recently indicated that the Coordinating Committee on Basic Competency Assessment and Assessment Literacy has been established to review the content for assessment, assessment items and operational problems of TSA in the coming three months. EDB has also issued updated guidelines to all schools in the territory to request them not to drill students for TSA. The Secretary for Education has reiterated that TSA is presently the only assessment that objectively reflects the basic competencies of students in Chinese Language, English Language and Mathematics, and the decision to abolish TSA should not be taken lightly. In this connection, will the Government inform this Council:

(1) whether it reviewed the effectiveness of TSA in the past three years; if it did, of the details; if not, the reasons for that;

(2) whether it reviewed, in the past three years, what problems had arisen in the course of implementation of TSA; if it did, of the details, as well as the number of such reviews conducted;

(3) of the numbers of public hearings and seminars organised by the authorities in the past three years to gauge stakeholders' views on TSA, as well as the respective numbers of attendees who were parents, teachers, school principals, students and representatives of school sponsoring bodies;

(4) whether it held discussions with individual schools or their sponsoring bodies on the TSA data of the schools in the past three years; if it did, of the objectives of such discussions, and whether it made requests or suggestions to the relevant schools on improving their performance in TSA; if it did, of the details (including the number of schools involved); if not, the reasons for that;

(5) whether it has forwarded the TSA data of schools to external reviewers for reference; if it has, of the details, and whether it has assessed if such a practice will affect the independence, fairness, and impartiality of external reviewers in conducting their reviews;

(6) whether it has examined if the TSA questions, set by the Hong Kong Examinations and Assessment Authority in the past 10 years, were increasingly difficult; if it has, of the details; if not, the reasons for that;

(7) whether it has assessed if the existing TSA questions are overly difficult and the relevant assessment items too complicated and tricky, rendering students unable to answer them; if it has, of the details; if not, the reasons for that;

(8) whether it has invited independent professionals to conduct analyses and comparisons of the TSA questions used in the past years; if it has, of the details; if not, the reasons for that;

(9) of the other uses of the TSA data by EDB, in addition to the provision of such data to schools for enhancing learning and teaching;

(10) whether it has used the TSA data as indicators to judge schools' competence in and quality of education provision; if it has, of the details; if not, the reasons for that;

(11) whether it will conduct a consolidated analysis of the TSA data together with the academic standards achieved by students when they were admitted to the schools, so as to assess the schools' competence in and quality of education provision in a more accurate manner; if it will, of the details; if not, the reasons for that;

(12) whether EDB has ensured that the TSA data of individual schools are kept in strict confidence; if EDB has, of the details; if not, the reasons for that;

(13) whether EDB has issued internal guidelines to specify the time limit for keeping the TSA data confidential and that the data are not meant for other uses (such as being used as the basis for school quality assessment, closure of schools, etc.); if EDB has, of the details; if not, the reasons for that;

(14) whether it will ensure that the right to use the TSA data be surrendered to schools and that the data will only be used by schools as reference to help improve teaching and learning; if it will, of the details; if not, the reasons for that;

(15) whether it will ensure that the various units under EDB will not use the TSA data (i) to assess the performance of schools, and (ii) as grounds for directing schools or their sponsoring bodies to implement reforms; if it will, of the details; if not, the reasons for that;

(16) whether EDB will change TSA into internal assessments conducted by schools and take samples from such assessments to conduct a territory-wide comparison, so as to provide reference benchmark for schools; if EDB will, of the details; if not, the reasons for that;

(17) of EDB's justifications for not considering the abolition of the Primary 3 TSA for the time being;

(18) whether it will consider combining the Primary 6 TSA with the Pre-Secondary One Hong Kong Attainment Test, or abolishing either one of those public examinations; if it will, of the details; if not, the reasons for that;

(19) whether it will consider the suggestion of holding the Primary 3 TSA in alternate years; if it will, of the details; if not, the reasons for that;

(20) whether it will consider inviting more stakeholders to join the aforesaid Coordinating Committee to help enhance the Committee's recognition; if it will, of the details; if not, the reasons for that;

(21) whether it has made reference to and comparison with overseas assessment systems; if it has, of the details; if not, the reasons for that;

(22) given that the Secretary for Education is unable to attend the public hearing on TSA to be held by the Panel on Education of this Council scheduled for the 29th of this month due to personal reasons, whether the Secretary will change his itinerary so as to attend the public hearing and to listen to stakeholders' views directly; if he will not, of the reasons for that; and

(23) whether the Secretary for Education will attend another public hearing to be held by the Panel on Education of this Council scheduled for the 5th and 6th of next month; if he will not, of the reasons for that?

ReplyˇG

President,

     The Territory-wide System Assessment (TSA) is designed to gauge students' attainment of the basic competencies (BCs) in Chinese Language, English Language and Mathematics at the end of Primary 3, Primary 6 and Secondary 3 in order to progress to higher levels of learning. While the territory-wide assessment data help the Government review policies and provide focused support to schools, individual schools can make use of the school-level report to devise plans for enhancing learning and teaching.  

     The TSA is a low-stakes assessment that does not assess and report performance of individual students. Neither does it affect advancement in education nor allocation of school places for admission to Secondary 1. The data is not used for ranking or classifying schools. It is not an index for imposing measures on schools to cease operation.

     In response to the recent discussions on TSA in the community, the Coordinating Committee on Basic Competency Assessment and Assessment Literacy, established in October 2014 under the Education Bureau (EDB) will study and review the operation and different implementation arrangements of TSA in the next few months. At the same time, the EDB will continue to collect views of different stakeholders, including school sponsoring bodies, principals, teachers and parents, etc., through various channels to facilitate the review on TSA. In addition, the EDB updated and issued circulars in late October this year to urge schools to monitor issues related to drilling and homework load. Schools should make clear and accessible to parents their homework and assessment policy as well as the types of homework.

     The reply to the questions raised by Dr Hon Lam Tai-fai is as follows:

(1) to (4) Since the implementation of TSA at P3 in 2004, the EDB and the Hong Kong Examinations and Assessment Authority (HKEAA) have been attending to and collecting views and suggestions on the TSA from various stakeholders, including their concerns of workload for teachers and impact on students and schools.

     The EDB announced in November 2011 that the TSA would be reviewed to examine areas including the implementation arrangements, reporting functions, coverage and question items.  During the course of review, communication with stakeholders was maintained through different means:

(a) In 2012 and 2013, the HKEAA collected opinions from schools in focus groups regarding the TSA assessment design, the benefits of TSA on learning and teaching, the impact on students' learning performance, the implementation of the enhancement measures, and the alternate-year arrangement of P6 TSA. Participants included principals and vice principals of primary and secondary schools, primary school curriculum leaders, panel heads and subject teachers. Schools which voluntarily joined the P6 assessment reflected that the "Item Analysis Reports" were of reference value for them to understand the learning needs of students and plan the curriculum.  

(b) In 2013, with a view to facilitating thorough deliberations among different stakeholders and soliciting their views and suggestions on other possible enhancement measures for the TSA, the EDB consulted and met with the primary and secondary schools councils, the Committee on Home-school Co-operation, the Federation of Parent and Teacher Associations, the Hong Kong Professional Teachers' Union, the Education Commission, the Curriculum Development Council, as well as the TSA Concern Group.

     Consultation work and views of stakeholders were presented in the paper for the meeting of the Panel on Education on January 13, 2014.

     Taking into consideration the views of stakeholders collected through consultation, the EDB announced the results of the review on TSA and the related enhancement measures (including not disclosing attainment rates to primary schools, administering P6 TSA in alternate years and removing the TSA from the Key Performance Measures for primary schools, etc.) in April 2014, and continued to maintain close communication with all stakeholders. In support of the TSA enhancement measures, a series of seminars were held since April 2014 to inform schools of the implementation of the enhancement measures and listen to the views of the frontline personnel. In August this year, the HKEAA organised focus groups to consult primary school teachers who served as markers for their views on the enhancement measures for the TSA. The majority of them welcomed the enhancement measure of launching the interactive platform for online reporting and considered it of value for providing feedback to learning and teaching.

     In order to enhance understanding of schools regarding policies and arrangements for the TSA, the EDB arranged a seminar on October 30, 2015 for principals, vice principals, curriculum leaders and teachers of primary schools and representatives of school sponsoring bodies. Around 380 participants attended the seminar to share the effective use of assessment data to enhance student performance, making use of the interactive reporting platform to facilitate curriculum planning, designing tasks for assessments as well as making selection of quality textbooks and curriculum resources. Suggestions made by participants reflected that they recognised the merits of the TSA reports in helping them to identify students' strengths and weaknesses, develop curriculum and make adjustments to learning and teaching. Other than sharing their positive views on the enhancement measures implemented in 2014, the participants also expressed their opinions on further enhancement measures on the design of assessment papers (e.g. number of items, length of texts), the coverage and the enhancement of professional support for teachers.

     Since curriculum, pedagogy and assessment should be well aligned, schools should make effective use of different assessment information including data from tests, examinations, assignments, students' classroom performance and TSA data etc. to adapt the curriculum, improve the effectiveness of learning and teaching as well as provide constructive feedback to students. Through different channels and means of communication including school visits, inspections, daily contact etc., the EDB has been providing advice and support to schools on various aspects of school operation and learning needs of students so as to promote schools' self-improvement and sustainable development.

(5) Under the School Development and Accountability (SDA) framework, the EDB promotes evidence-based school self-evaluation (SSE) complemented by External School Review (ESR) to enhance the effectiveness of SSE for schools' self-improvement and sustainable development. While schools need to make use of different information and data to conduct own SSE, external reviewers also make reference to such information to review different aspects of school performance from a holistic perspective. In April 2014, the EDB announced the results of the review on TSA and related enhancement measures. The TSA has been removed from the Key Performance Measures for primary schools. However, in conducting SSE, schools can still make use of the TSA item analysis report as one of the references in reviewing student performance. In the process of ESR, the EDB officers will conduct professional dialogue and discussions with school personnel on how schools use the assessment information and data to feedback learning and teaching and to enable improvement in student learning.

(6) to (8) The assessment items of TSA are developed by the TSA Moderation Committee for each subject at each level as established by HKEAA in accordance with curriculum documents, curriculum guides and Basic Competency documents. The TSA Moderation Committee consists of an academic from the tertiary sector, curriculum experts, local teachers (including native-speaking English teachers for English Language), managers from the HKEAA and curriculum officers from the EDB, etc. Meetings are conducted regularly to deliberate on the items in terms of the level of difficulty, item types and marking criteria to ensure that the assessment coverage is in line with the curriculum and related to students' learning experience, and that the items are pitched at BC levels and not excessively difficult.

     After the release of the TSA results, the HKEAA will arrange meetings for the TSA Paper Review Focus Group for each subject and each level to review the assessment content, level of difficulty, types of items, rubrics of assessment and the overall percentage of correct responses (such as rate of correct responses being too high or low), etc.  Their comments are forwarded to the Moderation Committee for the development of the TSA papers in the following year. To keep the review objective, members of the focus groups, where possible, do not come from the Moderation Committees.    

(9) to (11) and (15) The Government can make reference to the TSA data to formulate related policies and measures, such as developing appropriate learning and teaching resources to cater for learner diversity, promoting "Reading to Learn" so as to enhance reading culture in school, etc. At school level, the TSA data can help teachers understand students' performance against BCs and identify areas for improvement for provision of timely support, which would enable enhancement of learning and teaching strategies, and improvement on students' learning effectiveness. The EDB seeks to understand different aspects of schools by adopting a holistic approach and TSA data is only one of the many references in the process. It is not an indicator for evaluating the effectiveness of school operation and school performance. Neither will it form a basis for imposing measures on schools or school sponsoring bodies for reform.

(12) to (14) The TSA school level report is a confidential document.  Viewing of related data is confined to the school management personnel and teachers concerned. Schools should agree and follow the protocol for using the TSA data. The protocol requires that schools keep confidential all TSA data. They should not release all or part of the assessment data via any means (e.g. printed matter, school websites, and publications). The protocol also states clearly that the TSA data provided are reference materials for schools to devise learning and teaching plans to meet the school development needs.

     We have been stressing that TSA is a low-stakes assessment which aims to provide schools with assessment data for teachers' reference to facilitate curriculum planning and adjust teaching and learning plans. As such, the EDB will not use the TSA data as a basis for imposing measures on schools to cease operation. Evaluation of school performance is conducted according to the framework and guidelines set for the purpose. Reference is made to a range of data rather than the TSA data as the only source of reference. In 2014, we removed the TSA from Key Performance Measures for primary schools to dispel worries of schools.   

(16) As part of our continual dialogue with different stakeholders, we have been soliciting their opinions on conducting the TSA using a sampling approach. There was no consensus on the approach to be adopted. While supported by some stakeholders, the proposed approach was deemed ineffective in alleviating stress on schools and fairness was also a concern.

     In addition, regardless of the particular sampling method adopted, the school level report will no longer be available. Teachers will lose access to objective assessment data which could otherwise be used as a basis for adjusting curriculum planning. There is a need for more research and consideration as far as the adoption of a sampling approach is concerned. The issue will be considered in the current review.

(17) and (19) On different occasions where stakeholders were consulted, there was divided opinion on the implementation plans for P3 TSA (e.g. cancellation, alternate-year implementation, adoption of sampling approach, etc.). No consensus has been achieved. In the current review, we will carefully examine the feasibility of different implementation arrangements to ensure that timely feedback can be provided to schools and students.

(18) In weighing the proposed merger of the P6 TSA and Pre-S1 Hong Kong Attainment Test (Pre-S1 HKAT), we have to take into consideration the objectives, essence, contents and operation of the two assessments. First, unlike the TSA which provides information on students' attainment of BCs in the three subjects of Chinese Language, English Language and Mathematics, the Pre-S1 HKAT only provides information on the relative strengths and weaknesses of students in these three subjects to facilitate secondary schools to plan in advance the learning and teaching of their S1 students. A large number of P6 students participate in the Secondary School Places Allocation (SSPA) System each year. While stakeholders in general consider overreliance on academic performance at the stage of basic education (including having a high-stakes examination) for the purpose of admission to secondary schools and hence possible distortion of the primary school curriculum should be avoided, they appreciate that a mechanism is required to differentiate the performance of P6 students to determine the order of allocation when the number of students choosing a school exceeds the S1 places of the school. Having balanced the views of various stakeholders, we have adopted the Pre-S1 HKAT which seeks to differentiate studentsˇ¦ performance in schools. Besides, the Pre-S1 HKAT is conducted in a secure manner to scale the internal assessment results of the following cohort of P6 students progressing to S1 with a view to minimising incentive for drilling.  

     When the Working Group on Review of P6 Assessment Arrangements explored the feasibility of different modes of operation for the TSA from 2010 to 2011, it pointed out that if the different objectives of the TSA and Pre-S1 HKAT were to be attained, the assessment upon the proposed merger, as compared with the existing Pre-S1 HKAT, would have a much larger quantity of assessment items with more complicated contents. In addition, more rigorous pre-tests and careful calibrations, etc. would be required, which would involve a longer period. As such, the proposed merger would fail to alleviate the pressure arising from the TSA of some primary students. Worse still, it might affect the stability of the SSPA System.

(20) The EDB established the Coordinating Committee on Basic Competency Assessment and Assessment Literacy (the Committee) in October 2014. The terms of reference of the Committee are to advise on the direction relating to the development of the Basic Competency Assessment project (including the Territory-wide System Assessment and Student Assessment) as well as the enhancement of assessment literacy in schools. The Committee is composed of academics from the tertiary institutes, principals and teachers. We will consider the participation of parent representatives. Review meetings are scheduled in the next few months to consider and discuss different implementation arrangements.

     In the meantime, the EDB will continue to collect views from stakeholders including school sponsoring bodies, principals, teachers and parents, etc. to facilitate the review on TSA.  

(21) We have been making reference to assessment practices in other countries. Care has been taken to observe the differences in national contexts, curriculum requirements and academic structure of a particular assessment. Direct comparison or replication may not be appropriate.  However, these system assessments share the same objective, which is to provide an objective and quality assessment tool. Such tool can provide objective information to facilitate appropriate allocation of education resources, ensure teaching quality and equal learning opportunities for students. A more important mission is for providing timely feedback to improve learning and teaching effectiveness. With learning strengths and difficulties identified, schools can implement early facilitations for students so as to lay a better foundation for their progressing to the next stage of learning.

(22) and (23) The Secretary for Education (SED) values and holds in high regard opportunities to communicate with different stakeholders through various channels. SED seeks out for opportunities in his schedules to keep in touch with different parent groups to understand their views. In attending school functions such as speech days, SED holds discussions with parents. He also seizes the opportunities of meeting parents when he participates in large-scale events.

     SED and colleagues of the Bureau are willing to keep up the communication with all stakeholders to understand their views. As no prior notice regarding the date for the public hearing has been given to the SED office, it is unfortunate that SED has not been able to accommodate the meeting. The EDB will arrange personnel appropriate for attendance. The EDB has informed the Secretariat of the Panel on Education that if necessary, SED would arrange to attend a meeting on December 13.

Ends/Wednesday, November 18, 2015
Issued at HKT 18:49

NNNN