Round table discussions: How to ensure the quality of exam materials

Continuing the cycle of round tables dedicated to discussions on more efficient implementation of the state matura exam, the State Matura Project organised an expert meeting on November 16, 2022, on the topic "How to ensure the quality of exam materials?". At the meeting, a serious discussion was held to obtain as much helpful information as possible, which, after the completion of the project, could remain practical guidelines for the education system.

The round table was planned and organised after the October visits to the Dutch National Institute CITO and the national examination centres in Slovenia and Croatia, to point out the good practices of institutions with decades of experience in testing and evaluation in education. Participants also discussed the procedures that have already been implemented in our system and those used during the piloting the state matura exam.

Preparing exam materials within the CITO Institute has eight steps similar to the steps through which exam materials for piloting the state matura exam were developed. The procedures in Slovenia and Croatia are almost identical to our practices during the preparation of the final exam at the end of primary education, representatives of the Centre for Examinations from the Institute for Education Quality and Evaluation pointed out.

When it comes to vocational education, there are fewer examples of good practices, considering that vocational matura at the national level is rare in other education systems. However, the representatives of the Institute for the Improvement of Education believe that the system that has been functioning so far at the school level is well-established and that there will be no problem organising vocational matura at the national level. In addition, the knowledge about the preparation of test materials acquired through the State Matura Project is significant and will significantly facilitate the standardisation of testing at the end of secondary education.

In the introductory part, the representatives of the State Matura Project shared with the participants the conclusions and recommendations that were reached through cooperation with the working groups to develop exam materials. One of the key recommendations is that when preparing for a high-risk exam, such as the state matura exam, one should stick to simplicity because the test is not a test of creativity for authors, but should be adapted to the needs of students and the testing process itself.

The discussion opened several topics, including the implementation of standards, the content of textbooks and the capacity building of all relevant participants. The participants agreed that the basis for preparing test materials is achievement and qualification standards. Teaching and learning plans, programmes, and textbooks give another degree of legitimacy to tasks. The competence approach is a novelty that could significantly contribute to the quality of education but also solve the problem of different contents because the standards could be checked based on competence, regardless of whether particular contents exist in textbooks. This approach, however, must be introduced gradually through testing because all participants in the process must accept it. The development of exam culture will eventually affect the teaching itself.

Cooperation with universities and the participation of faculty representatives in working groups are crucial to ensure the quality of exam materials and build the trust of higher education institutions. Selecting working group members and reviewers should be standardised and more functional.

Quality is the most critical concept in the state matura exam. It is a big job to establish, train and equip the Examination Centre because the verified and safe matura quality guarantees that the matura exam will be accepted and all target groups will be satisfied.  

As part of the State Matura Project, a guide for working groups and a code of ethics have been prepared, and these documents can be used as a starting point for the work of working groups in the future.

In the final part of the meeting, the participants summarised the recommendations and made suggestions for more efficient work of the working groups and ensuring the quality of the exam materials. It is crucial to connect the entire system; exchanging experiences among working groups is also essential, as it will provide the uniformity of exam materials and enable horizontal learning. The capacities of all participants in the process must be strengthened. In this sense, the more significant involvement of professional societies in the state matura process is a potential that should be used. Continuous promotion of the state matura exam and systemic support to schools is necessary to build trust in the quality of exam materials and the entire process of the state matura.