Round table discussions: Evaluation of state matura results - analysis and availability of data

How institutions use data from national tests and how the analysis of state matura results can help to improve the quality of education were the questions to which representatives of educational institutions offered answers during the round table organised by the State Matura Project and the Institute for Education Quality and Evaluation (IEQE) on October 19, 2022, in Belgrade.

At the beginning of the meeting, existing practices in the analysis of test results were presented, and it was pointed out that the use value of such analyses is excellent if it is taken into account that the educational achievements of students are one of the most important indicators of the accomplishment of educational goals and outcomes. From the system's perspective, the analysis provides critical data in terms of evaluating the effects of education and monitoring indicators defined at the system level.

The discussion began with an answer to the question of how the institutions use the presented data and whether they still lack some analyses for more efficient work.

The usability of the data is outstanding, the participants agreed. Still, to obtain the data, it is necessary to have a vital institution that deals with data processing, analysis and interpretation. The Institute for Education Quality and Evaluation should and can be such an institution, but it is necessary to strengthen its capacities and resources. The Director of the Institute pointed out that software solutions should be used so that most of the analysis can be obtained automatically, and experts can then deal with interpretation and recommendations for the educational system.

A pedagogical added value that can be measured based on data is essential for vocational schools because the most important thing is to raise student achievements, representatives of the Council for Vocational Education and Adult Education pointed out.

Representatives of the Society of Mathematicians of Serbia suggest including software that would enable the monitoring of entire generations from elementary school to college and early assessment of the possible educational path of each student. The assistant minister for secondary education and adult education confirmed that the strategic goal of the Ministry of Education is to monitor the success of students in the education system, but also to cross-check this data with data on the continuation of the educational path or employment, as this will enable the creation of educational policies based on relevant data.

In the continuation of the discussion, the secrecy of the data from the state matura examination was discussed, that is, which data should be available to professionals and which to the general public.

The previous practice of the Institute for the Education Quality and Evaluation was that school reports - achievements by departments, assignments, and areas within the subject, were seen only by the school that has access parameters for its data segment, explained the head of the Centre for Examinations at the IEQE. When it comes to data, everything must be regulated in advance, so it is necessary to pass the regulations that will prescribe what the state matura reports should be.

The comparability of data has its good and bad sides, the experts of the State Matura Project pointed out. The competitive spirit and motivation for schools are greater, but there is also a danger of students dropping out in schools that do not achieve the best results at matura exams. It is, therefore, more acceptable to know the national average so that each school is compared to that average, district or school administration.

A lengthy discussion in which the participants presented opposing arguments about whether schools should or should not be ranked was brought to an end after the joint conclusion that pedagogical added value is what makes a school good - if a school gets poorer students who have high achievements at the end of their schooling, it means that worked much harder than someone who got better quality students at the start. Still, it takes several years to get such data. On the other hand, the public is not interested enough to understand the nature of education, so the public ranking of schools would be a serious step backwards.

The ranking of schools should be discussed only after ten years of matura implementation when there will be enough data on whether a school from the bottom of the list at the beginning of the measurement has, in the meantime, broken out into the top 30 per cent. In the meantime, experts should handle the data because the system will only improve if what has been done up to that point is improved every year. The education system lacks systematic monitoring of the quality of education. That system should be established, and the data from the state matura examination will be a significant part of that system. All the necessary data are already collected; it is only required to connect them, regulate their processing legally and be careful with the publication of particularly sensitive or compromising data to the broader public, the participants of the meeting concluded.