Large-scale learning assessments (LSLAs), conducted at national and cross-national levels, have been on the rise in the last few decades. The evidence generated by such initiatives is assumed to be a key ingredient for improved performance of education systems and for learning achievement. Sustainable Development Goal 4 has further galvanized attention to the centrality of learning data in an unprecedented way, making LSLAs a crucial tool both to monitor learning and to guide policy action.
Yet, this growth has not always led to effective use of the resulting data in educational policy-making and planning. The potential of LSLAs to ensure greater quality and equity in education is conditional on both an effective design and an appropriate use of the resulting data. Admittedly, there is no universal blueprint or magic-bullet approach to ensure that LSLAs adequately serve the multiple purposes attributed to them in all contexts. Arguably, a critical understanding of the limitations and challenges posed by the design of LSLAs and use of data can provide useful insight to overcome limitations and avoid potential risks. The promise of large-scale learning assessments: acknowledging limits to unlock opportunities (UNESCO 2019) critically reviews cross-national and national experiences highlighting both the limits in the design of LSLAs, as well as the potential unintended consequences in the use of results. It argues that a fuller understanding of these concerns - by various stakeholders - can help ensure that LSLAs contribute more effectively to improving learning quality and equity. So what should education planners retain?
Integrating LSLAs into broader assessment systems
The partial and imperfect nature of the data produced by LSLAs call for caution, in particular when such data are given prominence over other sources of information on student learning. Excessive focus or credence given to assessment results can have a misleading and diverting effect on the priorities and behaviors of a wide range of stakeholders. For example, it can result in incentivizing policy action oriented at manipulating figures, encouraging uninformed policy borrowing, and promoting the use of results-based funding schemes. To avoid such risks, it is important that education planners and policy-makers complement LSLA data with other information sources, including the results of public examinations, and qualitative data provided by other administrative data, household surveys or other quality-assurance arrangements.
Disseminating results, but preserving nuance
The influence of LSLAs is also largely determined by their impact on public opinion. As a result, it is important to ensure that results are communicated in a contextualized manner and in a format appropriate to the needs of different audiences. Furthermore, ensuring that complementary sources of information are accessible to the media and the general public, and encouraging and disseminating studies on the determinants of learning or longitudinal trends, may help prevent the propagation of simplistic, fatalistic or misleading messages.
Ensuring country ownership and favoring institutionalization
Ensuring country ownership over both the assessment process and results is an essential condition to ensure that LSLA data are effectively used to guide policy action. This, however, is not always the case. It is therefore important to guarantee that countries have control over their assessment strategy, design and analysis – and that financial and technical support provided for such initiatives encourage capacity development, institutionalization and country ownership.
Preserving the formative role of LSLAs
Many of the risks typically associated with LSLAs are indeed a product of their combination with - or subordination to - high-stakes accountability systems and frameworks. When assessment results have direct and clear consequences for teachers or schools, they are likely to encourage questionable practices aimed at maximizing test scores – including teaching to the test and the narrowing of curriculum to measurable learning domains. LSLAs should therefore not be used for summative purposes. Rather, it is important to privilege their formative and informative roles and to use data to identify priority issues, orient policy-formulation and guide policy implementation.
Effective use of learning data is linked to the design
In the final analysis, the use of learning data is linked to the initial design of assessments. A better understanding of assessment design allows planners to unpack learning data for more focused decision-making and plan for greater efforts to accommodate a more diverse pool of learners – from children with mild to severe cognitive disabilities, to those with little to no command of the language of instruction. This would also allow them to enrich existing evidence on the performance of educational systems, by providing valuable information on teachers, classrooms, parental support and school resources, all of which contribute to improved quality and equity in learning.
The promise of large-scale learning assessments: acknowledging limits to unlock opportunities is available for download in English, French and Spanish (Arabic forthcoming).