Making visible subjects in the landscape, and directing their activities through shaping their mentalities is, as Miller and Rose (1990) point out, an important object of governing. And governing education in Mexico through large scale assessments is at the heart of Israel Moreno Salto’s book – Harmony and Cacophony in Large-scale Assessments in Education. What’s new, you might ask, after more than three decades of work that has graced the pages of books, journals, and blog posts, and rested on the heaving bookshelves of libraries around the world?
Drawing on the seminal work of Foucault and his governmentality lectures, Miller and Rose (1990) were some of the first academics to direct our attention not simply to the institutions of government, but to the way in which governing emerges out of the choreographing of an assemblage of discourses, policy instruments, strategies, and tactics, that regulate the social and economic life of a population. Just as we don’t view an orchestral piece through the activities of the conductor, nor ought we view the regulation of citizens through the institutions of government. Hardly surprising, Foucault’s contribution to governing has been powerfully taken up by a wide range of researchers seeking to understand how governments in national and subnational settings, state like actors like the European Commission, or multilateral organizations like UNESCO and the OECD, govern contemporary life.
This is particularly so in education, where multilateral institutions, such as the Organization for Economic Cooperation and Development (OECD), The World Bank (WB), and the United Nations Educational, Scientific and Cultural Organization (UNESCO), have taken on an expanded agenda regarding directing national education systems around the globe. Where once the OECD’s (now infamous) large-scale assessment, the Program for International Student Assessment (PISA), collected data from its member states, now a suit of assessments takes in other populations from adults to early years learners, it ranges well beyond its member states, and its areas of assessment include not only science, mathematics, and literacy, but also critical thinking, well-being and global competences.
Paralleling the spectacular rise of the OECD over the past three decades has been a program of research, much of it critical of the OECD’s instrumental
This is not to suggest that such studies are wrong, or even limited, but they do deflect attention away from historicizing the challenge of governing when the governor has limited juridical powers over a population, or the value of empirical studies of policy enactment rather than policy production to shed light on the governed, and their material practices. To riff off the metaphor in the title of Moreno Salto’s book, a musical score might look ordered and symphonic on the page, but what does it sound like when the business of playing gets under way? Whilst for the conductor a cacophony would be a disaster, it nonetheless is a distinct possibility, if a particular set of circumstances were to eventuate. Getting it right takes more than the skill of the conductor; it requires the whole ensemble to come together with their various instruments, notes, and nods that register engaging.
So what might historicizing governing by numbers tell us. One is that this is not a new instrument of government; it became possible through the collection of census data, and was perfected, for example, by the highly centralized French state over several centuries. And indeed, the go to, and well cited work, The Politics of Large Numbers: A History of Statistical Reasoning (2002) was written by France’s chief statistician, Alain Desrosieres.
Second, that governments in systems (especially Federal) with limited constitutional powers to direct subnational governments have turned to the use of indicators, big data and statistics to govern in the absence of sovereignty. In the late 1950s the United States faced just such a challenge. The Russian’s launch of Sputnik galvanized the US Federal government into action, allocating funding to states across the USA to invest in national interest initiatives that included large investments in science (labs, curriculum), languages and technology. But given the Federal government could not demand that the various recipient states be accountable to the Federal government, it was challenged to come up with a set of tools to steer a system of accountability to the Federal level. Its development of a national evaluation center that collected data around sets of indicators set of stage for the OECD, who would learn from and import this model as it established itself. As numbers, statistics oozed objectivity, and with objectivity came trust (Porter, 1995). Legitimacy is then secured in the face of limited or little sovereignty.
Third, in the early years of the OECD indicators and statistics were deployed for the purposes of directing development, mindful of an alternative model
But what does governing really look like down in the trenches, or to continue with our musical metaphor, in the orchestra pit? What do those who the OECD says it is providing data for to enable teachers to inform their classroom practices, know about large scale assessments, and how does it shape what they know and do? This is an important empirical question and one that has been neglected in the work on large scale assessments. Answering this question could indeed raise all kinds of questions; is the outlay by governments on rounds of PISA data collection, for example value for money? What kind of moral economy of value and worth is set in motion? Could the money be better spent elsewhere? Is it simply a question of ‘lost in translation’ – between the bars and lines of data collection, outcomes to practices, and what crucial steps are missing, if indeed a school or system believes that large scale assessments have value? Do large scale assessments end up distorting education systems aims and goals by encouraging quite a narrow focus on specific skills? Might the outcomes present the large-scale assessment industry with an inconvenient truth: that the assessments get limited traction on practices in classrooms and have amplified effects at the level of politics? After more than two decades, large scale assessments clearly have not exhausted the kinds of questions we need more research on, and reflections regarding why it matters.
This book by Israel Moreno Salto takes us on a very important journey that overcomes some of the concerns I outlined above. To begin, it is a corrective to a policy as discourse only analysis. This is a theoretically informed empirical enquiry that uses different methods to get access to different actors in the education policy and practice world in Mexico. Second, by bringing in multiple
In all, this is an incredibly important book. It is rare in its conception, articulate in its arguments, and has a depth of insight that adds to its virtuosity. And from it we have not only a deep and rich understanding of education in Mexico, but as a rigorous piece of work on large-scale assessments in Mexico, it ought to be on the desks of all policymakers.
References
Desrosieres, A. (2002). The politics of large numbers: A history of statistical reasoning. Harvard University Press.
Grek, S. (2009). Governing by numbers: the PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37.
Miller, P., & Rose, N. (1990). Governing economic life. Economy and Society, 19(1), 1–31.
Porter, T. (1995). Trust in numbers. Princeton University Press.
Robertson, S. L. (2022). V-charged: Powering up the world-class university as a global actor. Globalisation, Societies and Education, 20(4), 423–424.