What can we learn from large-scale surveys about our students learning of maths?

Large-scale surveys on Maths and Science learning (such as OCSE-Pisa, IE-TIMMS, TIMMS Advanced and the INVALSI in Italy) have a strong influence on public opinion in all countries and, in a top-down process, on decisions by policy-makers and administrative stakeholders, organization of the education system, official curricula, actual curricula, and the didactic choices of teachers. This process is activated principally by a mechanism of comparison and ranking which is implicit in the published results of these surveys – a mechanism which induces effects which are not always positive (such as teaching to test). This study sets out to show, with some examples from the ongoing research, how it is possible to analyse macro-phenomena revealed by the survey results with conceptual tools of mathematics education, in order to look beyond the statistical data of individual students’ performances or of the sample group as a whole. The quantitative analytical tools used in processing the information collected in these surveys can be used to suggest valuable clues in understanding the nature and origins of common misconceptions and difficulties, and how these are linked with didactic practices. The first case that we consider regards the answering of questions which highlight a strong difference between male and female students (the so-called gender gap): which questions are these and why? The second case is the analysis of some INVALSI questions through which it is possible to quantify a well-known didactic phenomenon: the “loss of meaning” in algebra. The second case regards the answering of questions which highlight a strong difference between male and female students (the so-called gender gap): which questions are these and why? The third case that we present shows how it is possible to study how students’ behaviour (and their ability to find problem-solving strategies, for example) is influenced by the layout and wording of the question. These and other examples show how mathematics education can greatly benefit from the use of mixed methods (quantitative/qualitative) in surveys and research.