In an attempt to improve incentives for university teaching, the UK government recently introduced a Teaching Excellence Framework. This column uses data on contact hours and class size to measure how much teaching students receive at university. The results reveal large variation across disciplines, and even larger variation within disciplines.
What is the fundamental purpose of a university education? Most of us would probably agree that we aim to provide our students with the ability to ‘think things through’. Asked to define this phrase, we might list a set of skills that include some or all of the following: logical and critical thinking, clear writing, numeracy, and some understanding of the challenges involved in assessing of evidence. In the UK it has been taken for granted that because we punch above our weight in terms of the number of ‘world class’ universities, it follows that the education they offer is also world class.
Drawing on US data, Roksa and Arum (2011) raise doubts about the value of a college education. When their sample of students, enrolled at a wide range of colleges, took the Collegiate Learning Assessment (CLA) (a standardised test) the results made depressing reading: “a high proportion of students are progressing through higher education today without measurable gains in critical thinking, analytical reasoning, and writing skills as assessed by the CLA”.
The explanation given was that students spend little time studying. The authors provide no definitive explanation, but one obvious candidate would be that “[students] are not being asked to do much by their professors. Fifty percent of students the sample reported that they had not taken a single course the prior semester that required more than twenty pages of writing. One-third of students did not take a course the prior semester that required on average even 40 pages of reading per week.”
Like most research in higher education, the primary focus of Roksa and Arum is the behaviour of students. However, their findings hint at the possibility of complementarities between how much time we spend teaching and how much time our students spend studying.
How much teaching do university students receive?
To investigate the teaching side of the relationship, we undertook (what we believe to be) the most detailed examination of the teaching arrangements at UK universities since the Robbins Report in 1963. In our paper, we use the rights contained in the Freedom of Information Act, to collect data from 67 universities about the teaching of economics, history, and physics (Huxley et al. 2017).
We propose a novel way of measuring how much teaching students receive at university, weighting contact hours by teaching intensity (the ratio of teachers to students). The metric, TEACH (Total Equivalent Adjusted Contact Hours), makes it possible to define a unit of teaching equivalent to an hour of one-to-one contact with a staff member. This allows us to compare teaching delivered in very different ways, for example a student receiving only small group tutorials with a student taught through a mixture of lectures, seminars and labs. Crucially, the metric is a measure of ‘how much’ and not a direct measure of learning gain. We found:
- There was large variation across disciplines: on average students studying economics or history received less than half the teaching received by physics students. There was even larger variation within disciplines; for example, an economics student in the top decile receives almost five times as much teaching as an economics student in the bottom decile.
- There was no relationship between how much a student paid in tuition fees and how much teaching they received.
- Very little of the variation in TEACH can be explained by factors such as the fee, research reputation, or endowment; accident and history appear to determine the outcome.
Figure 1 Variation in TEACH
The enormous variation in TEACH that we find has echoes in Howard Bowen’s ‘revenue theory of cost’ (Bowen 1980). Bowen demonstrated that asking ‘how much does college cost?’ is like asking ‘how long is a piece of string?’. There is no meaningful answer: “at any given time, the unit cost of education is determined by the amount of revenues currently available for education relative to enrolment. The statement is more than a tautology, as it expresses the fundamental fact that unit cost [i.e., the cost of education] is determined by hard dollars of revenue and only indirectly and distantly by considerations of need, technology, efficiency, and market wages and prices.”
The crucial difference is that in our data, there is almost no variation in revenue per student. The implications are as follows. Since there is no agreement on the characteristics of an undergraduate education, cost and revenue cannot be determined. This means that governments can set the level of funding per student at almost any level they choose.
What we have shown is that even where revenue is constant, how much teaching students receive can vary by at least as much as the variation in revenue per student. The degrees of freedom in the system are even larger than Bowen suggested.
The policy implications of these findings are sobering. Price signals in higher education are weak to the point of being nonexistent. This gives rise to enormous scope for cross subsidy. These cross subsidies can arise between subjects, and between teaching and research. The interests of students get forgotten when competitive pressures are absent and governments prioritise the number of world class universities.
The following passage, written more than 50 years ago in the Robbins Report, bears repeating:
“The extent to which a narrow criterion of academic excellence has invaded British universities is sometimes overstated. But we are convinced that the danger exists. … Publication, by which a man earns a reputation in the open forum of the world’s opinion, should always be regarded as an important qualification for promotion; but it should not be the only qualification.” (Robbins 1963)
Arum, R, and J Roksa (2011), Academically adrift: Limited learning on college campuses, University of Chicago Press.
Bowen, H R (1980), The Costs of Higher Education: How Much Do Colleges and Universities Spend per Student and How Much Should They Spend?, Taylor & Francis.
Huxley, G, J Mayo, M W Peacey and M Richardson (2017), “Class Size at University”, Fiscal Studies.
Robbins, L (1963), The Robbins Report on Higher Education, Report of the UK Government Committee on Higher Education.