Rethinking the value proposition of assessment at a time of rapid development in generative artificial intelligence

Abstract

An important lesson from business is that, in times of disruption, distinguishing between your organisation's value proposition and its operational processes is essential. Clearly, the rapid rise of widely available generative artificial intelligence (GenAI) is a time of disruption for medical education. The high workloads and competing pressures that currently afflict many of our educators makes ‘What is the value proposition of medical education?’ a particularly important question.

Already, learning, teaching, assessment, curriculum design and working practices are being reshaped in response to the emergence of GenAI technologies, and this reshaping is set to expand. It is tempting to change incrementally, making occasional tweaks. However, in doing so, it is easy to neglect what we are trying to achieve. Our response to GenAI should align with our value proposition and not purely react to the threat or challenge we face.

Deciding upon that response is challenging, in part, because a dissonance exists in medical schools between the espoused value propositions of education and those of assessment. Modern views on education typically promote lifelong learning and competency development, collaboration, professional-identity formation, managing uncertainty, student agency and generation of new and contextualised knowledge. Assessment, in contrast, typically values milestones and mastery, one-time measurement, competition, certainty and linear, predictable and guaranteed outcomes.

Even before the recent emergence of GenAI, information technologies made this discrepancy clear. Modern medical students have access to vast technological possibilities, including location- and time-independent communication, through which they can contact anyone, anywhere, immediately. This allows them to be connected with communities, perspectives and information sources globally. They can participate in the democratisation of knowledge through creation collaboratives (e.g. wikis, forums). They can not only access resources on almost anything but they can also create them (e.g. instructional videos, diagrams, presentations, podcasts). Where past medical students could only carry small pocket handbooks, nowadays, students can carry whole, easily searchable libraries and knowledge communities within a smart device.

Such developments have occurred over the course of years, but we have been slow to factor this evolving picture into medical school curricula and assessment. Now, with GenAI, the discord is even starker. Asking students to pass tests, or to produce artefacts, that can also be produced by AI, casts doubt on the relevance of our assessments. In responding, we must return to our assessment's value proposition and focus on producing the kinds of medical graduates needed today, and in the near future, rather than on protecting established assessment processes. For this, we will need a sophisticated view of the human–technology relationship.

A common, less sophisticated, perspective sees technologies as tools that can substitute for cognitive activity or learning. From this vantage point, it is easy to consider using GenAI in assessment as cheating (i.e. as students not ‘doing the work’). A logical response is then control, regulation and proctoring. Although understandable from a nostalgic perspective, such responses are, to us, misaligned with current needs. One problematic assumption is that assessment tasks should be successfully managed only by what students have in their ‘biological’ memory. Consequently, external tools or resources should be avoided because their use is seen as ‘cognitive offloading’, associated with mental or cognitive laziness.¹ Related arguments arose around the invention of electronic calculators, spellcheckers and even writing.² While human abilities do change in response to our environments (which include technology), these changes are not inherently bad, but are part of how we continue to adapt to the world around us.

An alternative perspective sees technology as an integrated part of complex, distributed activity.³ For example, the lens of distributed cognition considers solving authentic problems by a professional to be an interactive process between their ‘biological’ competence and their ability to mobilise, manage and combine different technological resources.⁴ From here, using technology is not cheating or laziness, but an integral and inevitable aspect of modern problem-solving and the negotiation of complex situations. This is not a cop-out: Limitations of independent knowledge are compensated, not simply by knowing where to find information, but by a sophisticated ability to find, appraise, synthesise and integrate multiple and interactive knowledge sources into one's existing expertise, in real time. This adaptive ability to navigate and negotiate an uncertain and unpredictable world requires both knowing things and a capacity to proceed, competently, where things are not yet known.⁵

Although this more complex perspective has always been desirable, since adaptively learning to use technology has always been important, it is now crucial if we are to find effective ways of reshaping education in the face of increasingly rapid technological change and disruptions of established practices.

What does this mean for assessment? Firstly, assessment programmes focussing only on individual artefacts or on one-off examinations are losing relevance. Modern medical professionals have access to technology, as do their future patients. AI applications like SciSpace Copilot already support lay people in finding up-to-date published research on any question and help them translate jargon-rich passages into accessible language. Medical education and assessment will have to educate doctors who are prepared for such situations.

Secondly, the value proposition of assessment programmes must be brought into alignment with that of education and its focus on learning. Fortunately, much is known about assessment-for-learning,⁶ and that knowledge can be used to develop more modern assessment programmes. These are difficult transitions, however, because traditional assessment programmes have become an embedded part of society's acceptance of assessment as producing competent graduates. Progressive, learning-focused assessment programmes that allow students to use AI technologies will face resistance as some important stakeholders will likely continue to regard the use of AI in assessment as cheating or cognitive offloading.

Finally, assessments must move away from focusing primarily on the student's ‘biological’ memory, towards their ability to manage and navigate complex social and material situations. This requires a fundamental rethink of the content of assessment, the overall programmatic design and how we define competence.