Expert accountability: What does it mean, why is it challenging—and is it what we need?

Abstract

When the Norwegian Parliament opened the southeastern Barents Sea for petroleum activity in 2013, it did so based on an impact assessment report prepared by the Ministry of Petroleum and Energy (Norwegian Ministry of Petroleum and Energy, 2012–2013). In 2016 the first licenses to drill in this area were awarded, and two environmental organizations sued the Norwegian state for awarding them. To prepare for the court case the organizations commissioned two economists with expertise in resource and environmental economics to review the impact assessment report and these commissioned economists quickly discovered major errors.

For one, the report did not discount the expected costs and future incomes—a standard procedure in such reports. Discounting with a 4% real interest rate would have reduced the estimated net income from opening the area by approximately 38%. This would make the opening a net loss (Greaker & Rosendahl, 2017a, 2017b). The report also estimated the gross income in the scenario with low petroleum findings to be twice what the underlying numbers showed (Greaker & Rosendahl, 2017b). These two mistakes together with optimistic estimates of employment numbers and the future oil price made it seem like there was no economic risk associated with opening, and this may have contributed to the Parliament's decision to open the area in spite of resistance from both environmentalists and the fishing industry (Greaker & Rosendahl, 2017a; NTB, 2017; Tomassen, 2017).

A standard remedy when mistakes are made in political life is to find someone to hold to account. But when experts are involved in governance processes they are rarely held to account for their mistakes. This is sometimes seen as a legitimacy problem of expert-reliant governance, and warnings are given about handing over more power to “unaccountable” experts. So, is the solution to make experts more accountable?

The popularity of “accountability” as a remedy for all kinds of problems has led to inflation and fragmentation of meanings attributed to the term, and also to uncritical uses of accountability measures. Still, this article argues that accountability of experts remains crucial for addressing the legitimacy problems brought up by increasing expert dependency. Yet, instead of proposing a new type of accountability regime applicable to all contexts were experts take part, the article takes a step back and asks: What exactly does it mean to hold experts “accountable”? And what are the distinct challenges of expert accountability?

There are several studies of expert accountability in relation to particular institutions, such as public agencies (e.g., Busuioc, 2013; Schillemans et al, 2021), central banks (e.g., Heldt & Herzog, 2021), judicial review (e.g., Contini & Mohr, 2007), and parliaments (e.g., Crum, 2017; Eriksen & Katsaitis, 2020). Yet, these studies are typically difficult to compare because they define accountability in different ways or leave definitions implicit or underspecified. They also rely on different taxonomies of subtypes, and make different, or have unclear, assumptions about what account-holding is good for. Furthermore, they focus on different types of institutions, and types of experts and expert roles that differ in ways that will be relevant for what can serve an appropriate and balanced set of expert accountability relations.

We believe there is need for a more general and systematic approach to expert accountability, and in contrast to some previous attempts (e.g., Holst & Molander, 2017; Moore, 2017), our contribution connects in more detail the literature on expert governance to the vast literature on accountability, so as to tease out the finer set of complexities in a more accurate manner. The aim is to provide conceptual and analytical tools that facilitate further comparative and evaluative research on expert accountability, and in particular for research on experts who take part in governance processes, as many experts do. The article uses the impact assessment case to exemplify definitions and distinctions , but also to illustrate the distinct challenges, and the tradeoffs, that are involved when holding experts to account.

The first part of the article clarifies our use of the term “expert,” and looks at the wide range of roles that experts can have in the context of governance. The second part works out a narrow but differentiated conception of expert accountability that helps us see how different subtypes of accountability may draw in opposite directions. The third part demonstrates how this conception of expert accountability allows us to map accountability relations that an expert is a part of in a more precise way. Part four looks at the criteria for saying that an expert is, or is not, sufficiently accountable. Part five analyzes the challenges of holding experts accountable. Two special challenges to holding experts to account are identified—the problem of epistemic asymmetry, and the problem of expert biases and mistakes. These influence other more general challenges of accountability, such as the problem of forum drift and the problems of many eyes and hands. The most proper response to these challenges will vary with institutional context and the more particular goal of account-holding, but the article argues that horizontal forms of accountability—peer professional and reputational accountability in particular—are key to expert accountability in many settings. However, expert bias, expert insularity, and forum drift can prevent these relations from functioning properly. There are moreover tensions and tradeoffs between horizontal accountability types and types of accountability which are designed to ensure democratic control and abuse of power. Finally, it is emphasized that expert accountability alone cannot address the normative problems of expert-reliant governance.

In Norway the decision to open new areas for petroleum activity lies with the Parliament. However, the Ministry of Petroleum and Energy are required by law to prepare an impact assessment report before a parliamentary decision can be made. This report must be sent out for public consultation, and the Ministry must sum up the findings of the study and the inputs from the public consultation in a white paper that should be presented to the Parliament. The decision to open the southeastern Barents Sea thus involved a range of in-house experts in several ministries and agencies, in addition to contracted experts from research and consultancy. Political parties, the media, and stakeholders—from local communities and environmental NGOs to the petroleum and fishing industry—also consulted experts on a range of topics relating to the opening. In short, various types of experts were sitting in several capacities, at all sides of the table, and could exert a significant amount of power.

The opening of the southeastern Barents Sea is far from special in this regard. Political decisionmaking standardly involves a range of experts, as well as a plethora of expert bodies, from public agencies and central banks, to advisory committees and courts. Commentators have characterized this as the “expertization” of society and policy making (Turner, 2003), and as the “rise of the unelected”—the development of a new branch of government made up of those with expert knowledge that cuts across the traditional separation of powers (Vibert, 2007), as well as conventional distinctions between government and civil society, public and private (Moore 2021).

To discuss the extent to which accountability is the answer to the challenges of expert-reliant governance, we need at least a working definition of expertise. In line with Alvin I. Goldman's influential approach, we define experts as those with considerable knowledge, and more knowledge than most others, in this or the other domain, who are also able to use this knowledge skillfully on novel problems (2001(2011), p. 114). This makes the scientist or academic an important expert category. In line with this we often see terms such as “experts” and “scientists” used interchangeably, and distinctions between experts drawn based on their disciplinary background (economists, lawyers, engineers, etc.). However, experts can also be identified in other capacities (Grundmann, 2016), and there are sources of expertise other than scientific training, such as especially relevant experiences or becoming knowledgeable about something through practical engagement with certain issues over time (Collins & Evans, 2007). Experienced civil servants or judges, for example, can possess this kind of practically gained expertise in addition to what they possess of scientific and professional skills; the same can apply to civil society groups with substantive sector expertise.

Yet, experts as we shall refer to them here, not only have a certain level of competence in some area. Their knowledge must also be of interest to someone and called for by them (Gundersen, 2018). Knowledgeable people become then, according to this definition, experts, only when their knowledge is considered significant and relevant, and their guidance is asked for. Finally, within this category of socially recognized knowers, we focus here on those that are included in governance and policy making. Roughly speaking, this adds up to the professional class of “unelected” knowers that are arguably increasingly influential in present day politics (see Habermas, 1992 for an early diagnosis of “expertocracy”).

Accordingly, the “governance experts” we have in mind can work in civil service, courts, consultancy, organizations, associations, or in research institutions and universities; they can be information providers, advisors, agenda setters, or even decisionmakers—as when a judge decides a court case, or a central bank board decides on interest rates.

And we ask, What does it mean to hold this varied, but distinctive, category of experts “accountable”? What is expert accountability?

Accountability is sometimes used in a wide sense to denote persons and organizations’ inner responsibility to moral values and public interest norms, willingness to be responsible, transparent, and responsive, or to engage in dialog and justification (Behn, 2001, p. 3–6; Bovens, 2010, p. 946; Dubnick, 2007; Mulgan, 2000, p. 555). The term may also allude to institutional arrangements such as separation of powers, constitutionalism, and judicial review (Mulgan, 2000, p. 563). Such broad approaches tend however to miss what is distinctive about accountability, and our starting point is therefore rather what has been called accountability in the core sense, meaning “answerability for performance” (Romzek, 2015, p. 28), or more precisely, “the process of being called 'to account’ to some authority for one's actions” (Bovens, 2007, p. 450). Accountability thus understood refers then to a specific social relation¹ between an actor², and a forum³, where the actor is under an obligation, formally or informally, to explain and to justify his or her conduct, and the forum has authority to request information and explanation of the actor's actions in a domain, and subsequently to sanction the actions (Bovens, 2007, p. 450).

When trying to delineate what accountability of experts in this core sense means, a first observation is that the governance experts we have in mind, despite their internal diversity, generally have a considerable scope of discretion in virtue of their expertise and thus typically have comparatively wide agent autonomy (Lindberg, 2013, p. 209). Accordingly, even if these experts may be held to account for many aspects of their performance, expert accountability is first and foremost a relation or mechanism aimed at controlling or constraining this scope. Accountability according to the core definition is furthermore ex post facto, meaning that the expert is held accountable for past (and not for future) performance.

Importantly, for an expert to stand in an accountability relationship, all of the things listed do not have to take place. What is important is that they can happen, and that all involved are aware of this, something which presumably shapes the expert's expectations and behavior.⁷

It is often said that experts in governance processes are unaccountable, but at a closer look we find that experts typically stand in multiple accountability relationships. Take an in-house economist of the Ministry of Petroleum and Energy, one whose area of responsibility includes writing the part of the impact assessment report that summarizes the commissioned expert reports on the social and economic consequences of petroleum development in the Barents Sea. Obviously, such an expert will stand in a bureaucratic accountability relation to his administrative superiors in the Ministry, a type of accountability relationship that is internal to the organization, vertical upwards hierarchical, and marked by high control. In this type of accountability relationship, the expert will primarily be accountable for abiding by the rules and procedures (procedural accountability), how money is spent, both for private expenses such as travel allowances and budgets for research (financial accountability, auditing), but also accountable for outputs and outcomes (product accountability). The expert is legally accountable for every part of his job that is regulated by laws, and he can be summoned before a court and held legally accountable in a civil case for things like harassment, and in a criminal case for things like corruption. Also, the expert will be audited internally or be held fiscally accountable, or both, for private expenses such as travel allowances and budgets for research.

The in-house economy expert is not in any direct political or democratic accountability relationship to voters. However, he can in theory be summoned before a parliamentary committee, but unless the expert has done something that can be punished in criminal law, it is the Minister who will ultimately be held to account for mistakes made in his ministry (a hierarchical accountability relation). The Minister of Petroleum and Energy may be said to be the expert's direct political principal and will have a strong interest in the content of his report. While there are no formal procedures for holding the expert politically accountable, the political leadership of the Ministry is likely to assess the outcome of this experts’ work also based on political considerations. Formally, the political leadership may not interfere in the expert economist's expert judgments, but it is not unlikely that political principals will have a heightened level of scrutiny vis-à-vis in-house and contracted experts if the expert's findings go against the principal's political wishes. If so, the in-house expert stands in an informal political accountability relation.

Importantly, the expert also stands in a peer professional accountability relation internal to the organization. Peers here would be other experts in the same domain. This type of accountability relation is horizontal and typically marked by a low degree of formal control and assesses the expert's work based on professional norms and standards. In addition, comes reputational accountability to peers outside the state apparatus, such as professors of economics in academia. Civil society and media can also try to hold the expert to account in a societal accountability relationship.⁹ Activist groups may for example ask the expert to explain and engage in a discussion of his assessment of the existing research. In our case Greenpeace and Nature and Youth tried to hold those responsible for the impact assessment report, including the experts writing it, accountable by pursuing both legal, reputational, and societal accountability simultaneously, that is, by suing the state for the opening decision, hiring external expert economists to help them document the mistakes, and simultaneously bringing media attention to the mistakes of the report.

This attempt to capture accountability relations for one of the many types of experts involved in the opening of the southeastern Barents Sea is incomplete but suffices to illustrate how one may embark on a mapping of the different accountability relationships which experts involved in governance are part of. Yet, mappings of this sort do not directly address the legitimacy worry connected to expert's influence on public policy. Yes, experts stand de facto in multiple accountability relations, but are they held sufficiently to account? Or, put differently, when is there an “accountability deficit”?

Let us once more concretize by going back to our case. Some of the things for which the impact assessment report was criticized were noticed already in the public consultation, such as not including the social and economic costs of increased carbon emissions. Other omissions were first discovered by the two economists commissioned by the environmental organizations. The two economists discovered that the report had failed to discount the expected costs and future incomes, and also that the estimated future gross income on the low scenario (with small petroleum findings) was the double of what one would expect. The latter mistake was traced back to a single Excel-typo in an underlying report provided by the Petroleum Directorate (Greaker & Rosendahl, 2017b; Taraldsen, 2017b). Also, other methodological choices in the report strengthened the impression that opening would be very profitable, for instance the projection that the oil price would remain at 120 USD per barrel. By the time the first licenses were awarded the oil price was down to 45 USD. Some of these mistakes in the report also found their way into the white paper presented to Parliament (Norwegian Ministry of Petroleum and Energy, 2013b). The result was that the economic risks of opening this part of the Barents Sea were underplayed, and this is likely to have influenced Parliament's decision to open. In such a situation, what would it mean for the expert economists in the Ministry who wrote up the Impact Assessment report, and the experts in the Petroleum Directorate who prepared the underlying report with the Excel-typo, to be sufficiently held to account?

When we ask whether an economic expert preparing an impact assessment report is sufficiently accountable, we must reflect on the implicit normative criteria relied on: Is the main problem one of ensuring more political or democratic accountability? Prevent corruption and power abuse, or secure epistemic quality, professionalism and induce learning cycles within the organization? Or some specific combination of these goals or values?

In general, some types of accountability relations are more conducive to securing the aim of a democratic rule than others, for example, the possibility of democratically elected forums, like parliaments, to hold actors to account. A mix of audit, bureaucratic, and legal accountability relations is typically seen as contributing to preventing corruption, while peer professional and reputational accountability relations are often considered to be primarily conducive to securing epistemic standards and learning. Thus, it is natural that actors with different roles and normative priorities in a case will be led to focus on different types of accountabilities. Yet, often one wants to achieve all these goals, but as we shall see, different types of accountabilities can easily come into conflict and also create accountability overloads.

Political and societal actors tend moreover to disagree on what the main normative problem is, and who should be held to account: When the expert mistakes in the impact assessment report became known, there were several calls for heads to roll: Some asked the Director of the Petroleum Directorate to step down (an example of hierarchical bureaucratic accountability where the leader is held to account for mistakes made in the organization), while others demanded a hearing with the sitting Minister of Petroleum and Energy in the Parliament's Standing Committee of Scrutiny and Constitutional Affairs (hierarchical political accountability). One also called for the very practice of impact assessment studies to be reviewed by the Office of the General Auditor (a horizontal audit relation) (NTB, 2017; Taraldsen, 2017b; Tomassen, 2017).

These calls for accountability were primarily voiced by environmental activists and by politicians who were critical of opening new areas for petroleum activity. Some of these critics characterized the mistakes and what they saw as a skewed focus in the report as possible corruption, or deliberate fraud, with the purpose of securing continued petroleum production in Norway, while some framed the report's mistakes primarily as a democratic problem, arguing that the government had presented Parliament with an inadequate knowledge basis for opening the area. Other actors, like the Director of the Petroleum Directorate, focused primarily on the Excel typo and characterized it as a “human mistake,” arguing that it had no significant impact¹⁰—a problem characterization that would point in the direction of accountability measures that could improve learning and the epistemic quality of future impact reports.¹¹

As we have seen, economic experts in the Ministry of Petroleum and Energy are not directly democratically accountable to voters, but they can be called to give an account of their performance to a democratically authorized forum such as a minister or a parliamentary committee. These indirect political accountability relations give a certain democratic control, and they can in some cases help prevent certain types of corruption and also induce learning. Yet, as we shall see, strong accountability relations to a minister or a minister's proxies, can also compromise an expert's professionalism and independence, and thus undermine the epistemic qualities of the outcomes which are central to legitimizing experts’ role in governance in the first place.

Including experts in the process of governance has been presented as the “filter” that ensures the “truth-sensitivity” of political decisions, reflecting how the primary normative justification of involving expertise is typically epistemic (Christiano, 2012): If we at all find it defensible to consult or delegate decisions to the most knowledgeable in some area, we do it because we believe their extra knowledge can enlighten policy making and political processes. On this assumption, the main focus of proper accountability regimes for experts should be to ensure the epistemic quality of experts’ work (Holst & Molander, 2017), meaning also that there should be a particular attention to induce learning from past mistakes. This typically requires an emphasis on horizontal forms of accountability. However, shifts toward more horizontal accountability—such as more internal or external peer review—may often be resisted, for example within a ministry, because it can reduce the democratic or bureaucratic control over the experts and the outcomes.

Still, it is important to see that even when there is agreement on the right balance of normative aims, a range of challenges will occur in trying to hold experts accountable.

There are at least two particular challenges when holding experts to account. One of them, the problem of epistemic asymmetry, stems from the specific characteristics of the expert/nonexpert relationship. The other, the problem of expert biases and mistakes, connects with some well-known distinctive features of expert behavior. Besides, there are several general challenges of holding actors to account that tend to intensify when the actor is an expert, such as the problem of forum drift, the problem of many eyes, and the problem of many hands.

Expert accountability—in the core sense which we have delineated—can contribute to preventing corruption and undue inequalities in political power, as well as to minimizing plain mistakes, agency drift, sloppy work, overconfidence, group-think, and ideological bias in policy- and decisionmaking. Initially, holding experts to account is thus essential to addressing the legitimacy problems of contemporary expert-reliant governance. Still, these problems come in many different varieties, and there are inevitably tradeoffs, so no accountability regime can achieve all goals even when they are settled.

When experts are “decision-makers,” such as a judge or the chair of a central bank, there will, at least in well-functioning political systems, be formal accountability mechanisms in place that are deliberately designed to balance concerns about professional integrity and epistemic quality, possible corruption, and sufficient democratic control and separation of powers. In many other cases, for example, when experts shape policies primarily as knowledge-providers and agenda setters, accountability regimes are typically more haphazard, and not necessarily designed with enough attention to the overall balance between professional integrity, preventing undue influence, and democratic accountability. Lack of attention to professional integrity is likely to be most intense for governmental in-house experts, and particularly in cases where the political and bureaucratic principals do not respect the experts’ professional independence and integrity.

The Excel-typo that was made by an expert in the Petroleum Directorate was most certainly a “human error.” However, the ministry's failure to notice the resulting high gross income on the low scenario, and some other mistakes of the report, seem to point to biases or groupthink in a self-selected group of energy economists working in an institutional setting where it has been taken for granted that petroleum activity is highly profitable. The mistakes in the report also suggests insufficient internal and external peer professional relations and review procedures. Yet, as it became clear in the third round of the climate court case, too strong political and bureaucratic accountability relations also seem to have been an important factor for the mistakes. Here it was revealed that the Petroleum Directorate had made a more updated estimate of the economic potential of opening the southeastern Barents see on their own initiative, and that this new report estimated a significant risk of net loss. But the findings of this new report were not included in the ministry's impact assessment report and not presented to Parliament (Fjeld, 2020). When this was revealed in the court proceedings, it led to a political accountability process in which the former Minister of Petroleum and Energy had to answer questions in a parliamentary hearing.

In this article we have suggested a context-sensitive approach to expert accountability, where we start by identifying the problems the accountability regime is supposed to solve: In the case under scrutiny, is the main problem that the involved experts are sloppy or make mistakes? Is it that they are biased, have vested interests, or insufficient independence? Or is the main problem rather that they misuse their power, or have the power to make decisions that they should not be authorized to do? And what is the optimal balance between these aims, for these types of experts, in the given context?

As we have seen, it is itself often controversial to diagnose what is the main normative problem. Yet, given that experts’ role as a filter for more truth-sensitive decisions is what justifies giving experts a special role in governance in the first place, attempts to hold experts to account must have a special focus on upholding epistemic quality. The problems of epistemic asymmetry and expert mistakes threaten, however, the fulfillment of this filter function and generate particular challenges for expert accountability. Other general challenges to accountability, such as the problems of many eyes and hands, may also be larger when the accountee is an expert.

We have argued that horizontal forms of accountability, and peer professional and reputational accountability relationships in particular, are generally vital to address these problems. Yet, the persistent problem of expert bias suggests that internal peer accountability may be insufficient. Both external peer accountability and accountability to peers in neighboring domains of expertise—and in some cases also to affected parties and stakeholders—can thus be necessary to prevent mistakes, bias, and induce learning.

Forum drift may be the most acute challenge for well-functioning horizontal forms of accountability such as peer review and reputational accountability. Many factors contribute to expert forum drift, and no simple solutions are available. A sufficient number of permanent academic positions with the academic freedom, time, and job security to be able to take on peer review tasks seem important. Changes in academic institutional incentives and culture may also be necessary to ensure that peer accountability tasks are seen as meriting and prestigious, especially those that are voluntary.

Also, journalists with relevant sector and case knowledge and opportunities for doing investigative journalism seem crucial. Public consultation regarding legislative proposals and the knowledge basis of policies and political decisions—such as impact assessment reports—is furthermore important both for transparency in policy processes and for getting epistemic input from different relevant sectors and perspectives. Such input may uncover specific mistakes, but it may be more difficult to correct for bias and narrow framings, in particular if the public consultation takes place late in the political process.

Often then, a combination of internal and external peer review, including stakeholder involvement, early in the process, or on a regular basis, would be a way to go. As our case shows, highly motivated civil society actors who have their own experts, or who commission experts, may play a very important role. Still, for expert accountability to work effectively, it will often be necessary to have someone who is accountable for the horizontal account holding to actually take place. This may require an increased formalizing of horizontal accountability relationships. Again there is a delicate balance to be upheld, and there will often be unavoidable tradeoffs between the goal of increasing epistemic quality, and sufficient democratic control and reducing abuse of power: Adding too much political and bureaucratic or managerial accountability of experts, or ongoing control, will typically be counterproductive, both because the epistemic asymmetry is considerable, and because expert knowledge is particularly vulnerable to conflicting accountabilities and accountability overloads.

Finally, even the right, and rightly tuned, kind of horizontal accountability is not a panacea for all problems of expert-reliant governance. In particular it seems necessary to cultivate both individual and collective epistemic norms and virtues in expert communities. Any expert accountability regime will tend to be ineffective if expert accountees are not committed to a proper professional ethos, and if there is a lack of long-term cultivation of well-functioning expert cultures in society, including in higher education and professional training.

This article's conception of expert accountability provides analytical tools to discern between different types of accountabilities and aspects thereof, that makes us better equipped to locate the right balance of the right kind of accountability types. Contriving a meaningful accountability regime aimed at holding experts in governance to account, requires an awareness of the potentially conflicting aims and functions of account holding, and conscious treatment of what we have listed as five key challenges to holding experts to account. We have identified forum drift as a particular challenge for expert accountability that calls for structural changes to be overcome. Our discussion has also helped us see that accountability cannot be the only answer to the pressing legitimacy challenges posed by the “expertization” of governance.

We hope this approach can stimulate more research, both empirical and normative, on expert accountability. Nontrivially, we believe our notion of accountability in the core sense, and our way of analyzing sub-types of accountability relationships may be useful for conducting descriptive and comparative mapping of expert accountability in particular cases or contexts. More importantly, we think our approach will enable better normative assessments of expert accountability relationships. First, because it provides a systematic overview of implicit normative standards, and also an overview of challenges and dilemmas for setting up accountability regimes for experts; this can be further developed into case- and context-sensitive assessment schemes. Such schemes can then be applied in evaluations and critiques of existing expert accountability regimes, and in turn stimulate recommendations for re-designing them.