The experienced perceive that institutions work worse
Those who get in touch with rule-of-law institutions perceive that they work worse. Optimism on the functioning of institutions seems to be the privilege of the inexperienced, especially those with more education.
Evaluation of legal institutions is necessary for making good decisions when creating and reforming them. In recent decades we have relied on indicators based on surveys eliciting the perceptions of experts and citizens, as well as metrics based on assessments of formal rules. Prominent examples are the Governance Indicators, the Rule of Law Index, the Corruption Perception Index or WB’s Doing Business.
This reliance on general information makes sense if knowledge is held by experts and the general public. However, experts and the public are often asked for their perceptions without considering their position regarding the institution being evaluated. This differs starkly from the established practice for all sorts of organizations. The latter—from business firms to churches to universities—routinely collect the opinions of users and customers, who are assumed to be better informed on how the organizations satisfy their demands.
Can we do better with institutions? Given that information is costly, one should not expect the relevant information on institutions to be evenly distributed among members of society. Moreover, much information —in particular “knowledge of the particular circumstances of time and place”— is costly or even impossible to transfer. This “specific” information is likely prevalent for legal institutions, but eliciting it is particularly difficult: in legal institutions, the main channel for transferring information—prices—is often missing or distorted.
Moreover, as opposed to markets, artificial systems to gather information require identifying who knows most, and this is harder for institutions than for organizations. For instance, in a sense all citizens “use” general institutions such as the political system or the rule of law. However, even if all citizens benefit from the law, some of them are likely to be better informed on its functioning. In fact, the rule of law operates through supporting organizations such as the judiciary, the police and administrative bodies, and all citizens are unlikely to be equally well informed on how these organizations work.
This paper assumes that those who interact most closely with such organizations are better informed about the corresponding set of institutions. In particular, those who contract with them, experience contractual conflicts and litigate with them can be expected to have more information on the performance of contract law and litigation. Similarly, both victims and criminals should know more than the rest of the public on the functioning of the police and the courts and, therefore, on criminal law; those who most often deal with public bureaucracies should know more on administrative law; and parties to employment contracts should know more on labor law. In short: those with experience should know more because the experiences considered in this paper imply proximity to the public organizations and institutions being evaluated and are, therefore, likely to reflect greater knowledge on how they work. In a sense, experienced individuals are considered here as “institutional customers”. Basing performance indicators on their perceptions is akin to considering users’ satisfaction with their service providers.
To the extent that this proximity assumption holds water, it opens up many questions on how to design information systems to evaluate legal institutions and even on how to manage them. If some users or constituencies hold specific information about institutional performance, should we still focus our information-gathering efforts for evaluating those institutions on the whole population (as is often done) or, instead, on those particular users or constituencies?
As a first step, this paper focuses on more modest questions: How does experience affect perceptions of institutional quality? How does considering experienced subsamples modify performance indicators, both at the individual level and in terms of country scores, and what drives such changes? What are the consequences for our understanding of the distribution of knowledge across the population? Does experience affect perceptions more or less than, e.g., education and other demographic variables?
To answer them, using the World Justice Project general surveys, the paper explores how indicators of performance in four areas of the rule of law systematically vary depending on which persons we consider—the general population or experienced individuals—, and how experience may be more informative than variables such as education, age or wealth.
It finds out that individuals who have had experience with rule-of-law institutions evaluate them more negatively than the general population. Subject to the “strong ignorability” assumption, this experience effect holds two important consequences. First, indexes based on the assessments of experienced individuals rather than those of the general population rank countries very differently. Second, based on available information, experience is more influential on stated perceptions than alternative variables often considered as proxies of knowledge, such as education, gender, wealth or age.
To this extent, if confirmed and all other things being equal, these findings support focusing indexes of institutional performance more on experienced individuals. This could be achieved by selecting respondents, weighting the data towards experienced respondents or, at least, asking additional experience questions that would help in evaluating respondents’ knowledge. These experience-related questions would also provide additional evidence for clarifying many key issues, such as the presence of possible confounders and the effects of different types of experience, including the role of recurrent and expected experience, as well as the tradeoffs of relying more on experienced individuals.
In this vein, a likely advantage of experience is that it does not rely on informational intermediaries, but is more factual, governed by transactions and less centralized. On the contrary, perceptions by the inexperienced might be reflecting stereotypes that experts and the media create and educated individuals help to disseminate. Hopefully, there is a lesser risk that experienced respondents suffer the biases that have been pointed out in the literature with respect to subjective evaluations, which might be tainted by theories, ideologies and prejudices: e.g., basing their responses on countries’ reputations; suffering a “halo effect” that might lead them to consider economic growth as evidence that institutions are good; or basing their assessments on the same observed variables (e.g., authoritarian political regimes) that theories had been claiming to be driving institutional quality.
Source: Arruñada, Benito, “The Impact of Experience on How We Perceive the Rule of Law,” UPF Working Paper, December, 2019. Click here to download PDF.