Epistemic Fragmentation Poses a Threat to the Governance of Online Targeting

The following is a paper on epistemic fragmentation issued by Oxford academics’ Dr Silvia Milano, Dr Brent Mittelstadt, Professor Sandra Wachter and Dr Chris Russell; and edited by Jonathan Finlay.

Online targeting isolates individual consumers, causing what we call epistemic fragmentation. This phenomenon amplifies the harms of advertising and inflicts structural damage to the public forum.

The two natural strategies to tackle the problem of regulating online targeted advertising, increasing consumer awareness and extending proactive monitoring, fail because even sophisticated individual consumers are vulnerable in isolation, and the contextual knowledge needed for effective proactive monitoring remains largely inaccessible to platforms and external regulators.

The limitations of both consumer awareness and of proactive monitoring strategies can be attributed to their failure to address epistemic fragmentation. We call attention to a third possibility that we call a civic model of governance for online targeted advertising, which overcomes this problem, and describe four possible pathways to implement this model.

Online targeted advertising (OTA) is the engine of the digital economy. Many of the services that we have come to rely on, from e-mail to search engines, entertainment and social media, are financed through advertising. Platforms collect data from consumers as a condition of access to services and personalized content, typically through processes marked by stark information and power differentials that call into question the possibility of meaningful consent.

In turn, advertisers pay these companies to target their campaigns to specific audiences built on inferences drawn from user data. The technologies that draw these inferences, build audiences and personalize content create the fuel needed to drive the digital economy. OTA is not a benign application of machine learning. It can lead to individual consumers losing contact with their peers, a problem we call epistemic fragmentation.

Epistemic fragmentation increases consumers’ vulnerability to the harms of advertising, while simultaneously damaging information consumption in the public forum and hindering efforts to institute effective regulation. Despite the rapid evolution of OTA, regulatory frameworks internationally remain anchored to traditional methods of advertising.

This is changing, as governments are moving to draw new regulations, mobilizing huge economic and political interests. But effective regulation cannot be guaranteed if the underlying challenge is not fully conceptualized. This Perspective makes several contributions to conceptualize and combat epistemic fragmentation. First, we introduce a taxonomy of the harms of advertising.

We argue that targeting tends to normalize harmful content and makes it increasingly difficult to monitor compliance with existing codes of conduct in the advertising industry. The strategy pursued by regulators to address these issues at present is inadequate because it does not address their root cause, which we trace to epistemic fragmentation.

By hiding each individual’s personal context, targeting makes consumers more vulnerable and increases the overall costs of instituting protections via proactive monitoring. Instead, regulators should promote an active role for consumers in fighting epistemic fragmentation by adopting a civic model of governance for advertising.

We describe four potential pathways to implement our proposal, and conclude by summarizing our argument and pointing to future work.

Harms of advertising

From the point of view of consumers, the increasing ubiquity of OTA may increase access to relevant information and products. But personalisation also carries notable risks of harm.We can distinguish four categories of harms that advertising can cause to consumers, which are organized along two dimensions. One dimension groups harms that originate in the nature of the content that is either included or excluded from what is shown to an individual consumer.

Content that is genuinely bad, for example ads making false claims about a product or using racist stereotypes to promote a product, is harmful when it is included among what a consumer sees. On the other hand, if a piece of content is genuinely good or even vital, for instance an important public health announcement, it may be harmful for an individual not to see it. Along the second dimension, we find harms that do not stem from the nature of the content per se, but depend on the context in which the content is delivered.

An exploitative context is one where, independently of whether the content is intrinsically harmful, the way in which it is delivered exploits a consumer’s vulnerability. For example, ads for high-fat-content foods may not be generally bad, but they should not be targeted to children. Similarly, gambling ads can be exploitative.

Finally, a deprived context is one where a consumer is not shown ads that would be relevant and beneficial. For example, a deprived context might be one where a consumer who would be interested in finding a job is not shown ads for jobs in their area.Although these types of harms are conceptually distinct, they are not mutually exclusive and can co-occur.

For example, a racist message or a false claim may be presented next to reputable brands or news items, where the context helps to normalize the harmful message. Together, absolute and contextual harms can contribute to the degradation of public discourse.

Effective monitoring and enforcement mechanisms are essential components of regulation. It is here that OTA poses the greatest challenge. Current industry codes of conduct address most types of harms that arise from OTA, with the partial exception of contextual and omission harms that could nonetheless easily be incorporated going forwards.

However, updated codes will remain inadequate without effective methods to monitor and enforce advertiser and platform compliance. Traditionally, regulatory agencies have relied heavily on reactive enforcement mechanisms including consumer complaints and post-publishing reporting. The scale and method of distribution of OTA fundamentally challenge reactive monitoring regimes.

Human reviewers cannot manually inspect the huge volume of ads exchanged online on a daily basis. Screening ad content and monitoring the contextual effects of ad delivery are increasingly automated, despite the machine learning capabilities needed to replace human judgement in content review not yet being available.

Even if platforms, consumers or regulators could reliably check every advertisement for harmful content, contextual harms and harms of exclusion are much more difficult to monitor because they can present in myriad combinations. Regulatory bodies are nonetheless moving to fill this gap.

Two natural but insufficient responses

Regulators already recognize weaknesses in current monitoring and enforcement methods. A recent report by the UK Centre for Data Ethics and Innovation, for example, details regulatory proposals to enhance consumer protection through collaboration with social media platforms.

In the United Kingdom, the ASA’s More Online Presence strategy, launched in 2019, sees it in the words of the ASA’s Chairman David Currie—as “rebalancing away from reactive complaints casework and towards proactive tech-assisted intelligence gathering, complaint handling, monitoring and enforcement”.

The limits of consumer awareness

Part of the challenge created by OTA is that it hides from consumers when, how and why they are being targeted. In contrast, more traditional forms of advertising use more transparent delivery mechanisms. To give an example, consider newspaper advertising.

In the printed version, advertisers buy spaces and their ads are seen by all readers of the same newspaper. Readers can be certain they are seeing the same ads as everyone else in the same context; the advertising environment is the same for all readers. In contrast, readers of online versions of the same newspaper will not be able to make the same inference, as digital ads are typically customized to the reader on the basis of targeting data (for example tracking cookies, inferred interests), meaning different readers see different ads when visiting the same web pages.

One common approach to create equivalent protections for consumers against OTA is to give consumers more control over how they are targeted, for example control over the categories of data they share with advertisers or platforms. This ‘awareness strategy’, as we will call it, is a constant among the recommendations made across recent reports on regulating online advertising. Initiatives such as YourAdChoices aim to inform consumers about how they are targeted, giving them access to an explanation for why they see a certain ad, and options to control the types of ads they see.

Initiatives of this kind may be important to build consumer trust. However, shifting the responsibility back to consumers to protect themselves from unwanted targeting is an ineffective strategy. Current methods adopted by the industry fall short of providing adequate explanations of actual targeting mechanisms. They do not offer meaningful protection in cases where consumers are unfairly targeted or manipulated without their knowledge.

Consumers can-not be expected to identify when they are affected by differential pricing, or manipulated through repeated exposure and remarketing practices, because they lack contextual information about how and when these processes occur at a platform level. The awareness strategy therefore risks victimizing vulnerable consumers while doing little to monitor advertiser compliance with relevant regulatory frameworks and codes.

Moreover, identifying harmful content and contexts requires an especially critical and empathetic eye. Take, for example, a case of an ad that uses sexist stereotypes to promote a car, and that is somehow perfectly targeted to consumers who agree with the stereotype. Is this an objectively harmful practice? From the view of the targeted consumers, possibly not.

At a minimum, they would seemingly be less likely than an average consumer to raise a complaint or experience harm, either because they do not realize that the message uses a harmful stereotype, or because they agree with the stereotype. An explanation for why such targeted content is nonetheless harmful must consider the broader social effects of the stereotype. Transparency can also backfire if it reveals that advertisers used information that is unacceptable to the user, and do so via a plat-form that the user does not fully trust.

Moreover, the suggestion by platforms that consumers benefit from OTA because it helps them find more relevant content, is at odds with the strategy of improving awareness. If consumers must be more vigilant about how they are targeted, then this takes away some of the supposed benefit of targeting, which should reduce information overload faced by consumers.

Shifting responsibility onto consumers undermines the supposed efficacy of targeting; seeing more relevant ads is not worth being harmed in the process.Contextual harms can also go undetected. For example, targeting a consumer based on their inferred interest in gambling may be considered unethical, regardless of whether the recipients appreciate the ads. But of course, appreciative recipients are less likely to raise a complaint. Similarly, exclusion from opportunity ads, such as ads for high paying jobs, may go unnoticed without further investigation.

Giving consumers explanations for why they see specific ads may seem to be a natural solution to mitigate these concerns. However, transparency in practice fails to deliver in this regard.Finally, raising consumer awareness is ineffective for discovering systemic issues, such as statistically differential exposure to opportunity ads or discrimination by association.

In these cases, where the omission of relevant information harms a consumer, informing individuals about why they are seeing certain ads (even if platforms could provide adequate explanations, something that is not yet being achieved) would not be enough to raise awareness of being harmed, and would not provide a sufficient basis for recourse.

Failures of proactive monitoring

Given that promoting consumer awareness is an insufficient monitoring strategy, regulators need additional checks to ensure that advertisers comply with codes of conduct. In this context, a natural strategy is to augment the regulators’ online presence through a proactive monitoring strategy. Regulatory bodies in the United Kingdom, Europe and other Western countries are moving in this direction, working in partnership with online platforms.

For example, in the United Kingdom, the ASA is implementing this strategy in various ways, including the use of avatars posing as children to monitor the ads that a child would see online. Switching to a proactive oversight model implies that the criteria for what counts as harmful content, and which categories of consumers are protected, become controlled by either the platform, the regulator or some form of cooperation between the two.

However, ‘harmful content’ is a malleable category that evolves together with public awareness of social issues. It is likely that some of the messages that are acceptable today will be seen as problematic in the future. While benevolent paternalistic monitoring may appear effective in the short term, it could preclude social progress and freedom of speech.

The ongoing shift to automated monitoring using machine learning tools, which intensified during the Coronavirus pandemic, adds to this worry given well-established cases of applications of machine learning to other domains perpetuating discrimination and injustice. Contextual harms of exclusion, which by definition are produced by the absence of something, are particularly difficult to identify.

Individual consumers struggle to know what they are not being shown without outside help. Agencies filling this role would need to know which vulnerable groups or audiences to monitor. Consistent identification would require a data-set containing all (or a representative sample) of the advertisements served on a given platform categorized by audience, raising privacy concerns.

Regulators may also lack access to sensitive audience demographics data. Active oversight programmes are also expensive. To effectively monitor ad quality and the fairness of their distribution, platforms need to create monitoring systems, often using machine learning. However, such safeguards carry substantial costs, both in computation and human labour (for example, to label harmful content to train and maintain automated monitoring systems). These costs of monitoring, as well as the human and economic cost of maintaining current labelling processes, should be considered in future regulatory strategy.

A challenge from epistemic fragmentation

OTA thus poses new challenges to regulators, who are responding by introducing more proactive monitoring and consumer awareness campaigns, but both of these strategies face serious limitations. We trace the source of the issue to a systemic feature of OTA that we call epistemic fragmentation, and argue in favour of a civic form of monitoring to address this.

Epistemic fragmentation

The predominant self-regulatory framework used by the advertising industry has been reactive and complaints-based. Complaints have historically not been made by the most vulnerable groups or harmed consumers in isolation, but rather by concerned third parties or organizations acting from civic concern. Complaints about potentially exploitative ads are predominantly raised by consumers with relevant expertise or interests, who are likewise unlikely to be misled.

For example, a financial advisor could report an ad for fraudulent tax reduction services, or, a teacher could report an ad that exploits body image issues to promote a fashion product out of concern that it could harm a teenage student who may not realize how it exploits common insecurities.

In these examples, the consumers most likely to have the relevant information and motivation to raise a complaint are not themselves vulnerable, but are aware of those more vulnerable to harm. Parties raising a complaint have sufficient contextual information to recognize how the ad may harm others. Instances of contextual harms can also often only be flagged by third parties who have access to contextual information about targeting.

For example, identifying that certain opportunities are only advertised in newspapers read by specific demographics, which may be discriminatory, requires that someone with access to both contexts raises the complaint. Access to contextual information, as well as motivation to speak out in favour of an affected community, are thus necessary to identify and flag certain harms.

With OTA, this shared context is effectively destroyed. The teacher and financial advisor may never see ads directed at teenagers or people facing financial difficulties, and therefore never raise a complaint. Without shared advertising space, the extent to which opportunity ads are shown differentially to members of different demographics cannot be reliably measured by the public. Each consumer’s ‘personal context’ is hidden from others, meaning nobody knows exactly what others see and cannot raise a complaint on their behalf.

For our present purposes, we define a personal context as the sum of two components: the personal information about an individual consumer (that is, who they are, what they are interested in and value, where they live and so on) and the content that they see (that is, the specific ads that are served to them by the platform).

In offline advertisement, elements of consumers’ personal contexts are available to others. Full details of personal contexts, for example extensive personal (profile) data and ads encountered, are of course not routinely available, but sufficient contextual information exists to form an initial judgement and raise a complaint for further investigation. Contrast this to what happens online: personal contexts are entirely opaque.

Major platforms maintain ad repositories, but do not grant any specific tools to external organizations who wish to report on how ad campaigns are targeted at different demographics. As each individual consumer is served different (personalized) ad content, and as consumers do not know what ads others are seeing when visiting the same websites, each consumer’s personal context is hidden from others.

We refer to this lack of shared context in relation to a given practice of content personalisation as epistemic fragmentation. Epistemic fragmentation shares some similarities with other phenomena connected to content personalisation systems, such as so-called filter bubbles, which can arise when a content personalisation system limits individual users’ exposure to information and viewpoints that differ from their own.

While the extent to which filter bubbles polarize may be less than previously thought, the reason they seem to matter is that they could reduce individuals’ capacity to access relevant information and form opinions. This suggests that the way to address filter bubbles would be to grant individuals access to more varied information sources, limiting the extent to which discordant opinions can be filtered out by the system.

Epistemic fragmentation could be a factor determining the formation of filter bubbles, but the latter are not a necessary component of the former. For example, a consumer could be exposed to a diversified range of ads online, and so not live in a filter bubble, but at the same time have no information about what ads other consumers are seeing.

The reason why epistemic fragmentation matters, moreover, is not just because it limits individuals’ ability to access information that is relevant to them, but also because it limits their ability to assess the quality of content that is accessed by others. In doing so, it limits their ability to care for those others, and to take a role in the governance of the system.

A different model of governance

Regulating OTA is a pressing issue. We call the current approach, stressing consumer awareness together with increased proactive monitoring, the monitoring model of governance. In this model, individual consumers interact exclusively with the central monitoring agent, without access to others’ personal contexts. But isolated consumers, as we have argued, remain vulnerable.

In what we call the civic model, by contrast, consumers (and civil society more generally) are given the power, epistemic resources and procedural means to contribute to the monitoring of the system and affect change. In this model, epistemic fragmentation is reduced by design, ensuring that individual consumers have access to aspects of others’ personal contexts and to the operations of the monitoring agent.

The civic model of governance is consistent with recent proposals put forwards by regulators in the EU and elsewhere. The European Commission’s Digital Services Act proposal, for instance, contains recommendations to increase transparency and institute ad repositories that are publicly available and searchable, including information about the groups that receive an ad.

OTA has long been identified as a threat to privacy, but epistemic fragmentation is a systemic feature distinct from privacy. Improving the privacy of ad delivery systems, such as Google’s recent changes concerning third-party cookies, will not address the issues we have identified. Two things are worth noting in this context. First, based on our discussion, epistemic fragmentation emerges as a characteristic feature of online targeted advertising, but generalizes to other domains where information is shared online via systems that filter and personalize what each user sees.

Insofar as epistemic fragmentation hinders effective monitoring, it gives rise to systemic harms: by fragmenting individuals’ access to each other’s personal contexts, we lose vital access to the necessary information to identify and address harmful content, practices and omissions. Second, the harm caused by epistemic fragmentation is not captured by legislation such as the EU General Data Protection Regulation, which focuses on the personal data and rights of individuals, but largely ignores collective harms and protections.

Finally, we see three sets of reasons why civic governance is worth pursuing: epistemic, ethical and political. First, from an epistemic standpoint, as we have argued, distributed monitoring is more effective at surfacing cases of harmful content and/or context, which would be difficult or impossible to notice without the contextual information that is only available to uniquely situated consumers.

Addressing epistemic fragmentation removes the barrier to accessing this contextual information. Second, the current system of targeted advertising limits people’s ability to care for their communities. This gives ethical reasons in favour of civic governance. Third, from a political standpoint, epistemic fragmentation hinders informal ‘deliberation in the wild’ that is necessary to provide the proper input to the more formal aspects of democratic governance.

Towards civic governance

We propose four possible pathways to implement civic governance. With these proposals, our aim is to spark debate to identify new strategies for regulating OTA. None of these proposals would be sufficient in isolation. An independent regulatory agency responsible to set the codes, respond to complaints and enforce decisions is still essential.

Our discussion highlights that the debate on regulating OTA should move beyond privacy concerns, engaging instead with solutions to limit epistemic fragmentation, promote civic governance and ultimately improve the quality of public discourse.

Noisy targeting

Regulators could allow OTA, but only on the basis of coarse-grained categories that can be easily communicated to consumers, or introduce some noise in targeting so that out-group members are included. Regulations should define the level of granularity that is allowed for different categories of targeting, which groups of consumers can be targeted and how.

Google and Facebook already implement this strategy to some extent, as they do not allow advertisers to target their campaigns at a level of specificity that would be technically possible given their algorithms. However, this is not yet done under any kind of robust oversight. A clear limitation of this approach is that it primarily addresses privacy (and filter bubbles) more than epistemic fragmentation. To be feasible, there must be transparency about the context in which an ad is seen and access to other perspectives.

Targeting quotas

As a second option, a quota system could be introduced by which regulators limit the proportion of targeted ads per customer per platform, while the rest of the advertisements on display must be non-targeted. Alternatively, regulators could allow an ad to be targeted only a fraction of the times it is displayed, with the rest of the impressions via contextual placements. This would limit the possibility to exploit consumers’ individual vulnerabilities, reducing both contextual harms of inclusion and exclusion.

Ban targeting

Targeting hinders the ability of consumers to learn and care for each other by obscuring what ads others are seeing. Accessing this distributed knowledge is especially valuable. A radical but obvious response to these observations would be to ban targeted advertising entirely.

This suggestion has recently been advocated, for different reasons than those we present here, by others who point to the limited economic effects of targeting on consumer markets. A problem and possible limitation for this approach is that it does not address epistemic fragmentation if the content that provides the contextual anchoring for the ads is itself targeted.

Reconstruct the public forum

While the previous approaches are low-tech, another possible solution is to use technology to reconstruct a digital public forum. Instead of trying to reproduce the conditions for the traditional, offline monitoring regime, it may be possible to create new forms of digital public spaces that sustain civic governance. As an example, the Citizen Browser Project, recently launched by the investigative journalism organization The Markup, proposed a tool to improve the accountability of OTA.

As part of the project, a representative sample of internet users were to be paid to voluntarily install and use a custom browser, allowing researchers to gather data on what ads online platforms serve to individuals in different demographics. Generalizing the idea of this project, Wachter suggested that a way to address epistemic fragmentation would be to support independent research, ‘white hat hacking’ and collective and group rights to keep platforms and advertisers accountable.

While this approach would not be sufficient to solve epistemic fragmentation, it is nonetheless a step in the right direction in recreating the public forum. However, Facebook’s move to shut down this project, on privacy grounds, illustrates the difficulty of aligning the interests of regulators, tech platforms and citizens with respect to monitoring online harms.

Other technological means could help users contextualize the content they encounter and understand what other users are seeing. For example, Wachter et al. recently proposed that AI system controllers should be required to routinely produce summary statistics based on conditional demographic disparity to help fight algorithmic discrimination.

This type of disclosure can help reconstruct the public forum by showing how outcomes are distributed across groups affected by a given AI system, which helps identify disadvantaged groups that might otherwise go unnoticed. This approach could be adapted to facilitate conversations about the distribution of advertisements and outcomes across relevant groups to identify contextual harms of exclusion.

The four proposals above are merely starting points for further discussion to create a workable regulatory strategy to combat epistemic fragmentation. One may worry that, to operationalise any of these proposals, we must define a demarcation between problematic targeting and socially acceptable audience segmentation.

A wide-ranging and inclusive political debate over appropriate thresholds and levels of granularity in targeting is precisely what our proposals are meant to foster. This type of debate is an essential complement to ensure technological advances in flagging and reporting problematic personalized content serve the needs of the public. At the moment, these decisions are predominantly taken by advertisers, platforms and industrial bodies with little external oversight. Civic monitoring and technological innovation must be complementary to uphold the democratic governance of online spaces.

Conclusion

OTA creates a disconnection between individual consumer’s experiences and the experiences of their social circles. We refer to this as epistemic fragmentation. This phenomenon amplifies the harms of advertising and deteriorates the forum of public discourse around what, as a society, we consider harmful.

Absolute harms, especially harms of inclusion, can be normalized by presenting them to consumers in personalized settings where they are placed next to trusted sources, or showing them only to consumers who already agree with the harmful messages, and who thus fail to question them. Contextual harms, both of inclusion and exclusion, are especially likely to arise in the presence of targeting, and their co-occurrence can give rise to summative effects, amplifying patterns of discrimination and disadvantage.

However, epistemic fragmentation causes serious damage also in a second, more indirect way, by limiting our ability to act as empowered citizens, which is the foundation of civic governance.

The current way in which discussions around regulating OTA are framed portrays consumers in two ways: either as individual agents who must be educated about privacy and given more individual control over the types of content that they want to see (what we called the strategy of improving consumer awareness); or as passive subjects in need of increased protections, which should be provided by monitoring agencies in partnership with online platforms, who are the only entities that have access to enough data and resources (the proactive monitoring strategy).

We have argued that this binary framing is too restrictive. Under epistemic fragmentation, even educated individual consumers remain vulnerable to exploitation, and the epistemic resources necessary to implement effective proactive monitoring remain largely inaccessible, in addition to generating social and economic costs.

If epistemic fragmentation is a root source of these problems, then any successful approach at regulating OTA should address it. Restoring a shared public forum should be the priority if we want OTA, and personalized content more broadly, to be safer, more accountable and ultimately better for all of us.

We put forward four possible ways to do so: (1) blunting the precision with which ads can be targeted to individual consumers to increase the likelihood that harmful content or contexts are identified; (2) instituting targeting quotas to limit the ability of advertisers and platforms to decide who should be included or excluded from certain messages; (3) applying a blanket ban on targeted advertising; or finally (4) reconstructing a digital public forum via targeted technological interventions. All of these suggestions raise technical and political questions that should urgently be the object of debate.

Dr Silvia Milano: “Framing consumers as individuals that need educating about privacy or passive subjects that need greater protection from online targeted advertising is too restrictive. Consumers should be empowered to take an active role in regulating online advertising”.

The paper can be located in its entirety here.