What does consent look like in an age of AI?

AI’s evolving scope

One of the great sea changes of recent years has been the speed with which artificial intelligence has penetrated almost every aspect of everyday life. AI spending is anticipated to grow by 19.3% this year and a recent survey found two-thirds of professionals across sectors expect AI to disrupt their industry.

The effects can be felt in much of our day-to-day lives, from personalised content recommendations on streaming services to autonomous and driver assistance features in connected vehicles.

In retail, 81% of online retailers increased their AI budget in 2021 to stay competitive as shoppers increasingly demand more tailored shopping experiences online. Even outside of the digital space, Amazon Fresh has kicked off a trend for using sophisticated AI to make even the most casual purchases more efficient and attractive to consumers.

In my field, cybersecurity, AI has emerged as a vital component too. This has been fueled in no small part by the pandemic: the first 12 months of lockdowns saw a 600% increase in cybercrime.

Autonomous approaches to cybersecurity have been able to keep pace with this increase in threat, using AI to prevent unauthorised access to sensitive systems across industries.

What’s the catch?

A goal common to AI’s many applications is helping us make better, quicker decisions. If you’re driving an AI-enabled car, the idea is for the onboard computer to help you avoid collisions in split-second situations; if you’re shopping for new clothes, AI seeks to learn from your past activity to make recommendations.

In other words, AI has the potential to add ease and efficiency to modern life, ranging from the trivial to the life-saving. But at what cost?

One largely hidden peril (aside from the obvious risks of bias, defects or trolley problem-style quandaries) is the sheer quantity of user data to which AI systems have access, and the particular power these systems have to make vital decisions about which many people have no awareness. An increasingly frequently asked question is: how far have we, the end-user, actually consented to AI’s usage of our data?

This might sound like a philosophical question, but consent matters in a practical sense too. A recent illustration comes from the field of medical care: patients began raising concerns over the use of AI-driven diagnostic checks to which they had not consented in determining their care. Even where there is arguably no risk of harm, people have been left feeling angry that they weren’t consulted. Trust breaks down as a result.

This issue risks multiplying across the entirety of the emerging landscape of AI. Most people are not given the chance to understand how and where their data is being fed into this sprawling network and consent is becoming ever more neglected.

Where can we go from here?

So how can industries eager to enjoy the benefits of AI ensure they’re working harder to meet a good level of user consent before people’s data is put to work?

I put forward a new answer to this challenge that suggests AI can be used in the consent process itself. This approach has three key principles: ensure users’ initial data sharing permissions are meaningful; use these permissions to derive user intentions and enable these set permissions to grow to encompass the needs of modern life.

1. Implement meaningful permission for initial data sharing

Often users cannot keep track of what they have consented to, as permissions can be required across products, services and devices. To look at devices as an example, users are increasingly exposed to AI-driven functionality across traditional desktop contexts, but now also on mobile and smart devices.

Efforts have been made to extend more accessible information and choices to mobile. For example, Apple’s iOS 14 update earlier this year gave users the option to remove Facebook’s (now Meta’s) ability to track their activity across other companies’ apps and websites.

But this opt-in/ opt-out process continues to be imperfect because it is a singular action that is difficult to readdress at a later date, and so is difficult for users to keep track of.

Instead, we should strive for a more meaningful conception of consent in which we give the user control of their permissions outside of the immediate request from a digital service, for example from a unified control console.

User-Managed Access, the OAuth-based protocol better known as UMA, is a sound example of this, putting contextual permissions across the online space, from banking to medical records, into the hands of the consumer via a central user-centric console. This way, users achieve greater power and understanding of how data is being shared across services and devices.

2. Enable data permissions that usefully anticipate the user’s intentions

Of course, to create a consent-centred future of AI that requires user permission for each intricacy of data functioning would be punishing to the individual. No user concerned about the rising use of AI in day to day decision making wishes to click through “yes” or “no” to singular permissions for AI-based actions in an effort to increase disclosure.

The solution here is similar to how AI systems derive new data based on existing shared data. AI systems could be used to anticipate what the user is happy to accept, either based on previously granted permissions or by offering users simplified consent options that will justify actions for related permissions.

3. Allow permissions that can grow as the data being collected about the user grows too

Full consent is difficult because user data is something that grows, which in turn affects how AI uses it and the types of decisions it can make. In practical terms given the sheer amount of data being collected and generated about users and the number of services and devices used, permissions need to scale with the data.

Putting all these principles together suggests that AI may be a way out of the mess it contributed to in the first place, by applying it to an individual’s ability to manage data permissions. In this way, AI could empower users to make more informed choices about the use of their data.

Imagine an AI system that could recommend what personal information is worth sharing and what is not, and also help to visualise the consequences and the likely data flows through to third parties.

This could conceivably enable “true choice” for individuals at all digital levels and move us away from increasingly impractical opt-in/opt-out processes when using new services and devices.

Defusing the consent time bomb

AI promises to revolutionise how we live, work and enjoy leisure. But the speed, complexity – and frictionlessness – with which it has accelerated risks planting a ticking consent time bomb at the centre of the system.

Defusing it won’t be easy, but whether dealing with different types and sources of data, different platforms or the use of AI itself, our guiding principle should be enabling users themselves to make real, informed decisions about their own data.

Written by Eve Maler

Eve is CTO of ForgeRock, the modern identity platform for your consumers, workforce, and things. She is a strategist, innovator, and communicator on digital identity, security, privacy, and consent, with 20 years of experience leading standards such as SAML and UMA and publishing research.

As CTO, she is responsible for the Labs team investigating and prototyping innovative approaches to solving customers’ identity challenges, along with driving ForgeRock’s ForgeRock’s industry standards leadership. She hopes her duties still leave time to contribute to the rock ‘n’ roll outfit ZZ Auth and the Love Tokens.

Eve is a globally recognised strategist, innovator, and communicator on digital identity, security, privacy, and consent, with a passion for fostering successful ecosystems and individual empowerment. She has also served as a Forrester Research security and risk analyst.