To mark the launch of a new programme of work on analysing algorithms, the Competition and Markets Authority (CMA) has published a report analysing some of the potential economic harms that may arise from misuse of algorithmic systems. The report focusses on the perspectives of economics, computer science, and behavioural science.
The report acknowledges there is a fine line between personalisation, which enhances the user's experience and, digital manipulation which distorts a consumers decision making in respect of a product or service.
Much has been written about the risk of direct discrimination caused by algorithms. The CMA also warns about the ease with which indirect discrimination can arise. People with protected characteristics are unevenly distributed geographically, may 'like' similar social media posts or purchase similar services, which all become inputs to the algorithm's trend analysis.
Businesses should tread carefully where they look to:
- offer personalised pricing (advertising different prices to different people);
- automate decisions on how much to increase prices, what price to offer on renegotiation, and ‘win-back’ offers;
- present personalised ranking (using information about the user beyond the search query to decide which results to display and in what order) or pay for prominent ranking (consumers are more likely to select options near the top of a list of results);
- determine the range of information and options presented to a consumer;
- manipulate the consumer journey (e.g. targeting particular consumers at a time when they are more likely to give favourable feedback, distorting the review ratings); or
- favour your own products or services at the expense of your competitors.
A new Digital Markets Unit (DMU) is being set up within the CMA to implement a pro-competitive regime for digital markets.
Recognising that consumers may not understand the basis on which algorithms present information to them, the CMA proposes that existing consumer laws need to be strengthened to address these concerns. There is also a recognition that the use of algorithmic systems can cause harm to people in their role as citizens, and not just their role as consumers or business owners.
This report serves as a warning to those businesses investing in a more sophisticated online-presence and automating parts of their sales process, without doing the necessary due diligence. Algorithmic systems can provide substantial benefits to consumers but must not impede them from making effective and informed choices.
"...firms are responsible for effective oversight of such systems, which should include robust governance, holistic impact assessments, monitoring and evaluation."