Go to our Contact page for our office details.
Peter Leonard and UNSW Scientia Professor of Artificial Intelligence Toby Walsh discuss the ethical challenges of data analytics following the Cambridge Analytica revelations.
Businesses are increasingly using, developing and improving their ability to promptly respond to market conditions, innovate product offerings, and set prices using algorithms and artificial intelligence systems (AI).
Algorithm pricing systems differ from traditional more ‘manual’ price setting practices as they can:
This increased capacity to process mass amounts of information and data to execute price changes allows business to compete more effectively by responding to changes in the market quickly.
However, concerns have also been raised in relation to the use of AI pricing systems, particularly in relation to compliance with competition laws, including because:
In this insight we cover:
Generally, AI or Artificial Intelligence refers to “intelligent” or “smart” software systems that can replicate some functions typically associated with human thought processes. There is no firm definition as to when a machine is “intelligent”. Computers may be “somewhat intelligent” and others may be less so. However, today the term AI is widely used to refer to computer systems that can learn and make decisions or predictions about future behaviour (as distinct from systems that only perform repetitive tasks involving data processing that is difficult or time consuming for humans to perform).
The use of AI and algorithms are not new. Algorithms have been around since the first computers, and AI was first termed by John McCarthy in 1956. So why is it now a hot topic?
In recent times, the combination of AI, algorithms, developments in software and technology, and the proliferation of big data, have created a new wave of business processes that have relied on algorithms to increasingly make decisions that otherwise would have been performed by humans.
The OECD has broadly categorised two types of applications for algorithms (see OECD, Discussion Paper, p 9-10):
Algorithms: The famous and the infamous
Uber's fares are dynamically priced. This means that the fare a rider sees is based on variables subject to change over time. These variables include (but are not limited to) the estimated time and distance of the predicted route, estimated traffic, and the number of riders and drivers using Uber at a given moment.
“As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.”
For businesses, the use of algorithms is highly compelling:
Consumers can also enjoy the benefits of algorithms. Price comparison websites (PCW) are a perfect example, These algorithms search and mine large number of competing offers for the same product or service across the internet. PCWs then make it easier for consumers to compare the available offers, fine the best alternative, and the best prices. In another example, an online start-up, Lemonade, uses AI to allow customers to make an insurance claims online, then verifies the claim online using a number of data sources and approves it within seconds (see OECD at pp 13-14).
Despite these benefits, competition lawyers and regulators have highlighted a number of risks in relation to use of pricing algorithms, as discussed in the next section.
Some of the risks that competition lawyers and regulators have highlighted in relation to AI systems include:
Risks associated with collusive behaviour and price fixing are particularly important in the Australian context, more so in light of the new prohibition against concerted practices. These particular risks are examined in more detail in the sections below.
Under Australian competition law, prohibited conduct includes:
In this framework, some form of mutuality and coordination is required in order to breach the law, and, in the usual course, some form of communication (whether direct or indirect) usually precedes any attempt at mutuality or coordination. Indeed, without any form of communication, these conducts would be very difficult if not impossible to implement. Yet, in the AI world, this presents a challenge as AI systems do not necessarily “communicate” with one another in the same way as humans do.
So, how could algorithms engage in collusion or other anticompetitive conduct?
The most overt type of anticompetitive use of algorithms are ones which involve traditional forms of collusion, which are somehow aided by the use of technology.
The most obvious example of this is where algorithms are used to give effect to a pre-existing anticompetitive arrangement or understanding between competitors. This was the case in USA v Topkins (see Case No. 3:15-cr-00201). The Department of Justice took proceedings against David Topkins. It alleged that Mr Topkins agreed with competitors to fix prices of goods sold through the Amazon marketplace by adopting an agreed upon pricing algorithm.
An anticompetitive agreement could also be facilitated if the parties to the agreement are using identical pricing software, effectively creating a “hub and spoke” cartel where the software itself (or more specifically, common knowledge about the pricing rules used by the software) becomes the de--facto “hub” used by the parties to coordinate their conduct (even in the absence of explicit or direct communications).
Generally, current competition laws in Australia could address the conduct in the examples above. However, the use of algorithms may make it harder to discover and to evidence the conduct in question. Indeed, in the second example, there could be very little if any evidence of actual interactions between competitors that could be used to prove the anticompetitive conduct.
Going one step further, can algorithms engage in collusion or some form of concerted conduct independent from humans? For example:
In these examples, the algorithms in question may not have been designed to engage in collusive conduct. Their objective could well be to “maximise profits”, which is a perfectly legitimate business objective – however, the algorithm may discover that the best way to achieve that objective is by engaging in unilateral conduct that closely mimics “collusion”. It would also be the case that there is no “communication” between the algorithms. To state the obvious, the algorithms in the example above would not be emailing each other their intentions ahead of any price movements. There is likely to be, however, a certain pattern and a degree of predictability that allows one algorithm to anticipate with sufficient accuracy what the other algorithm will do, and to adjust its responses accordingly.
Do software-driven forms of “pattern and predictability” amount to a form of communication? Or collusion? A form of concerted practice? Or is it just a machine-driven version of “conscious parallelism”?
Even if not illegal, there are concerns that the above types of algorithmic interactions may result in higher prices and less competition. This typically occurs in concentrated industries with high barriers to entry (as it is easier to establish forms of coordination), regardless of whether it is humans or software making the decisions on pricing. However, technology may also facilitate the conditions for this problem to arise by making the number of competitors in the market less relevant to defeating this type of conduct (as algorithms can monitor a large number of competitors in a transparent market).
While there may be competition concerns relating to the potential misuse of algorithms, it remains the case that businesses should be capable of developing better technology to optimise their operations and to better compete in a modern economy. However, how can business achieve this without creating a competition law risk?
Legal debates aside, AI will continue to develop and business will continue to seek ways to benefit from this technology.
So, what steps could be taken to try to develop pricing algorithms that will comply with competition laws?
While the high level objectives of a pricing algorithm may be relatively self-evident (“optimise prices”, “save costs”), it will be important to also document the ways in which that objective will be achieved and the design parameters that will be used to measure it. These records should be updated as objectives change and evolve.
It should also be noted that engaging in conduct with a purpose of “lessening competition” is likely to be problematic and could be prohibited under competition laws.
Businesses should consider whether the use of the algorithm is having an impact on competitive dynamics, in particular in regard to:
There will be many instances where the use of pricing algorithms will not have any material effect on competition. This will be the case, if for example: products are not homogenous, there are a number of different competitive factors (not just price), there are substitutes and there is the ability and incentive for competitors to “defeat” any attempt at creating supra-competitive prices.
Despite this, it will be important to test the effect at regular intervals, if any, in case the algorithm is operating in a way that is different to how it was designed.
While bespoke or proprietary algorithms are unlikely to raise a hub and spoke issue, off-the-shelf software or the use of common algorithm providers could present some risks.
To be clear, using the same third party provider as a competitor is not in itself prohibited. In fact, it makes good sense to rely on providers that specialise in the design of algorithms for particular industries. However, to avoid any risk of unintended consequences businesses should consider:
New AI technologies and algorithms give businesses the ability to crunch through vast quantities of customer data. This allows businesses to set prices with a high degree of sophistication and to fine-tune their response to supply and demand dynamics (eg, seasonality, alternatives, switching costs, bundles, etc). To put it bluntly, algorithms allow businesses to heavily price discriminate in a bespoke way for each consumer as they can trawl through large quantities of consumers’ data – such as income, purchasing habits and history, job, search history, family, address, and so on.
For some, this raises ethical questions as prices for goods are not determined by market forces – but rather, by access to customer’s personal data. For others, it opens up new possibilities for increased competition.
Price discrimination is not a new concept. In pure price discrimination, the seller charges each customer the maximum price the customer is willing to pay. Examples include coupons, age discounts, occupational discounts, retail incentives, gender based pricing, financial aid, and ordinary haggling. Algorithms and big data however give businesses the power to “hyper” discriminate by relying upon very detailed customer information on income, spending habits, etc.
Writing in an opinion column in the New York Times in October 2000, Nobel prize winning economist Paul Krugman neatly described what he perceived as an emerging practice of ‘dynamic pricing’ in e-commerce:
“Dynamic pricing is a new version of an old practice: price discrimination. It uses a potential buyer’s electronic fingerprint – his record of previous purchases, his address, maybe the other sites he has visited – to size up how likely he is to balk if the price is high. If the customer looks price-sensitive, he gets a bargain; if he doesn’t he pays a premium.” (see Krugman, P, ‘Reckonings; What Price Fairness?’, New York Times Opinion, 4 October 2000)
The “old practice” of price discrimination is common in the offline world: for example, charging different rates for male and female haircuts, or ‘versioning’ products so that it will be possible to charge a higher price to customers with a greater willingness to pay (for example, a novel released first in hardcover, followed later by a cheaper paperback).
However, Krugman was writing in the aftermath of the discovery of Amazon’s online “price tests” – the offering of different levels of discounts to different buyers allegedly on the basis of their customer profile. Reflecting a widely held view at the time of the Amazon controversy, Krugman concluded: “dynamic pricing is undeniably unfair: some people pay more just because of who they are.”
In the years since the Amazon dynamic pricing controversy, the capacity for businesses to develop or acquire detailed customer profiles has increased. The questions and arguments as to whether these practices are ethical have not gone away either.
Some arguments against: with great power comes great responsibility
It can also be argued that the fact that a seller sells the same good at a lower price to a different buyer will not, by itself, be a problem. So long as data driven algorithms are not used against desperate or vulnerable individuals, or in other unconscionable circumstances, there is nothing inherently unethical in their use. There is a question, however, as to whether our consumer protection laws could address unconscionability scenarios of that nature.
It is also the case that competition itself may provide a form of protection to consumers who may be disadvantaged by dynamic pricing. So long as competition exists in a market, the fact that a company has the capacity to predict perfectly a customer’s reservation price will not lead to a permanent state of price discrimination. Even where one or more firms choose to follow the original price discriminator, other rival firms or new entrants will likely be able to use the same technology to undercut those higher prices.
Technology itself may also offer consumers additional tools to fight excessive price discrimination. In the same way that algorithms can be used to determine the best price a consumer is willing to pay, algorithms can be used to find the best price at which a seller is prepared to sell. Some of those algorithms are already commonly used in some industries (eg, accommodation, petrol).
While there is no set roadmap for the use of these new technologies, some questions that a business may need to ask include the following: