AI: a serious ethical challenge for retailers
Are retailers prepared to take on the challenges and seize the opportunities associated with machine learning and artificial intelligence?
In his 2021 Reith lectures on artificial intelligence, Professor Stuart Russell read an extract from Alan Turing’s 1950 essay on computer machinery. “Once the machine thinking method had started,” he read, “it would not take long to outstrip our feeble powers. At some stage therefore we should have to expect the machines to take control.”
This astonishing notion may seem like a topic better suited to philosophers or AI experts than retailers. But it is critical that any retail business developing or implementing AI solutions carefully considers the relationship between innovation, safety and ethics in AI.
For our sector, never has this been more pressing than today. The boom in e-commerce has provided never-before-accessed insight on customer behaviour, and the cashierless ‘just walk out’ technology sweeping the sector feels symbolic of a new era, one that is underpinned by machine learning and collecting vast swathes of consumer data.
However, while most companies understand AI’s transformative potential, few have real clarity on how to ensure – and on who is responsible for ensuring – its ethical use. Without a coherent strategy and a framework of accountability for the corporate governance of data and machine learning, businesses put themselves and others at risk. And in a highly challenging consumer landscape and a cost-of-living crisis, retailers simply cannot afford to lose customer trust.
Risks of AI in Retail
1. Data protection
At a minimum, incomplete governance around data usage means that any company that collects or processes personal information about EU or UK citizens runs the risk of non-compliance with GDPR guidelines. Moreover, as retailers diversify their services into quasi banking and insurance products, they will soon be put under similar microscopes to the financial services sector in terms of AI and data governance.
2. Pricing algorithms
Companies may also need to rethink the algorithms used in pricing. Thinking back to the early days of the pandemic, the ludicrous prices of antibacterial hand gel provided examples of what could happen when competitive pricing algorithms played out without human oversight.
Indeed, in some settings, we’re seeing algorithms price goods differently for different demographics of customers. It surely won’t be long until ‘variable pricing’ becomes perceived as ‘pricing discrimination’ – especially when groups of customers with similar ethnic and/or socioeconomic backgrounds are faced with more expensive products.
3. Hyperpersonalisation
There are also plenty of risks associated with hyperpersonalisation. One online retailer, for example, reportedly began recommending the products required to create a homemade bomb in its “frequently bought together” section. Another infamous example is of a supermarket, which sent coupons for baby products to a pregnant teenager based on her shopping habits, much to her unaware father’s alarm.
4. AI bias
Perhaps most pressingly, the potential for bias in AI presents huge moral and reputational risks. AI and other cognitive systems are trained by historic data sets collected by real people, and so are almost always laced with the unconscious bias inherent to humans. Retailers not mitigating against this bias, either in internal recruitment systems or in customer-facing algorithms, run the risk of damaging their employer brand and losing customer trust.
Against this backdrop, it is critical that retailers put in place a robust governance framework for the ethical use of data and AI. If AI technology malfunctions or produces unfair results, can the creator be held accountable? Is there recourse for consumers in the form of an accountable engineer, an internal governance board, or even external governance?
Who is responsible for AI malfunctions?
Research from The MBS Group published earlier this year found that most consumer-facing businesses cannot answer these questions. Very few companies have a structure of accountability for the ethical use of AI, leaving them unprepared to take on the challenges and seize the opportunities associated with this technology.
In retail, maintaining a business’s ethical values, guaranteeing legal compliance, and ensuring that new technologies are being used optimally requires leadership from the non-executive Board.
Today, many NEDs recognise the importance of data and ethics, but few are equipped with the expertise needed to take final responsibility (indeed, most NEDs will have been executives at a time when data or AI was not as integral to a company as it is now). Across the sector, Boards need to grasp the power and risks associated with data, and include at least one non-executive director who can participate in – and lead – conversations around ethical use of AI. Most Boards aren’t here yet – and Chairs must think carefully about making appointments which can provide sufficient checks and balances to the executive.
How to effectively govern AI in retail
We recommend four actions for retail Boards to effectively govern AI:
- Separate AI ‘doing’ from governance – The team responsible for AI innovation and technological development should not be responsible for governing its ethical use
- Make ethical use of data and AI a standing item on risk committee – AI should be a regular item on the risk committee agenda. Consumer setting up a standalone ethics committee.
- Identify one NED to be responsible AI, data and ethics – Name one non-executive board member, who is appropriately trained and/or qualified, to be responsible for AI, data governance and ethics on the board
- Measure and audit – Bring in third-party bodies to independently audit algorithms and policies around the collection and implementation of customer data – reporting directly to the Board
How equipped is your business to grapple with this new horizon? Do let me know your thoughts – or indeed any best practice you have to share from your own organisation.
To find out more about The MBS Group and the services they provide to the retail industry, click here.
This article was also published in The Retailer, our quarterly online magazine providing thought-leading insights from BRC experts and Associate Members.