Data Ethics – What Is It Good For?

Data ethics might be complicated, but it is essential to business success

The field of data ethics (or tech ethics, or AI ethics, or responsible AI…) has been having something of a moment. A recent study showed there were close to 100 sets of data or AI ethics principles out there. (Full disclosure: I’ve worked on some too!)

There’s been enough of a trend that it has already got its own backlash on various grounds: that the ethics are good but not practical enough to make decisions (and so we need tools…); that (some) organisations are building principles in an attempt to avoid regulation (ethics washing); that human rights legislation already protects people and the attempt to make AI rules look like a different thing is to duck engaging with that, as well as contributing in itself to a story about the inevitability and power of tech.

Given all this, if you’re in the tech business, what do you (or might you) get out of having data ethics in your organisation? If organisations are going beyond compliance with the law, what are they trying to achieve?

It seems that many leaders and workers inside organisations are aware of the power of their tech and genuinely concerned about possible harms to society – and obviously this is a key reason to aim at developing ethics to guide their projects and company direction. However, there are also reasons touching more on reputation.

Working out the importance of different factors can help shape your decisions about how to build in tech ethics. For instance, if a key motivation is having an ethical reputation, it probably makes more sense to sign up to an external and reputable ethics code rather than build your own.

Protecting public reputation

Beyond concern about doing good, one of the key reasons for looking for AI ethics credentials is marketing to the public. There is both justified and unjustified fear about new technologies – for instance, that your always-on microphone in your home is listening to you, or that jobs are at risk and surveillance can hurt us.

Companies choosing to make positive statements of principles around things like risk mitigation and explainability can help tell a different story. Especially for smaller organisations trying to compete with incumbents, positioning themselves as the ethical alternative can work for business.

A common critique, especially of the larger companies, is that the ethics is limited in scope – they may try and make ad serving less biased, but won’t change the fundamental ‘surveillance capitalism‘ revenue model of amassing as much behavioural data as possible, even if that seems like an ethical option. Start-ups do have the potential to show a difference.

In some cases, ethics and business are aligned – for instance, ‘explainability’ is often included in ethical principles because it allows human autonomy, and explainable systems may also be more trusted and therefore more likely to be purchased.

Likewise, accuracy is sometimes seen as an ethical goal (tied to an ethical principle of beneficence – ie. you are actually helping people) and having an accurate machine learning system, and knowing what cases are beyond the limits of the model, is also often sound commercial sense.

When data and ethics collide

In other cases there may be conflict. One of the most famous recent examples is a hiring algorithm developed by Amazon to automatically prioritise CVs for hiring.

The company took the ethical step of getting an external audit of the tool to assess for historic bias – in particular, there was concern that the algorithm, trained on historic hiring practices in the tech area that prioritised white men, would replicate that.

The audit found that this was indeed happening – even after removing names and signifiers like ‘Women’s,’ the machine-learning tool was powerful enough to find other patterns that acted as proxies for gender, including word use by candidates and CV items like ‘softball’ (which is played by more women in the US).

Amazon decided that the tool was not fit for purpose and did not continue with its use. This sounds like a positive ethical endeavour by the company – testing a tool for bias, finding it isn’t fit for purpose, and not using it.

However, a lot of reaction was negative – because of the associated evidence that historic hiring had been so biased that an algorithm couldn’t unlearn it.

Attracting (and keeping) tech workers

We are seeing increased awareness from data scientists themselves that the technology can be problematic. There have been university protests around allowing Palantir at campus recruitment fairs because of its problematic work with immigrant deportation and police forces.

Other tech companies are also receiving more scrutiny. The burgeoning of ‘tech for good,’ non-profit organisations also shows that tech workers want to do meaningful, ethical work. Showing ethical credentials will help recruit tech workers – but if it is for show rather than enacted in daily culture, they may not stay for long!

A part of most ethical principles around data and AI is giving people a voice, including building diverse teams. But diverse teams are only valuable if the people within them can express their own perspectives and discuss the ethics of their work.

Google had AI ethics principles but continued work on Project Dragonfly (to build a censored search engine for China). Although this was eventually abandoned – in part because of workers’ protests on ethical grounds – it is still an example that shows that merely creating principles is not enough to be ethical, and that workers will notice that!

It is also worth mentioning here that a lot of the work in tech is done not by these high-valued, qualified programmers but by an invisible and often sub-contracted workforce of content moderators and data labellers – ‘ghost workers’.

These workers may not have the same power to hold companies to account (indeed, if sub-contracted, they may not even have visibility on which companies they’re ultimately working for) but there has been some organising around mutual workouts and solidarity, which means that the in-house workforce will also be watching how contractors are treated.

Getting started with ethics in your business

For a company that wants to do (and show) ethical AI, having some principles or values is a first step.

There may be tension between reputation gains and having practically applicable ethical principles for your organisation: a company may find that none of the main sets of ethical principles out there fully work with their sector or business model.

Building your own values in-house is a bit like building software yourself rather than buying something off the shelf – you can get a better fit for your work, but it can also mean more resource to develop and maintain, and may fail to hit expected performance (or ethical) standards.

In the case of making your own ethics principles, it can also look like ‘special case’ pleading to try and avoid meeting stronger external requirements. You should have a good reason for not sticking to a well-endorsed set, and be prepared to explain that choice.

In the non-profit sector this happened when I helped build an ethical framework for digital health for AMRC (the Association of Medical Research Charities). While many principles out there were similar to what they needed, we ended up putting together, and justifying, a bespoke set. Linking these explicitly to other people’s work, and showing that the aim of the charity sector was to meet a higher standard, was crucial in this work.

Application anxiety

After developing these principles or guidelines, applying them in practical questions of AI projects and data science is another issue. Using tools that allow for agile iterative development while also incorporating multiple views on what is ethical, including from outside the organisation, is an ongoing problem.

For more from our expert guest contributors, sign up to our weekly newsletter here.