Conversations on Ethical AI: Workshopping Methodology 

By Mark Findlay and Josephine Seah

 

CAIDG’s AI Ethics Hub 4 Asia ('The Hub') is running a set of conversations on ‘Ethical AI’ with AI practitioners. These conversations are stepping stones in the development of a method that might be used to deepen our understanding of the place of ‘Ethics’ in AI Governance. 

The following are conversations snippets and some preliminary observations from our first workshop held on 2 March 2020. 

---

Participants 

• Mark Findlay. Deputy Director, CAIDG 
• Josephine Seah. Research Associate, CADIG 
• Young AI Practitioner. Software engineer of an online marketplace company. 
• Young AI Practitioner. Currently working with organisations to automate routines and previously involved in automating decisions in large platform. 
• Experienced AI Practitioner. Marketing/Advertising background with wide engagement in areas of data management for enhancing decision-making. 
• Experienced AI Practitioner. Worked in banks with experience in Fintech, consulted for the development of the Monetary Authority of Singapore’s FEAT Principles. 

 

General Discussions Points

Participant introductions, comments on how they currently use AI and big data in their work, and their other experience in the industry. General discussion of what issues concern participants regarding AI, big data and ethics.

• ‘I don’t think we have might choice giving away our data — big data allows people to know regardless of you submitting your own data willingly (e.g., if you don’t want to submit your data that is also a category that can be profiled)’
• ‘…there is too much talk about ethics, all organisations have an ethics code the same way that they have a motto — but when it comes to ethics and AI, the starting point should be about data — should we be masking certain attributes when profiling to avoid unfairness and harm?’

With the assumption that ethics are useful in dealing with big data and AI participants commented on whether ethics influenced their work in the AI/data fields.

• ‘I’ve worked with user log data that sometimes contains personal information, even though we are just using it for good purposes such as improving advertising and preference matches, there is concern about a data breach and this data leaking and being used for other problematic purposes — I believe that a lot of people are aware of these considerations.’
• ‘In my experience, it seems that people are mostly unconcerned about ethical breaches. But when it comes to combining big data, for everyone there is a line where they ask, ‘Why are we doing this?’ — but that line is quite far off and sometimes it is very grey.’
• ‘No, ethics is mostly reactionary, an after-thought, everything that Google has done to react to ethical challenges has been after influence of the GDPR — it really is about defining ethical data use, e.g., is it okay to consider gender when looking at data for credit evaluations? How can we set boundaries for ethical data use?’
• ‘Ethics is very much principle based, and differs geographically but also shifts due to profit considerations or competition in particular market settings — but also, computer scientists are not sufficiently learning ethics or secure coding in their curriculum, and most of technical training is focused on efficiency — recently companies are expected to have ESG reports, perhaps something similar should happen for ethical AI?’
• ‘I think people at the work face want to be ethical, and companies know that an ethical reputation will attract the best workers.’

Next, participants were asked they considered to be the most important ethical principle which should influence front-line work with AI and big data.

• ‘Don’t harm / non-maleficence.’
• ‘Data is about awareness but the definition of harm is so difficult — defining boundaries of use is so important — we are too caught up in harm to humans, what if we changed this to acknowledging harm to humanity?
• ‘Protect your reputation and that of the company.’ 

Then participants were asked to consider a list of ethical principles that had been drawn and prioritised from a survey of AI ethics guidelines.

• ‘Looks similar to moral values, which might be more of a minus than a plus’
• ‘Actually more plus because these might prevent minuses (i.e., bad things from happening)’
• Regarding the details of these principles and how they are understood: ‘if you were talking to R&D developers then they would know, because their peers might strike them down.’
• ‘Developers should know, but I’m not sure if they are concerned about it.’
• On explainability and transparency: ‘from my own experience working on explanations for automated decisions — ultimately explained through examples: a compromise between what is actually inside the model — not really maths, but how the decisions is determined by its inputs.’
• ‘Unsure about the utility of anything beyond the top 3 (transparency, justice and fairness, non-maleficence). For example, sustainability is not an AI question . The first three are highly relevant, but everything is so context specific, even when it comes to fairness what it means to be fair really comes down to a question of context.’
• On how the principles are dependent on market contexts: ‘if we removed competition will companies be ethical? Most likely not, competition prompts ethics.’
• On bias: ‘look at sustainability, corporations have to have a sustainability report, can there be something like that for AI Ethics? Something that is trackable. Look at the recent case of Apple’s Card and its credit scoring algorithm, data issues here were due to biased data.’
• ‘Not just an issue of biased data, but when the data is submitted also influences the outcome as well.

 

Responding to Hypotheticals

At this point participants were presented with hypotheticals featuring key decision-sites that arise in project development and asked to respond to them. 

Hypothetical One — featuring biased data, law enforcement decisions leading to unfair outcomes, clients refusing to identify pre-existing decision biases 

• ‘The challenge is that you as the designer are given a black box.’
• ‘2 solutions here: (1) define biases and then clean it out, and (2) build a solution based on abstract data, because it might not be worth it to clean the data in the first place.’
• ‘Prediction is something that is application specific — what they do with the prediction by profiling is something else, e.g., putting people on a watchlist — is there a benefit here that society needs? What is the problem here that you are solving for?’

Hypothetical Two — featuring facial recognition and the risk of data marketing 

• ‘Data privacy issues are the challenge here, we currently have too much of an opt out system right now, we should have an opt in system instead. We are told that our data is being shared with third parties, but you need to know what kind of third parties and for what purposes.’
• On the choice to opt-out: ‘even by making this choice (of opting out) you still become classified.’

Hypothetical Three — featuring allocative harms and data leakage in health records 

• ‘The program is not robust enough to address the underlying problem.’
• ‘Data leakage is a problem, the solution is to segregate the data or to protect private data. Whether people will accept that on a tight timeline is another question.’
• On shared responsibility: ‘Post-GDPR, getting client data, we had a data governance council that involved the data team because we were liable — we need platforms/forums for discussion and raising these issues. Another issue is: what happens when developers are external to your organisation?’

Hypothetical Four — featuring corporate data auditing and market pressures

• Responding to the notion that data cannot be audited because it can’t be explained: ‘this is unprofessional.’
• On shifts in company culture ‘Yes, start-ups in particular usually operate in an extreme market environment — idea is to solve everything now and quickly — only once they’ve grown to a size that they cannot side-step the question of ethics (reputation) do they start to address this — previously the idea of ‘we are just a tool’ cannot stand up to scrutiny and reputation damage from a loss of trust — had to intervene in communications between the platform and its users’

Hypothetical Five — featuring the meaning of ethics and the role of the end-user 

• On whether language is the problem ‘No — if you create the gun you should have some responsibility for its abuse (if the creation is difficult to regulate such as digital printing) — should build something that is possible to regulate.’ 
• ‘It’s like patches as fixes — why isn’t that framed as a possible solution?’
• ‘Designers might not actually have the responsibility — but another issue is that larger companies, big tech firms like Google and Facebook, are setting bad examples’

---

Preliminary Observations 

 

Moving past the identification of ethical challenges

How might one intervene once they see something that might be problematic? It’s clear that the challenge is not simply identifying problematic uses of data or the ethical quandary, but rather what do we (as individuals and teams) do to help people understand both why that challenge occurs (in the context of production, in the context of their own organisation) and what they might do once they see something that should be questioned.

When it came to suggesting solutions, participants mentioned not using data presented, or correcting for bias within the dataset, or removing key attributes like gender/ethnicity. No one brought up the use of existing toolkits such as IBM’s AI Fairness tools or Google’s What-If Tool, both of which were launched with relative fanfare. If one way of addressing ethical challenges is to look at possible points of intervention — during data collection, during model development, during testing — it would be useful to know whether these toolkits are on practitioners’ radars, and whether they’re seen as a viable option for companies to audit their models.

 

Expanding the notion of ‘shared responsibility’

If, again, it is less a question of identifying ethical challenges, then this goes back considering whether the issue is more about developing shared responsibility within building an AI-enabled product/service.

What we’re seeing is that it might be less about what ethical principles a company might have in their ethics documents, but what routes exist within the organisation to enable those certain principles to be realised — and conversely what gaps prevent this. For developers, their position/age within the hierarchy of the organisation might be made explicit as a problem for ethical attribution and assertion. Is there a risk for their job security by wanting to bring ethics into the conversation? Is that possible to generalise when some organisations value ethical reputation more than others? No doubt there is a need to have platforms and forums discussing and identifying challenges for ethics so that concerns might be raised. How do we address developers’ individual responsibility and their specific company cultures?

All of this speaks to transparency within the organisation itself — perhaps something is required as a pressure valve for ethical identification preceding the more dramatic whistleblowing policies? How does one decide who should sound the alarm up to hierarchy of the organisation? 

Aspirations for transparency can break down between organisations when they share design activities, and also connecting the end user — less of a case where a single organisation is involved in data collection, cleaning, labelling, augmentation, classification, and profiling. If so then what are the lines of communication between organisations when a data set gets handled off, or when a model is built and sold to a user? What would shared responsibility be in that scenario and how could it be policed?

 

Tension between production pressures and reputation-preserving tactics

This was a recurrent implication in the discussion without being directly addressed other than, “If so, I’m not sure what to do”. For example, the profit motive/competition motive leads to companies taking dubious decisions, or the start-up environment that tends to be extreme in a ‘move fast and break things’ way. And what about the commercial/operational value of the connection between ethical practice and reputation as an individual and organisational benefit?

---

The Hub is a space for developing conversations about the impact of ethics and principled design by looking at the whole anatomy of AI development and big data use. More information about The Hub can be found here.