missing alt

The Pitfalls of AI: Part 2


In last week’s discussion, User Researcher, Lucy Hutchinson and Director of Service Design, Dave Jackson, pointed out the risks of everything from data privacy to bias when it came to potential pitfalls of AI. Here to continue the talk in a second part, are our Head of Analysis, Michael Baines, and Senior User Researcher, Daniel Finnigan.

Forgetting that good data underpins good AI solutions


One of the key aspects in building anything relating to artificial intelligence is data quality. Think of data as the foundations of a building: if you’re using the wrong materials or building blocks, it won't be stable enough to add onto and is more likely to collapse at some point in the future. The same can be said for AI’s dependency on good data that’s robust, free from errors and representative of real-world scenarios to give you the best possible chance of success.

You always need to be aware of bias. Though it can easily creep into data unintentionally, the last thing you want is to build models that perpetuate or even amplify any bias. Starting with data that lacks diversity, clarity, or is skewed in any other way might lead to models that inadvertently reproduce these things when trained and then put into production. This is exactly why it’s so important to identify and address bias during data collection and cleansing so that AI products and models can develop robust outcomes.

Michael urges organisations to invest time in robust data collection and curation processes and remain vigilant so they can make sure the data used is diverse and contributes to the ethical and responsible use of AI. “Data quality and bias go hand in hand.

Not being transparent


AI can operate as a black box, creating a gap between the data that goes in and what it puts out. If things aren’t made very traceable with built-in transparency, it can take away from the quality of the data. And Daniel thinks that starts with the underlying methodology. It's important to think about how you gather data, how it's handled and the ethical implications of the outcomes.

Think about the Horizon scandal, where the postmasters got blamed for an error inside the system. If that had been an AI system with a data gathering question, a black box technology and a whole set of methodology questions underneath it, how hard would it have been to prove culpability of the data? That proves how crucial transparency and careful steps forward are to AI. Good methodology is key; back it up with strong discussions between data scientists and researchers.

Transparency is also vital from a consumer's perspective. There’s a danger when AI products are making decisions that consumers or individuals could deem as unfair. From a GDPR perspective, they have every right to understand the basis of the decision making. It doesn’t look good from a compliance and GDPR standpoint if organisations are unable to explain this basis.

“Think about equality of opportunity in the way that algorithms are being used to select CVs; data is being fed into AI models and if those models hold bias, so does the selection process,” warns Daniel.

Future-proofing data is the way to go when it comes to transparency. You want to have a well-organised database so you can answer any potential GDPR queries in the future. If someone asks what information of theirs is being used, you want tagged data that’s able to provide those answers.

If future proofing data is something you need help with, Transform are always happy to help you evaluate where you are in the AI journey. To talk to one of our experts, get in touch on transformation@transformuk.com. 

Related Blogs