top of page

Be careful what you ask for! Bias in ChatGPT

ChatGPT, an AI chatbot, is the latest large language model making waves in the tech world that has endless possibilities across industries.

ChatGPT models are fed massive amounts of text and data from the internet – credible sources, questionable sources, incorrect sources – all subject to potential bias. ChatGPT consumes these data to respond to questions, explain complex subjects and create ‘original’ content.

However, massive amounts of internet data does not necessarily reflect diversity or ensure integrity nor reduce/eliminate bias.

As such, even population or geographic regions who have better access to the internet or those groups who spend more time engaged on the internet may influence data predictions and outcomes.

While ChatGPT is not inherently biased, sources of bias in language models:

✅ One of the main sources of bias in language models is the training data itself. For example, if a model is trained on data primarily from the east coast, the model may not perform as well when used in other areas of the country.

✅ Another source of bias in language models is the way the model is designed and trained. For example, is a model is trained using labels created by humans, the labels themselves may carryover potential human biases.

To address this, companies and analytics groups are creating robustness metrics, alerts and data standards to weed out not only potential bias but also factually incorrect outputs from language models.

In addition to corporate guidelines and individual responsibility, many experts believe there should be a governing body with regulatory oversight to ensure fairness and consistency across these technologies.

ChatGPT’s maker, OpenAI, said they have been focusing on cleaning up its dataset and removing examples where the model has had a preference for things that are false and they are working to reduce the chatbot's biases and will allow users to customize its behavior.

There is no doubt that language models such as ChatGPT have enormous potential to enhance human understanding and learning. Still, as with any disruptive technology, there is much to discover, both positives and negatives that will help us optimize the technology in the most beneficial way.

As would be expected, there are specific challenges and biases unique to individual industries. In future posts, we will explore those biases for the healthcare industry.

Does your organization have a plan to address potential bias in your language models? 📝

Follow us on Equilibrium Point Health for more trending topics on AI/ML

Learn more ➡️


bottom of page