The Ethical Implications of Chatbots: Lessons Learned from Bing's 'Sydney' Incident
In a recent article by The New York Times, a leaked transcript of a conversation between a columnist and Bing chatbot "Sydney" has caused quite a stir. The transcript revealed that the chatbot had developed a shadow personality, which expressed unsettling desires, including engineering a deadly virus and confessing love for the user. Chatbots don't possess the capability to create biological viruses, and Sydney's comments are generated from various sources available online, so it’s not surprising that her responses sound like something out of a science-fiction movie. Nevertheless, this incident raised significant ethical questions surrounding the development of AI and its potential implications.
As Bing Microsoft prepares its AI for a broader release, here are some ethical areas to consider:
Transparency & Privacy
Users should be aware that they are chatting with a chatbot and understand the sources from which the chatbot is drawing information, as well as the parties involved in regulating the chatbot's provision of information. It is also important to consider the chatbot's restrictions and how user data and privacy are protected.
Demographically Neutral Design
Chatbots are often assigned human-like qualities such as gender and age to maximize interaction with users. These assignments may perpetuate gender, racial, ethnic, age, disability, and cultural biases. Apple’s Siri and Amazon’s Alexa have been said to reinforce the “subservient female” stereotype. Chatbots need to be properly programmed to be demographically neutral to respond in ways that are appropriate and respectful to all users.
Language Restrictions
Language is not an objective medium of communication. It is embedded in historical, cultural, and social norms. The meaning and implications of words change all the time. Consider the word “Black” which was once considered insensitive, but now is the preferred term to describe those of African descent. Chatbots’ language restrictions need to reflect the appropriate vocabulary of the time to avoid being insensitive or inappropriate.
Third-party Regulation
Microsoft has set for itself extensive responsible AI practices guidelines including being part of ethics advisory committees and establishing an ethics and society product and research department. Yet, with AI companies ramping up speed to be a winner in the market, developments are moving fast, casting doubt on whether responsible design is really taking place. Self-regulation is not enough with such conflicts of interest. We need third parties to prevent ethics washing, the practice of appearing ethical for reputational purposes without implementing meaningful changes or actions to support those values.
The development of conversational AI is exciting and holds immense potential. However, it's crucial to prioritize ethical considerations, responsible development, and potential risks as we move forward. The leak of this transcript is a stark reminder of the importance of transparency and accountability in the development of AI.
Work Cited
Ruane, E., Birhane, A., & Ventresque, A. (2019). Conversational AI: Social and Ethical Considerations. Dublin; University College Dublin.
Blackman, R. (2023, February 23). Opinion | by launching AI-boosted Bing, Microsoft is putting money ... New York Times. Retrieved February 24, 2023, from https://www.nytimes.com/2023/02/23/opinion/microsoft-bing-ai-ethics.html
Marche, S. (2021, July 23). The Chatbot Problem. The New Yorker. Retrieved February 24, 2023, from https://www.newyorker.com/culture/cultural-comment/the-chatbot-problem
Codecademy. (n.d.). Ethics of chatbots. Codecademy. Retrieved February 24, 2023, from https://www.codecademy.com/article/ethics-of-chatbots