A number of my friends and previous colleagues made me aware of a series of interesting links after I asked them for more details on how chatbots like ChatGPT are programmed and trained. I also wanted to share with them my worries about the societal consequences. Here is a collection of the links we exchanged including short excerpts of opinions and explanations. The dates refer to day of exchange a message. The links are sometimes much older.
- Kevin Roose reported on his two hour conversation with a test version of Microsoft’s Bing’s chatbot. Here is the transcript as published in the New York Times, dated Feb 16, 2023. The Guardian published a commentary by Jonathan Yerushalmy on Feb 17, 2023. I asked my colleagues for comments and explanations. Feb 21, 2023.
- How are the various languages treated that are used on the internet treated? Are they all used by automatic translations to and from English? It appears that for different language groups different models are used. See Meta’s project “No Language Left Behind”, which aims for direct translations between about 200 (sometimes obscure) languages. The paper they published is pretty long. Section 6.3 reports how they deal with self-supervised learning to get a good (semantical) representation of the various languages. Feb 22, 2023
- Some interesting insights in machine translation are found here in a presentation from DeepMind. Check from view graph 104 onward or from 1:23:08 in this video. They use self-supervised learning to get a good representation of two languages and they claim they then can make the translation work by just a few hundred supervised examples. Feb 22, 2023
- ChatGPT was most likely trained on a corpus of many languages, see this UCL-DeepMind lecture by Irina Higgens and Mihaela Rosca. Whether that has retained the subtleties of different perceptions of the world in different languages is an open question. Feb 22, 2023
- The incredible number of free parameters in these systems is a fundamental problem. ChatGPT’s GPT3.5 has about 175 billion free parameters, whilst the successor GPT4 supposedly might have 1 trillion free parameters. These systems must be underdetermined and prone to mistakes. There is an interesting article by Wolfram. Feb 22, 2023
- Problems of the large language models are, among others, discussed by Emily Bender and Timnit Gebru (paper). Feb 23, 2023
- Large language models are trained on huge corpus of text and apparently they learn content and style which allows the models to perform language related tasks. In the case of ChatGPT besides unsupervised and supervised learning, it also includes a human in the loop reinforcement learning. They use the latter for fine-tuning, correcting, and also to incorporate fixed answers to given questions that for some reason should not be answered (ethical, legal).
However, people have realized that by asking things differently they can get answers that are usually banned. Therefore, it is still work in progress and they are using the current free interactions as data for the reinforcement learning. Google had a language model already before ChatGPT, but they did not dare to publish it due to the same issues we are seeing now. I do think that for some applications and tasks it is a useful tool. However, I absolutely agree that to open this tool to the general public it requires a framework including transparency, explainability, traceability, and description and disclosure at all levels of the system training and functioning. Hallucination is a known issue of the system for example, it cannot always be trusted for the answers it gives. Feb 23, 2023 - Another article by Stephen Wolfram on ChatGPT. Pretty long, pretty insightful. It does not describe the ‘reinforcement learning with human feedback’ aspect of ChatGPT. March 16, 2023
- This video may clarify many aspects of the ChatGPT models, including potential harms. In general they are trained using all the text resources. The models predict the next word in a sequence, so if the prompt sent to it somehow reflects a strange type of interaction it is likely that the model will respond also with strange interaction. It is just a statistical model in the beginning. On top of it, they are trying to improve the answers for those questions and interactions where the tools may provide misleading, harmful, or non-factual answers. March 16, 2023
- RLHF (Reinforcement Learning from Human Feedback) can be very important to make ChatGPT safer and more sensible than Bing’s chatbot. It may be the case that many many hours where necessary for this fine-tuning operation. If you ask ChatGPT how it feels it says: “As an artificial intelligence language model, I do not have feelings. However, I am programmed to simulate human-like responses to facilitate conversations and provide helpful responses.”. March 16, 2023
- An open letter to pause giant AI experiments. In a concept was also the name of Bill Gates. In the final version this was gone. Interesting in the light of Microsoft’s interest to construct Bing, a commercial version of ChatGPT. March 28, 2023.
- A short column by Moshe Vardi (entitled “Who Is Responsible Around Here?” in the latest issue of Communications of the ACM) might be of interest. March 28, 2023
- The next iteration of ChatGPT, GPT-4, with an extensive overview of its capabilities. April 5, 2023
- A good, general discussion between Joshua Bengio & Tom Bileau (Tom invites interesting people to talk to). The Dangers Of AI Are WEIRDER Than You Think. Another good talk is with Donald Hoffman about the primary aspect of consciousness. April 14, 2023
- New York Times interview with Geoffrey Hinton when he announced to leave Google. Shocking that he regrets what has been the result of his research. May 1, 2023
- A BBC follow up on this news. May 2, 2023
- A NRC follow up on this news (in Dutch), May 3, 2023
- Harari in The Economist, This is behind a paywall. Just a characteristic phrase here: “AI tools have emerged that threaten the survival of human civilization … by … some remarkable abilities to manipulate and generate language, whether with words, sounds or images. AI has thereby hacked the operating system of our civilization“. May 4, 2023
- Jaron Lanier: There is no A.I., an article in The New Yorker: “There are ways of controlling the new technology—but first we have to stop mythologizing it.”.
- Chatbots Don’t Know What Stuff Isn’t. The struggle of chatbots with negation illustrates the difference between statistics and understanding concepts. May 15, 2023
