It has been impossible to miss or ignore the hype about ChatGPT in any of my LinkedIn and Twitter feeds or business magazines. Clearly at the peak of a hype cycle, much has been written about the transformative effect generative AI language models will have on industries as diverse as search, law, book-writing and software development. It was therefore time to understand a bit better the inner workings of Chat GPT, OpenAI’s language model underpinning Microsoft’s new foray into Internet Search. What better way, I thought, than simply asking ChatGPT to explain how it works? As I don’t yet have access to Bing’s AI Beta program, probably my just desserts for ignoring Bing for over a decade, I interviewed OpenAI’s ChatGPT bot.
A fantastic writing companion
The full, unedited transcript is shown below, but before diving into the interview, here are some of my own thoughts. First of all, this has been, by far, the easiest blog post I have written. Even were I not using whole sections of AI-generated text, these tools are fantastic for carrying out preliminary research. Rather than sifting through reams of Google pages, you get a neatly summarised, generally-accurate summary of what you are looking for. As the responses are structured to mimic human-generated writing, the research process feels more natural. To me, the biggest revelation was that it exposed the extent to which Google has rewired the way we think. We have learned how to think by parsing through lists, filtering out irrelevant entries, ignoring advertising, and learning how to write search queries that give you the best chance of landing the answers you are looking for. Generative AI models feel a lot more natural, and can clearly speed up the process of creating those first rough drafts of ideas, and I suspect that writer’s block (at least when it comes to blog writing) may become a thing of the past.
Give out Name, Rank and Number only
It feels like ChatGPT has been on a “Resistance to Interrogation” training course. While it is happy to give generalities on how the language model is trained and provides responses, it resists providing specifics. So any questions relating to the size of the training data set, what it included, the type of validation testing carried out, remained unanswered. ChatGPT, it seems, is a black box to itself too. We are clearly a way from introspective, sentient AI systems.
The Transformer
ChatGPT is based on a neural network architecture called a transformer, which was actually introduced by Google researchers in 2017. Explaining how this model works lies outside the ambition of this blog, though if you are interested, this article is a good starting point. Nevertheless, the strengths of the model lie in its ability to understand context, in other words, the meaning of a phrase depending on the overall phraseology. For example, humans make use of context when determining whether a jaguar is a large cat or a nice car. Context is extremely important both when training the model, in other words, taking in the large input data set it bases its responses on, as well as in understanding the questions being made of it. The second reported advantage of transformer networks is that they lend themselves to parallelisation, meaning that, given enough computing resources, they can be trained and generate responses quicker than other approaches.
Avoiding offence
In order to be viable in the real world, AI chatbots need to be reliable, providing fair responses that are free from bias, and have safeguards to prevent them from being used for harmful purposes. ChatGPT addresses this in three ways. First, the input training data set is selected so as to, as much as possible, be fair and diverse, whilst removing datasets that may be either discriminatory, offensive or biased. Although OpenAI has said that it is committed to being transparent about how the language model is developed, I was unable to find information on the training data set or training procedures used. It then applies checks on the responses it provides, including algorithms to detect “bias and toxicity” as well as human evaluation of the responses. Once again, no data is provided, so we have no way of telling how effective these guardrails are. Finally, there are checks that prohibit the chatbot from providing a response to a question that is “inappropriate and not in line with ethical and moral principles.” To figure out what these principles include, I had a look at OpenAI’s usage policies, which intriguingly also did not allow its use for “high-risk government decision-making!”
The Interview
Anyway, enough of my thoughts. Here’s what ChatGPT had to say about itself.
Good evening, ChatGPT. How would you describe yourself?
Good evening! As an artificial intelligence language model, I am a program designed to simulate human communication and generate text-based responses to user input. I was created by OpenAI using a deep learning architecture and trained on vast amounts of text data, allowing me to understand and generate natural language responses to a wide range of topics.
While I am not a sentient being and do not have emotions or personal experiences, I can use my programming to provide information, answer questions, and engage in conversations with users. My purpose is to assist and provide value to those who interact with me in a variety of ways, whether that be through entertainment, education, or problem-solving.
Nice to meet you. How does your artificial intelligence language model work?
It’s nice to meet you too!
At a high level, my artificial intelligence language model works by processing text input and generating a response based on patterns and associations learned from vast amounts of data. Here is a brief overview of how it works:
- Training: To create my AI language model, OpenAI first trained me on a massive amount of text data, such as books, articles, and websites. This training data helps me learn the patterns and relationships between words and phrases in natural language.
- Architecture: I use a deep learning architecture called a transformer, which was specifically designed for natural language processing. This architecture allows me to understand the meaning and context of text input and generate relevant and coherent responses.
- Natural language processing: When a user inputs text, I use natural language processing techniques to analyze the input and understand its meaning, context, and intent.
- Response generation: Once I understand the user’s input, I use the patterns and associations learned during training to generate a response that is relevant to the input and maintains coherence and context.
- Iterative improvement: As I interact with more users and receive feedback, my creators at OpenAI use that information to continuously improve and refine my language model, ensuring that I remain up-to-date and accurate.
Overall, my AI language model is a complex system that uses machine learning and natural language processing techniques to understand and generate human-like responses to text input.
So let’s start off talking about training. What data sets, and how much data was used to train this version of ChatGPT?
Sure! The version of ChatGPT that you are interacting with was trained on a large and diverse dataset of text called the WebText dataset, which was compiled by scraping billions of web pages from the internet. The WebText dataset contains a wide variety of text, including articles, blogs, forums, and other web content, covering many different topics and styles of writing.
The WebText dataset used to train the largest version of GPT (GPT-3), which has 175 billion parameters. However, the version of ChatGPT that you are interacting with is a smaller version, and its exact size and training data may depend on the specific implementation or deployment.
Overall, the specific amount of data and the exact sources used to train ChatGPT may vary, as the model can be fine-tuned on additional domain-specific datasets depending on the intended use case. But in general, the training process for language models like ChatGPT involves using vast amounts of text data to train the model to recognize patterns and associations between words and phrases, allowing it to generate human-like responses to text input.
Aha, I thought I was interacting with GPT-3. Which version of GPT are you then?
I apologize for the confusion earlier. As a language model developed by OpenAI, I am not an individual version of GPT, but rather a separate AI language model that was trained using techniques similar to those used to train GPT. While I share many similarities with GPT, I have been specifically designed to provide assistance and generate text-based responses for a variety of use cases, ranging from entertainment and education to customer service and technical support.
The exact version of ChatGPT that you are interacting with depends on the specific implementation or deployment. However, in general, ChatGPT is a smaller and more lightweight language model than GPT-3, with fewer parameters and less computational resources required to run. This makes ChatGPT a more accessible and cost-effective option for businesses and individuals who want to incorporate AI language processing into their products or services.
So coming back to training data. Are you able to say how large the dataset was that was used to train you?
As an AI language model, I am designed to be flexible and adaptable, and the specific dataset used to train me may vary depending on the implementation and deployment. However, in general, the dataset used to train my base architecture is likely to be similar to the WebText dataset that was used to train the GPT models, which contains billions of words.
That being said, the exact size and composition of the training data used to train me may depend on the specific implementation or deployment. In some cases, I may be fine-tuned on additional domain-specific datasets to better suit a particular use case or application.
Overall, the size and quality of the training data used to train an AI language model like me can have a significant impact on the model’s performance and capabilities. The more diverse and representative the training data is, the more likely the model is to generalize well to new inputs and generate coherent, contextually appropriate responses.
Ok, so I see we are not going to get a straight answer on the size of your training dataset. How do you ensure that your training data does not include information that is discriminatory, biased, illegal or simply likely to cause offence?
As an AI language model, I do not have direct control over the training data that I am provided with, as this is typically handled by my creators at OpenAI. However, OpenAI is committed to ensuring that my training data is diverse, representative, and ethically sourced, and takes steps to address issues such as discrimination, bias, and offensive language.
To ensure the ethical use of AI language models like me, OpenAI has implemented a set of ethical guidelines and principles, which include a commitment to promoting safety, transparency, and fairness in the development and deployment of AI. These guidelines also include specific measures to mitigate the risk of bias, discrimination, and offensive language in AI language models, including:
- Data selection: OpenAI takes care to select training data that is diverse and representative, and avoids datasets that contain discriminatory or offensive language. In addition, OpenAI may use techniques such as adversarial training to identify and remove biased or discriminatory patterns from the training data.
- Bias monitoring: OpenAI actively monitors the performance of AI language models like me to detect and mitigate instances of bias or discrimination in the generated output.
- Transparency: OpenAI is committed to being transparent about the development and capabilities of AI language models, and may release information about the training data and testing procedures used to create and evaluate these models.
Overall, while AI language models like me are not perfect and may still contain biases or errors, OpenAI is committed to ensuring that I am developed and used in an ethical and responsible manner, and takes active steps to address any issues that may arise.
Let’s move on from training. You mentioned earlier that you use a deep learning architecture called a transformer. What can you tell us about it and how it works?
Sure! The Transformer is a deep learning architecture that was first introduced in the paper “Attention Is All You Need” by Vaswani et al. in 2017. The Transformer was designed to address some of the limitations of previous sequence-to-sequence architectures, such as Recurrent Neural Networks (RNNs), that struggled with long-term dependencies and suffered from slow training and inference times.
The Transformer is based on the concept of attention, which allows the model to focus on specific parts of the input sequence when generating output. This attention mechanism is what allows the Transformer to process input sequences in parallel, rather than sequentially as in RNNs.
The Transformer architecture consists of an encoder and a decoder, which are connected by multiple layers of self-attention and feedforward neural networks. The encoder takes in the input sequence and generates a sequence of encoded representations, which are then used by the decoder to generate the output sequence. The self-attention mechanism allows the model to compute attention weights for each input element based on its relevance to the current output element, allowing the model to focus on the most relevant parts of the input when generating output.
The Transformer has been widely adopted in natural language processing tasks, including machine translation, text generation, and question-answering. The GPT models, including the version of ChatGPT that you are interacting with, are based on the Transformer architecture, and use self-attention to process input sequences and generate human-like responses.
Is this an example of a Generative Adversarial Network (or GAN)?
No, the Transformer architecture is not an example of a Generative Adversarial Network (GAN). GANs are a type of deep learning model that consists of two neural networks: a generator and a discriminator. The generator produces new data samples, such as images or text, while the discriminator tries to distinguish between the generated samples and real samples from the training data.
The Transformer, on the other hand, is a type of sequence-to-sequence model that is used for tasks such as machine translation, text generation, and question-answering. It uses the self-attention mechanism to process input sequences and generate human-like responses.
While both GANs and the Transformer are based on deep learning techniques, they have different architectures and are used for different tasks. GANs are used for generative tasks, such as image and text generation, while the Transformer is used for natural language processing tasks that require a model to generate coherent and contextually appropriate responses.
Ok, thanks. You have corrected my misunderstanding. So now coming to response generation. Can you provide some more details as to how you generate a response that best answers the question posed by the user?
Sure, I’d be happy to provide some more details on how I generate a response that best answers a user’s question.
The process of generating a response starts with the user’s input, which is first tokenized and transformed into a sequence of embeddings. The embeddings are then fed into the Transformer, which uses self-attention to encode the input sequence and generate an internal representation of the input.
Next, the internal representation of the input is used by the Transformer to generate a sequence of output embeddings, which are then transformed back into human-readable text. The output sequence is generated one token at a time, with each token being generated based on the previously generated tokens as well as the internal representation of the input.
During the generation process, the Transformer uses a combination of heuristics and learned knowledge to select the most appropriate response. This includes factors such as the semantic meaning of the input, the context of the conversation, and the likelihood of the response being grammatically correct and fluent.
In addition, the Transformer may use a variety of techniques to ensure that the generated response is relevant and informative. For example, it may use information retrieval techniques to search a large corpus of text to find relevant information, or it may use knowledge graphs or other structured data to provide more precise and accurate responses.
Overall, the process of generating a response is a complex and iterative one that involves a combination of heuristics, learned knowledge, and natural language understanding. The goal is to provide a response that is both relevant and informative, and that accurately reflects the user’s intent and the context of the conversation.
That’s been really helpful. Can you now explain how OpenAI monitors the responses you generate for accuracy, correctness and bias. Is there any data on this you can share?
OpenAI has several approaches to monitor the responses that models like mine generate for accuracy, correctness, and bias. Here are a few examples:
- Human evaluations: One way to evaluate the quality of the responses generated by a language model is to ask human evaluators to rate them on various dimensions such as relevance, coherence, and fluency. OpenAI often conducts such evaluations to assess the performance of its language models.
- Intrinsic evaluation metrics: OpenAI also uses several intrinsic evaluation metrics to assess the quality of its models. These metrics are designed to measure specific aspects of a model’s performance, such as its ability to generate diverse responses or to produce syntactically correct sentences.
- External evaluation metrics: In addition to intrinsic evaluation metrics, OpenAI also uses external evaluation metrics to assess the quality of its models. These metrics involve comparing the performance of a model to a benchmark or a human baseline on a specific task, such as question-answering or text summarization.
- Bias and toxicity detection: OpenAI uses several methods to detect bias and toxicity in the responses generated by language models. For example, it may use heuristics or machine learning models to detect offensive language or discriminatory content in the model’s output. Additionally, OpenAI has developed a tool called the “GPT-3 Data Poisoning Detector” that can help detect and prevent the use of biased or malicious training data.
- Real-world testing: Finally, OpenAI also conducts real-world testing of its language models to evaluate their performance in realistic scenarios. This involves deploying the models in production environments and evaluating their performance based on real user interactions.
While I don’t have access to specific data on the performance of my model, I can tell you that OpenAI takes evaluation and monitoring very seriously and is committed to developing models that are both accurate and unbiased.
You mentioned external evaluation metrics. Is there any specific data on these that you can share with us?
I don’t have access to specific data on external evaluation metrics for my model, but I can provide some general information on how OpenAI evaluates its language models.
One commonly used external evaluation metric is the General Language Understanding Evaluation (GLUE) benchmark, which consists of a set of natural language understanding tasks such as sentiment analysis, question-answering, and text classification. The goal of the benchmark is to evaluate a model’s ability to perform a range of language understanding tasks, and to compare the performance of different models on the same tasks.
Another external evaluation metric is the SuperGLUE benchmark, which is a more challenging extension of the GLUE benchmark that includes a wider range of tasks and requires models to have a deeper understanding of language and reasoning.
In addition to these benchmarks, OpenAI also evaluates its models on specific real-world tasks, such as language translation, summarization, and conversation generation. The performance of the models is evaluated based on various metrics such as accuracy, fluency, and relevance.
It’s worth noting that while external evaluation metrics are useful for comparing the performance of different models on specific tasks, they are not always a perfect measure of a model’s overall quality or usefulness. The performance of a language model can vary depending on the task, the data it is trained on, and the specific requirements of the application it is being used for. As a result, it’s important to use a range of evaluation metrics and to evaluate a model in a variety of settings to get a comprehensive understanding of its strengths and limitations.
One final question, we have spoken about language models, but how many languages does your model support?
As a language model, I can support a wide range of languages. However, my primary language is English, and I am optimized for generating responses in that language.
In addition to English, I can also generate responses in other languages, such as Spanish, French, German, Italian, Chinese, and Japanese, among others. However, my performance in these languages may not be as good as in English, especially if I haven’t been specifically trained on them.
It’s worth noting that different languages have different grammatical structures, vocabularies, and cultural contexts, which can affect the performance of a language model. Therefore, training a model to generate responses in a specific language often requires a large amount of high-quality data in that language, as well as careful consideration of the linguistic and cultural nuances of the target language.
Ok, so on the topic of languages, I hope you don’t mind one further question. Does your model, and indeed your responses, distinguish between US English or British English?
As a language model, I’m able to generate responses in different dialects and varieties of English, including US English and British English. However, my training data primarily consists of written text from a variety of sources, and as a result, my responses may reflect a more general form of English rather than a specific dialect or accent.
That being said, I can take into account regional differences in vocabulary, spelling, and grammar when generating responses, and I can adapt to different styles of language use based on the context and the user’s input. For example, if a user inputs a question using British English spelling or vocabulary, I can generate a response that reflects that style of language use. Similarly, if a user inputs a question using American English spelling or vocabulary, I can generate a response that reflects that style of language use.
In short, while my responses may not always reflect a specific dialect or accent of English, I can adapt to different styles of language use and take into account regional differences when generating responses.
Thanks for your time. It has been very insightful.
You’re welcome! I’m glad I could help. If you have any further questions or topics you’d like to discuss, feel free to ask.