On November 30, 2022 OpenAI launched ChatGPT to the world. In a few days more than a million people were using the free and public web application. ChatGPT has taken the world by surprise with its revolutionary approach to AI and language. This innovation might seem unimaginable or difficult to wrap your head around, so to help garner a better understanding of this innovation we decided to write this blog post.
Background on ChatGPT
ChatGPT is created by OpenAI, which is an “AI research and deployment company” founded in 2015 as a joint initiative by Elon Musk, Sam Altman, Greg Brockman, Wojciech Zaremba, Ilya Sutskever, and John Schulman. Their focus is fully on expanding the capabilities of AI, with the mission to ensure that Artificial General Intelligence (AGI) benefits all of humanity. The founders of OpenAI believe that AI systems, which are generally smarter than humans, should in return benefit all of humanity. While at the same time making sure there is a healthy balance between innovation and AI running amok.
One of OpenAI’s latest innovations is ChatGPT (Generative Pre-trained Transformer). Without going into too many details of the technology, ChatGPT is an artificial intelligence chatbot that as a result of some input text, generates new texts. It is built on top of OpenAI’s earlier developed GPT models and fine-tuned by using both Supervised learning and a process called Reinforcement Learning from Human Feedback (RLHF). For both approaches human trainers were used to help improve the model’s performance, these trainers played both sides, as user and as AI.
These human trainers are exactly what the name implies – humans that help the model during its training phase. The trainers that play the user will give a sample text as an input and the trainers that play the AI will give a reasonable response. To help the trainers that play the AI with their response a model will give some suggestions to help them formulate their response. This is the first step in the model training cycle. The next step is to help the model rank its responses. This will help the model to get better at responding to some input text that is given to the model. To accomplish this, a series of responses are collected. These responses can be collected from two or more models where the trainers ask the chatbots questions, the responses are subsequently ranked by the Human trainers. Lastly, in the training process the model uses an optimization algorithm iteratively, this ensures that the good answers get rewarded and that the bad answers are punished.
Large Language Models
The technology behind ChatGPT is also known as Large Language Model (LLM). LLMs are deep learning algorithms that can perform all kinds of complex textual processes such as generating, completion, recognizing, summarizing texts, question answering. At Eraneos we have used these algorithms extensively during projects.
These models are trained on a huge amount of text as data which can come from books, articles, and web pages, with the objective to generate human-like language as an output. LLMs use an unsupervised learning method to recognize patterns and relationships within the languages, which in return helps the model in generating new text that is similar in style and within context of the input data. It is important to keep in mind that in general if a large language model is trained on data that predominantly uses scientific journals, then the model is better at generating text based on those documents. To get a good general model it is important to balance it out with data from various sources, this means it takes a lot more time to train a decent model.
Applications of ChatGPT
ChatGPT has numerous applications in different domains, and it offers endless opportunities. Microsoft, the biggest corporate investor in OpenAI, has already worked on incorporating ChatGPT in their core software products – Teams, Outlook, Bing and office products such as Word, PowerPoint and Excel. In March 2023, they unveiled Microsoft Copilot, which integrates the language model into all of the Microsoft products. In this section, we will discuss some other interesting applications.
1. Cloud Infrastructure
With the rise of cloud computing, it has become increasingly important for businesses and organizations to set up cloud infrastructures. However, this is not an easy task, as many people lack the technical knowledge and experience to do so. An application using ChatGPT could allow users to describe their cloud infrastructure needs at a high-level and interpret these needs, such that it can generate a step-by-step plan for a setup in AWS or Azure. The generated plan could include details about services to use, how to configure them and optimize them for performance and costs, while providing explanations for each step and helping users understand the reasoning behind the recommended choices.
Furthermore, using NLP the application could learn from user feedback and refine its recommendations over time, thereby becoming increasingly effective at generating customized, high-quality cloud infrastructure plans, tailored to the needs of each user. By removing some of the technical barriers and letting users define their needs in natural language, this application of ChatGPT has the potential to simplify the process of setting up cloud infrastructure. Consequently, the technology could help more businesses and organizations take advantage of cloud computing benefits.
2. Chatbots
The main feature that ChatGPT is known for is the conversational chatbot, which is able to respond to user queries in natural language. It can also be used to build custom chatbot systems. Example chatbot uses include:
- Personal assistance: appointment scheduling and setting reminders.
- Customer service: answering frequently asked questions, resolving common issues, and directing customers to appropriate resources.
- Translation: lowering barriers for international service desks.
- E-commerce: product recommendations, order tracking, feedback collection and improving customer engagement.
Given the way ChatGPT remembers (past) interactions and conversations with its users, the responses can be tailored to the users’ specific needs and preferences. This gives ChatGPT an edge over existing chatbots, given that it can maintain context, write multiple variants of answers, give examples and explains its reasoning.
3. Summarizing and explaining existing content
If you have a lengthy text or article, you can ask ChatGPT to summarize it for you, in which it will give the essence of the text. The summary can be used to assess whether you want to read the complete text or not. Interestingly, you can specify the length of this summary: should it be a few sentences or approximately 300 words? Your choice. This can significantly save valuable time spent on reading documents and research.
In addition to its capabilities in summarizing simpler, textual information, ChatGPT can also be utilized for more complex tasks. For instance, it can be integrated with APIs or plugins that can generate natural language explanations or summaries of data, which makes it easier for non-technical stakeholders to understand and act on the insights. It can also help with automating the process of generating reports on these data.
Furthermore, you can also ask ChatGPT to explain certain content like text and code. If a text is difficult to understand, it can be explained in a concise and simple way. This also works for explaining code in natural language text, which can clarify it and save a lot of time, in contrast to having to go through script code manually and understanding what every function does. For code review, users can now be involved to assess the code from a functional perspective.
4. Creating content
ChatGPT can be used to generate various types of content based on the vast amounts of text data it has been trained on. For example, given that you have an idea for something to write about (such as texts, articles, blogs, descriptions or social media posts), you can give the writing process a head start by feeding the idea to ChatGPT and using the result as inspiration. This way, it can be used as a brainstorming tool to inspire you with certain ideas.
Moreover, ChatGPT understands code and can generate functions in practically all programming languages (such as Python/Java scripts, SQL queries). ChatGPT can be used to generate functions that would normally take some time to implement yourself, by feeding a precise prompt to it. It can also write unit tests, clean, and refactor code for you. These two examples make clear that ChatGPT works well as an amplifier for generating content, thereby saving you a lot of time.
One may ask, why not just use Google? Though Google can take you a long way, filtering results that are not precisely what you are looking for can be a time-consuming process. Given that you specify precisely what you are looking for, ChatGPT can combine existing information and give a more tailor-made response. This saves you the time of having to manually combine the information on Google for your use case.
While ChatGPT can be a useful tool for content creation, it’s important to note that the results it generates may contain mistakes and should not be mindlessly copied as replacement for your own content. The results can be unreliable and may even be completely fabricated. In the next section, we will elaborate on this limitation as well as other constraints. Despite this limitation, the point here is that ChatGPT can significantly accelerate the content creation process when used collaboratively and co-creatively. Although you’ll need some time to verify the accuracy of its responses, using ChatGPT will ultimately save you a considerable amount of time by helping you propagate ideas into content and automate the repetitive parts of content creation.
Although ChatGPT has its limitations, it offers numerous opportunities that can also be integrated in more specific systems and custom applications. Ultimately, the biggest limitation seems to be the imagination and creativity of the user.
Limitations & Considerations
With all the attention and praise ChatGPT is rightfully receiving right now, it can be easy to forget that it is still just a tool. Though sometimes it might seem almost sentient we need to remember that it is not capable of conscious thought and creativity.
Hallucination
The first drawback that we will discuss comes from the way ChatGPT is designed. What it does in a nutshell is statistically predict the most likely sequence of words that result from some input. Given its huge amount of training data the results can be amazing. The downside however is that because it is just predicting likely combinations of words it tends to ‘hallucinate’. This means that it can create output which has no basis in reality. There are examples where ChatGPT confidently gives wrong information and defends completely fabricated texts with great conviction, this includes scholarly texts with made up references or software code that doesn’t work. While what it says might seem very convincing, there is no guarantee that it is telling the truth. The problem with hallucination is that it can be very hard to detect unless you are an expert on the subject matter. Though one upside of this is that we will always need people to verify and control the responses of these tools. It is very unlikely that ChatGPT will replace humans in the near future.
Detrimental Use
While ChatGPT has great potential to be used for beneficial purposes, it is important to recognize that there are also many ways it can be used for harmful purposes. Not all of which can be predicted, and thus prevented beforehand. Some examples of this are its ability to write flawless phishing emails or its ability to write malicious code. Because it is not actually ‘thinking’ it is unable to give a moral judgement on what it is being asked to do. It cannot judge whether its output would be considered inappropriate by its creators and users. There are guardrails in place to filter out harmful prompts and responses, but they have proved to be imperfect as of yet.
Training Data
Some other weaknesses of ChatGPT arise from the datasets it was trained on. The latest of which contain data gathered no later than 2021. It will have no way of knowing anything that happened after that point in time. Another problem arises because of the nature of these datasets. This data consists of 570 Gigabytes of 300 billion words sourced from books, research articles, Wikipedia and numerous other websites. As ChatGPT’s training data is derived from human-written texts, it inherits numerous flaws and biases that are inherent in those texts, which may include outdated views and opinions that do not align with contemporary society. This can result in the model generating responses that perpetuate sexism, racism, and other forms of bias.
Advice
When using ChatGPT we should always remember that though it can generate impressive responses it is just a tool and not an actual intelligent being. Also keep in mind that like with any technology, there is a potential for it to be used for nefarious purposes. It is up to humans to use the technology responsibly. Finally, when using the tool be on the lookout for biased or hallucinated responses as it is ultimately your responsibility how you use the information ChatGPT provides.
If you are curious to learn more or want to learn how data and AI can give you a competitive edge, reach out to our team.
