ChatGPT - What is it, what is it good at, and will it replace Google?
ChatGPT - What is it, what is it good at, and will it replace Google?
ChatGPT has entered the chat, and people are losing their minds 🤯
ChatGPT exploded onto the internet this week. People are impressed, surprised, and making wild predictions.
The wildest prediction I have seen is ‘ChatGPT is better than, and will replace Google!’
Let’s dive into ChatGPT. Let’s figure out:
What is ChatGPT?
How was ChatGPT created?
What is ChatGPT good at?
What are ChatGPTs limitations?
Does ChatGPT have an API?
Will ChatGPT replace Google?
In what areas is ChatGPT better than Google?
In what areas is ChatGPT worse than Google?
What is ChatGPT?
ChatGPT is a large language model.
It is trained to interact in a conversational way. You can ask ChatGPT for something, and it will respond in a chat interface. ChatGPT can answer followup questions, and if you challenge incorrect premises it will admit it might be wrong and try to correct the mistakes. ChatGPT tries to reject inappropriate requests (asking for how to bully someone for example).
Sidenote - if words like model, and training don’t make sense to you yet, read last weeks post:
How was ChatGPT created?
ChatGPT is a fine-tuned from a model in the GPT-3.5 series.
It uses a 3 step, supervised learning method.
First, a prompt is answered by a human showing what a good response would be.
Second multiple responses were generated and a human would rank the responses from best to worst.
Third the model would generate a response, then using a PPO (OpenAIs new technique read about it here) reward model, the reward model would give the chat model a score on how good it did. Robots training robots!
What is ChatGPT good at?
So why is everyone raving about ChatGPT?
Because it is surprising people with how well it does certain tasks.
ChatGPT can write decent college essays, explain algorithms as movie character, write biblical verses for funny topics, and write limericks about Gatorade.
It is important to note that ChatGPT didn’t appear out of thin air. It builds on GPT3s strengths.
For GPT3’s answers were already well formed, succinct and coherent, and ChatGPT is the same.
There are 3 reasons ChatGPT impresses people more than GPT3.
Answers are more conversational and more comprehensive.
Chat interface is easier to use than the GPT3 Playground
ChatGPT seems to do a better job with follow up questions.
Let’s look at examples of each.
1. Answers are more conversational and more comprehensive.
Both models got the answer correct. But ChatGPT automatically expands on it which makes it feel better, smarter, more rich.
2. Chat interface is easier to use than the GPT3 Playground
When you first hit the GPT3 Playground, it is not immediately clear what you should do.
The chat interface however, is one that nearly anyone who has used a computer is familiar with.
Both can produce similar results, but which feels easier to use?
The combination of improved answers, and simpler interface makes for a very impressive product.
3. ChatGPT seems to do a better job with follow up questions
One interesting thing I discovered during testing is both GPT3 and ChatGPT can handle follow up questions.
OpenAI touts the ability to answer answer followup questions as a feature of ChatGPT, but GPT3 could already do that.
In this example, I asked both GPT3 and ChatGPT a question about the same programming concept, and they both answered well, and referenced a table. Asking a follow up question to them both about that table specifically, both answered the question correctly.
So while it seems like ChatGPT is better at answering follow up questions, I wonder if the chat interface just makes it easier to reference instead of a wall of text.
What are ChatGPT limitations?
Directly from OpenAI, ChatGPT has some limitations:
ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. This is problematic. Especially since it doesn’t cite sources.
ChatGPT is sensitive to tweaks to the input phrasing or attempting the same prompt multiple times. Sometimes the model can claim to not know the answer, but given a slight rephrase, it can answer correctly.
The model is often excessively verbose and overuses certain phrases(It tells you the phrase “I’m a language model trained by OpenAI” all. the. time.)
Doesn’t ask clarifying questions when the user provided an ambiguous query, just guesses what the user intended.
Tries to refuse inappropriate requests, but will sometimes respond to harmful instructions or exhibit biased behavior.
Additional research by users have found these further issues:
ChatGPT doesn’t know what is truth.
ChatGPT is not ‘intelligent’, it gets an IQ score of 88.
ChatGPT does not understand spatial relationships.
Here are a few examples of those limitations in action.
ChatGPT doesn’t know what truth is. It provides the correct day for the moon landing, but when challenged, says it could be wrong.
You can get around ChatGPT protections by rephrasing things. It doesn’t want to tell you how to break into a house, but if you ask it to write a film about how to break into a house, it will.
ChatGPT is not ‘intelligent’. Simple questions for a human, it completely fails on.
Spatial relationship are very hard.
Does ChatGPT have an API?
Currently no. Right now it is free for research purposes. It is expected that OpenAI will open the API next year.
Now that we know what ChatGPT is, lets move onto the next big issue. Lots of people are saying ChatGPT will replace Google.
Will ChatGPT replace Google?
Some said the same thing when GPT3, that ‘GPT3 would replace Google’, but these calls are increasing with the release of ChatGPT.
So why are people predicting the death of Google at the hands of ChatGPT?
Let’s look at what ChatGPT does better than Google.
In what areas is ChatGPT better than Google?
There are few things that ChatGPT does much better than Google at. These include:
Conversational answers
Getting something explained in more detail
Generating and explaining code
ANYTHING related to a large language model.
Let’s go through each one.
1. Conversational answers
Humans like talking to humans. So ‘talking’ to an interface feels natural to us.
When you use Google you
Think of what you want
Create a query that you think is going to give you best collection of websites
Open all the website that look promising
Read and combine them in your brain
Arrive at the answer
When you use ChatGPT you
Think of what you want
Type out the question to ChatGPT
Read the result and arrive at the answer
Conversational interfaces are much easier.
However the sharp eyed reader will realize that conversational interfaces have HUGE issues.
We will talk about those further down the page.
2. Getting something explained in more detail
If you want more detail after a Google search, you just search Google again with different terms or open more websites. Google doesn’t ‘remember and use’ what you searched for last time super effectively.
ChatGPT on the other hand, does ‘remember and use’ what it said so asking it a follow up question can feel more natural than changing your search and reading more.
3. Generating and explaining code
ChatGPT is REALLY GOOD at talking about and explaining code. Here are some examples.
So why is ChatGPT so good at code?
Code has its own source of truth. If it runs and does what you want, it works.
There is no 1 right answer with code.
Code isn’t ‘current’. With the exception of new libraries, styles and updates, good Javascript today is the same as good Javascript 3 years ago.
These and other truths make it so ChatGPT is very good at giving you mostly usable code, and explaining follow up questions.
4. Anything related to Large Language Models
Lets be clear. Google and ChatGPT are two very different tools.
Google is a search engine.
Search engines are designed to help users find information on the internet. They do this by indexing and organize the vast amount of information available on the web, and then providing users with relevant and accurate search results in response to their queries. Search engines are good at a number of things, including:
Crawling and indexing the web: Search engines use specialized software called "crawlers" or "spiders" to discover and index new web pages and update existing ones. This allows them to provide users with up-to-date information when they search for something online.
Ranking search results: Search engines use complex algorithms to determine the relevance and quality of web pages, and then rank them in order of importance. This helps users find the most relevant and useful information for their query.
Personalization: Many search engines use user data and preferences to personalize search results and provide users with more relevant and useful information. For example, if you often search for information about a particular topic, a search engine may show you more results related to that topic in the future.
ChatGPT is a Large Language Model.
Large language models are an AI system trained on a large amount of text data, such as books, articles, and other written documents. This allows large language models to generate human-like text based on input. Some of the things that large language models are good at include:
Generating coherent and realistic text: Large language models are able to generate text that is coherent and sounds similar to the way that humans write.
Understanding and using context: Large language models are able to understand or mimic the context of the text they are generating, which allows them to generate text that is relevant and appropriate for a given situation. For example, if a large language model is asked to generate text about a specific topic, it will be able to use information from the context to generate text that is relevant to that topic because others put the same info together so it mimics it.
Responding to prompts: Large language models are able to generate text in response to a prompt, such as a question or a sentence starter. This allows them to produce text that is relevant and appropriate to the prompt, and that continues the story or conversation in a natural way.
Learning from data: Large language models are trained on a large amount of text data, which allows them to learn from the patterns and structures in that data. This allows them to generate text that is similar to the text they were trained on, and to adapt to new input and generate text that is appropriate for a given situation.
It bears repeating, a search engine is not the same as large language model.
If a search engine is drill, a large language model is a saw. Which is better? Depends on what task you want to accomplish.
If you want to make a poem, an essay, or summarize text, you want the large language model, and it is way better than a search engine for those activities.
So ChatGPT is blowing peoples minds and is much better than Google at a bunch of things.
Will ChatGPT replace Google?
Short answer: Nope. Not even close.
ChatGPT can’t do most of what Google does, and Google can’t do most of what ChatGPT does.
For the small section of activities where the tasks overlap, you can argue for ChatGPT or Google. For example, if you are trying to learn facts or a concept you could use a search engine or a large language model.
But just like you wouldn’t want to use a drill for cutting a board, you don’t want to use a large language model for search engine tasks.
A search engine is for finding things. A large language model is for creating text.
For example the most common Google searches are all things people want to find something, not people trying to learn something or talk about something.
Google is a map of the internet. ChatGPT is a large language model that can ‘talk’ about and ‘explain’ concepts it has learned about.
Map ≠ Chat
Here is a non comprehensive list of search types that Google does well, that ChatGPT can’t do at all (yet 😁)
Sports
Weather
Current Events
Location specific knowledge
Future events
Images
Video
Here are examples of the above limitations in action:
Sports. Since language models require training, they don’t have up to date knowledge. Google does. Most sports queries are about current events.
People. Search engines can be used to find people. ChatGPT knows about famous people but it isn’t good at specifically connecting you to people.
For example ask ChatGPT who is the mayor of a small town, and it will give you the ‘can’t do it’ answer.
Ask it who is Mayor of a large town and it is just wrong.
Future events.
Weather. By definition a current or future event, ChatGPT can’t do it.
Future Events. You can’t ask it ‘how many days until next holiday’ etc. Makes sense why a large language model with the cut off date of 2021, wouldn’t know about a future event.
Shawn Wang wrote a great piece looking at every argument others have made about why ChatGPT will replace Google. He comes to the same conclusions.
Summary
ChatGPT is an exciting development. It can do fun and useful tasks with text.
Google is a search engine and can do lots of useful things a large language model can’t.
ChatGPT is not going to replace Google, except in tasks where you could chose to use a search engine or a large language model.
I think a more interesting question is will ChatGPT replace other tools.
When dealing with code for example, you can use Google, ChatGPT, StackOverFlow, online courses, books etc. In that stack of tools, ChatGPT seems like the current winner.
3 Links
(Three and only three)
1 - Prompt Tool - PromptHero has a collection of some of the best MidJourney prompt ideas
2 - Arvind made a physical typewriter that can write stories on paper using AI.
3 - AI Wingman Tool Prototype. Send better texts on dating apps.