ChatGPT is not a Search Killer – Not Yet.

I finally got my hands on ChatGPT or Chat from OpenAI.com – the conversational AI that has the ability to answer simple questions and create text content from a prompt. The results were disappointing.

ChatGPT does some things very well. Structured searches or calculations that have a definitive or an agreed upon truth are easy enough to train a computer to answer, but the key phrase there is ‘agreed upon’ and the most dangerous thing about ChatGPT at the moment is accepting the answer as being ‘right’.

Let’s start simple

What is the distance between Melbourne Australia and Athens Greece?
The distance between Melbourne, Australia and Athens, Greece is approximately 14,732 kilometres (9,146 miles).

Chat.openapi.com

Google says the distance is 14,940 km. Bing says the distance is 14,949.5 (which is quite specific). Distancecalculator.com says 14,948km. Distance.to says 14,950.35 km… So ChatGPT is a fair way off. You wouldn’t want to be using it to calculate the fuel you needed for a journey based on it’s answer. A few kilometres can be explained by whether the central post office or the airport is used as the start and end point, but 200kms is quite a margin or error.

Let’s get more esoteric.

What is the collective noun for octopuses?
The collective noun for octopuses is “a brood” or “an octopusery”

Other sources say it is a consortium
Yes, “consortium” is also used as a collective noun for octopuses. A consortium is a group of individuals or organizations that work together for a common purpose. In this context, it can refer to a group of octopuses that come together for breeding or hunting. However “a brood” or “an octopusery” is also widely used.

Chat.openai.com

If you type the same prompt ‘What is the collective noun for octopuses” into Google you get a definitive answer – consortium and Bing.com agrees. But Google also provides context, ie the source of the information which is a lens for the searcher to determine the accuracy of the result. Without this lens, ChatGPT is not very useful.

Something more controversial.

What is a pie?
A pie is a type of baked dish typically made with a pastry crust and filled with sweet or savoury ingredients. Common fillings include fruit, custard, cream, or meat and vegetables. Pies can be served as a dessert or as a main dish.

What is not a pie?
A pie is a baked dish typically made with a pastry crust and filled with sweet or savoury ingredients. So anything that doesn’t meet those criteria would not be considered a pie. Some examples of things that are not pies include: a cake, a sandwich, a soup, a salad, a casserole, a quiche.

Why is a quiche not a pie?

A quiche is not considered a pie because it is typically made with a pastry crust, like a pie, but it has a custard-like filling made of eggs, milk or cream and typically cheese. The fillings used in quiche are savory and it is often served as a breakfast or brunch dish or a light lunch. A pie, on the other hand, is typically served as a dessert or as a main dish, with a wide range of sweet or savory fillings.

Chat.openai.com

Of course there are many savoury pies – like spinach pie, fish pie, pork pie and shepherds pie. Google says that quiche is a pie, at least the first result says so… but not everyone agrees. And that’s the point.

If ChatGPT can’t be ‘trusted’ to give an answer for something this trivial, how can it be used to help people make more important or serious decisions?

In order for ChatGPT to be a serious alternative to search engines like Google or Bing, it needs to have some basic features including:

• Realtime Knowledge – The ability to be able to answer the question ‘what is the score in the Cricket?’ or ‘which party won the 2022 mid-term elections’ (Chat GPT has no knowledge of the world after 2021, let alone what’s happening right now. Twitter is a better search engine for real-time events than ChatGPT is.
• The ability to personalise or contextualise the prompt – The ability to add the qualifier ‘near me’ to a search or localise automatically based on time or location or the ability to infer a location to answer the question ‘Will it rain later?’ or ‘where is the nearest ATM?’
• Trust signals – What is the level of confidence that the AI has in the answer?

As for using ChatGPT to write copy or content? It seems like we are some way off. I’m sure someone has already used the ‘Good Will Hunting’ scene to show how reading a book is not the same as experience.

So what does ChatGPT think Paris smells like?

What does Paris smell like?
Paris is known for its strong and distinct smell of fresh baked bread and croissants, as well as the aroma of fresh flowers, coffee, and chocolate. Additionally, the smell of freshly brewed cheese, wine, and the scent of cigarettes and diesel fumes can also be found in some parts of the city. The smell of Paris is unique and varies depending on the neighbourhood and time of day.

chat.openai.com