Cloud Hosting

In todays A.I. newsletter, the third of a five-part series, I discuss some of the ways chatbots can go awry.

A few hours after yesterdays newsletter went out, a group of artificial intelligence experts and tech leaders including Elon Musk urged A.I. labs to pause work on their most advanced systems, warning that they present profound risks to society and humanity.

The group called for a six-month pause on systems more powerful than GPT-4, introduced this month by OpenAI, which Mr. Musk co-founded. A pause would provide time to implement shared safety protocols, the group said in an open letter. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.

Many experts disagree about the severity of the risks cited in the letter, and well explore some of them later this week. But a number of A.I. mishaps have already surfaced. Ill spend todays newsletter explaining how they happen.

In early February, Google unveiled a new chatbot, Bard, which answered questions about the James Webb Space Telescope. There was only one problem: One of the bots claims that the telescope had captured the very first pictures of a planet outside our solar system was completely untrue.

Bots like Bard and OpenAIs ChatGPT deliver information with unnerving dexterity. But they also spout plausible falsehoods, or do things that are seriously creepy, such as insist they are in love with New York Times journalists.

How is that possible?

In the past, tech companies carefully defined how software was supposed to behave, one line of code at a time. Now, theyre designing chatbots and other technologies that learn skills on their own, by pinpointing statistical patterns in enormous amounts of information.

A New Generation of Chatbots

A brave new world. A new crop of chatbotspowered by artificial intelligence has ignited a scramble to determine whether the technology could upend the economics of the internet, turning todays powerhouses into has-beens and creating the industrys next giants. Here are the bots to know:

ChatGPT. ChatGPT, the artificial intelligence language model from a research lab, OpenAI, has been making headlines since November for its ability to respond to complex questions, write poetry, generate code, plan vacationsand translate languages. GPT-4, the latest version introduced in mid-March, can even respond to images(and ace the Uniform Bar Exam).

Bing. Two months after ChatGPTs debut, Microsoft, OpenAIs primary investor and partner, added a similar chatbot, capable of having open-ended text conversations on virtually any topic, to its Bing internet search engine. But it was the bots occasionally inaccurate, misleading and weird responsesthat drew much of the attention after its release.

Ernie. The search giant Baidu unveiled Chinas first major rival to ChatGPT in March. The debut of Ernie, short for Enhanced Representation through Knowledge Integration, turned out to be a flopafter a promised live demonstration of the bot was revealed to have been recorded.

Much of this data comes from sites like Wikipedia and Reddit. The internet is teeming with useful information, from historical facts to medical advice. But its also packed with untruths, hate speech and other garbage. Chatbots absorb it all, including explicit and implicit bias from the text they absorb.

And because of the surprising way they mix and match what theyve learned to generate entirely new text, they often create convincing language that is flat-out wrong, or does not exist in their training data. A.I. researchers call this tendency to make stuff up a hallucination, which can include irrelevant, nonsensical, or factually incorrect answers.

Were already seeing real-world consequences of A.I. hallucination. Stack Overflow, a question-and-answer site for programmers, temporarily barred users from submitting answers generated with ChatGPT, because the chatbot made it far too easy to submit plausible but incorrect responses.

These systems live in a world of language, said Melanie Mitchell, an A.I. researcher at the Santa Fe Institute. That world gives them some clues about what is true and what is not true, but the language they learn from is not grounded in reality. They do not necessarily know if what they are generating is true or false.

(When we asked Bing for examples of chatbots hallucinating, it actually hallucinated the answer.)

Think of the chatbots as jazz musicians. They can digest huge amounts of information like, say, every song that has ever been written and then riff on the results. They have the ability to stitch together ideas in surprising and creative ways. But they also play wrong notes with absolute confidence.

Sometimes the wild card isnt the software. Its the humans.

We are prone to seeing patterns that arent really there, and assuming humanlike traits and emotions in nonhuman entities. This is known as anthropomorphism. When a dog makes eye contact with us, we tend to assume its smarter than it really is. Thats just how our minds work.

And when a computer starts putting words together like we do, we get the mistaken impression that it can reason, understand and express emotions. We can also behave in unpredictable ways. (Last year, Google placed an engineer on paid leave after dismissing his claim that its A.I. was sentient. He was later fired.)

The longer the conversation runs, the more influence you have on what a large language model is saying. Kevins infamous conversation with Bing is a particularly good example. After a while, a chatbot can begin to reflect your thoughts and aims, according to researchers like the A.I. pioneer Terry Sejnowski. If you prompt it to get creepy, it gets creepy.

He compared the technology to the Mirror of Erised, a mystical artifact in the Harry Potter novels and movies. It provides whatever you are looking for whatever you want or expect or desire, Dr. Sejnowski said. Because the human and the L.L.M.s are both mirroring each other, over time they will tend toward a common conceptual state.

Companies like Google, Microsoft and OpenAI are working to solve these problems.

OpenAI worked to refine the chatbot using feedback from human testers. Using a technique called reinforcement learning, the system gained a better understanding of what it should and shouldnt do.

Microsoft, for its part, has limited the length of conversations with its Bing chatbot. It is also patching vulnerabilities that intrepid users have identified. But fixing every single hiccup is difficult, if not impossible.

So, yes, if youre clever, you can probably coax these systems into doing stuff thats offensive or creepy. Bad actors can too: The worry among many experts is that these bots will allow internet scammers, unscrupulous marketers and hostile nation states to spread disinformation and cause other types of trouble.

As you use these chatbots, stay skeptical. Take a look at them for what they really are.

They are not sentient or conscious. They are intelligent in some ways, but dumb in others. Remember that they can get stuff wrong. Remember that they can make stuff up.

But on the bright side, there are so many other things that these systems are very good for. Kevin will have more on that tomorrow.

Ask ChatGPT or Bing to explain something that you already know a lot about. Are the answers accurate?

If you get interesting responses, right or wrong, you can share them in the comments.

Question 1 of 3

Start the quiz by choosing your answer.

Hallucination: A well-known phenomenon in large language models, in which the system provides an answer that is factually incorrect, irrelevant or nonsensical, because of limitations in its training data and architecture.

Bias: A type of error that can occur in a large language model if its output is skewed by the models training data. For example, a model may associate specific traits or professions with a certain race or gender, leading to inaccurate predictions and offensive responses.

Anthropomorphism: The tendency for people to attribute human-like qualities or characteristics to an A.I. chatbot. For example, you may assume it is kind or cruel based on its answers, even though it is not capable of having emotions, or you may believe the A.I. is sentient because it is very good at mimicking human language.

Click here for more glossary terms.

Read this article:
What Makes Chatbots Hallucinate or Say the Wrong Thing? - The New York Times