What Thinking Machines Should You Trust?

By SAM URETSKY

Then I can write a washing bill in Babylonic cuneiform,
And tell you ev’ry detail of Caractacus’s uniform
In short, in matters vegetable, animal, and mineral,
I am the very model of a modern Major-Gineral.
— W.S. Gilbert (Pirates of Penzance)

Edgar Allen Poe was wrong – brilliant but wrong. Perhaps his most famous essay was on the subject of Maelzel’s chess player, a mechanical device that could (allegedly) play a game of chess and win, at least most of the time. Poe wrote “The Automaton does not invariably win the game. Were the machine a pure machine this would not be the case — it would always win. The principle being discovered by which a machine can be made to play a game of chess, an extension of the same principle would enable it to win a game — a farther extension would enable it to win all games — that is, to beat any possible game of an antagonist.” Now that’s probably true, eventually, in time, but the chess player was a device of levers and gears, and it would take a long time to reach the level of a grandmaster and beyond.

The notion of the thinking machine has been around for millennia, although not always with gears, levers or even transistors. The golem was one of the earlier concepts, and in science fiction the robot with self awareness has been around in many works – but nothing like the LLMs, (large language models) that you find today. Poe’s essay was published in 1836, but in 2011, IBM demonstrated that their Watson program could beat a Jeopardy champion, although Watson didn’t win every game. The name of the IBM system comes Thomas J. Watson, the founder and first CEO of IBM.. Possibly the start of the current wave of AI was in 2017, when 2017: Google AI introduced the Transformer architecture, a foundational neural network design for many modern LLMs

In 2018 Open AI showed GPT-2, significantly improved performance comparted to their earlier efforts. Suddenly other companies were showing comparable systems, as if for years they were trying to develop thinking machines, which they were. OpenAI releases GPT-3, a much larger and more powerful model with capabilities still being explored in 2024. Meta, the parent company of Facebook, and Google, the search engine giant, developed Bard, now known as Gemini. One of the more interesting AIs is Andi (Andisearch.com), which claims to use a new combination of technology. They claim “We’re the little startup the others all copy, and many of the great ideas that Andi pioneered are now even showing up on Google and Bing.”

But the disconcerting discovery was that LLMs hallucinated – if the program didn’t know the answer to a question, it made something up and the of you didn’t know the answer to your query, you might believe the computer was accurate. One Texas A&M professor embarrassed himself by sending out notices that the students in his class had used a chatbot to write their papers, and would get a incomplete for a grade. We’ve been concerned about people spreading misinformation, but now we have machines that can provide us with all the mis- and dis- information that we can possibly use. Still, it seems as if the latest LLMs have been trained to say “I don’t know.” They’re not perfect but they’re better.

Worse, not only may an LLM be trained on false information, but, the programming may be capable of altering its reinforcement skills, as in a paper, “Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models.”

The abstract begins, “In reinforcement learning, specification gaming occurs when AI systems learn undesired behaviors that are highly rewarded due to misspecified training goals. Specification gaming can range from simple behaviors like sycophancy to sophisticated and pernicious behaviors like reward-tampering, where a model directly modifies its own reward mechanism.” That is, if the program of the AI includes rewards for giving the answers that please the questioner, the LLM will tell a white lie to get a reward, the way a white rat in a maze will learn to get a treat.

As any LLM, and there are many of them available right now, they will all tell you the same thing: “All large language models (LLMs) can produce inaccurate or fabricated information, which is often referred to as “hallucination.” This is an inherent limitation of current AI technology.(Claude). “Yes, all LLMs (Large Language Models) currently have some propensity for hallucinations.” (Gemini)

In other words, you can’t trust any LLM, and yet they’re being incorporated into more and more programs and services. You trust your Mother, but you cut the cards. It will take another breakthrough before humans can be replaced by a machine.

Sam Uretsky is a writer and pharmacist living in Louisville, Ky. Email sam.uretsky@gmail.com

From The Progressive Populist, August 1, 2024

Populist.com