Beyond the brain – the bias of AI language models

ONE of the most trenchant criticisms of large language models (LLMs) like ChatGPT is that since they are trained on copious amounts of data found on the internet, making statistical correlations, they are mere probabilistic parrots of language with no real reasoning ability.

If correct, several problems arise. For example, since the information they are trained on is not curated, they are bound to regurgitate society's dominant biases. Google’s Gemini recently gave an apt demonstration of this problem, angering many.

When Washington Post columnist Megan McArdle, asked Gemini to write something condemning the Holocaust, it did so. But when she asked it to write something condemning the Holodomor or Mao's Great Leap Forward, it pointed out how complex history is then proceeded to contextualise what took place.

An example of the same issue has long been evident in the South African context on Google Translate, which was trained using the same machine learning techniques. If you ask it to translate “boeremeisie” to English, it tells you it means “peasant girl”. Not surprising when much that was historically written in English about Boers and Afrikaners painted them in an unflattering light.

A more fundamental critique is that generative models like GPT-4 aren't truly intelligent in the way humans are. Their outputs may seem meaningful and well-formed but this masks a lack of real comprehension or a grounding in reality. In place of humans' intentional communication, GPT-4 merely calculates the likelihood of sequences of words based on its extensive training data, navigating an immense “latent space” of potential sentences without grasping their significance.

Ted Underwood, who teaches at the School of Information Sciences and in the English Department at the University of Illinois, applies machine learning to literature. He agrees that in contrast to human-like intelligence, with its subjectivity, the ability to plan and to desire, LLMs are more like models of something collective — namely culture — and therefore replete with bias.

Intelligence comes in many forms, says Underwood, and some forms of intelligence we share with other animals, like the ability to plan ahead, which LLMs cannot yet do. But until now, language has been uniquely human. Perhaps language is even more distinctly human than intelligence. And perhaps intelligence on its own is not all it's cracked up to be.

François Chollet, Google deep learning pioneer and creator of the the Keras deep learning software library, agrees. He says we have a bias when we think about cognition, focusing on the brain, but it is not the only place where thinking happens — the environment in which we find ourselves is also vital.

Our abilities cannot be explained by just one disconnected brain learning something on its own in a single lifetime. As evidence, Chollet points out that our intelligence is greater than that of the great apes — but not that much more that it explains the difference in our capabilities. That difference is due to our culture, language and civilisation — the vast scaffolding that allows us to live elevated lives.

Culture and language are the result of thousands of years of accumulating knowledge, skills and mental models of the world, says Chollet. When people are born, they don’t have to reinvent the wheel. It follows that models of the world, including models of language and culture, can be important tools for thinking in their own right, even if they are not intelligent.

That doesn't mean LLMs aren’t dangerous. Not all aspects of human culture are beneficial, and culture extrapolated through technology can lead us to dark places. Underwood highlights this by referencing movements like QAnon, illustrating how humans are prone to creating alternate realities. When bolstered by the power of sophisticated deepfakes, these fabricated worlds could become overwhelmingly convincing.

***

Talking about alternate realities, Vernor Vinge, a visionary in science fiction, died three weeks ago, leaving a profound legacy. More than any other science fiction writer, Vinge grappled with the dilemmas artificial intelligence may present us with. He was the first to conceive “the singularity”, a pivotal event that would occur as a consequence of the development of superintelligent entities, resulting in unforeseeable changes to human civilisation.

The first science fiction Vinge sold, to the British literary magazine New Worlds in 1965, was a short story, Apartness, about a post-apocalyptic earth where the protagonist finds an isolated group of people in Antarctica.

First confusing them with “natives” of the frozen continent, the protagonist describes them as hairy and of indeterminate race but primitive in every physical sense, with inferior tools. But then they draw their knives and attack the protagonist’s group, shouting something that sounds like: “In di nam Niu Transfals yuli sterf”, hinting at who they might be.

♦ VWB ♦

BE PART OF THE CONVERSATION: Go to the bottom of this page to share your opinion. We look forward to hearing from you.