The Mechanical Turk, Part 1: Lying Liar Filled With Lies

At one end of the insanity range, we hear stuff like: “Inside a year, AI’s will go rogue and swipe your medical records and start selling your organs on the Dark Web. And funneling the proceeds to ISIS. hashtag-hellscape!!!”

At the other end of the lunacy spectrum, we hear stuff like: “By this time next week, AI will be performing life saving surgery unassisted. While doing kickass skateboard tricks and shredding on guitar. hashtag-rad!!!”

At this point, where AI is concerned, it’s become difficult even to adequately exaggerate the claims being made about it. And Large Language Models (LLMs) are by no stretch exempted from this frothing madness.

There is a period, early in the life of any technology - especially a technology with transformative potential that can be broadly applied - where just about anything we might have to say about it is wrong. The truth - as with most truths - is shaded with aspects of both extremes: LLMs have already proven to have great utility and to have transformed many aspects of living; they also represent a risk when misused.

Just about all the pronouncements and prognostications, often as not delivered by those attempting to monetize their own pronouncement-making, are wrong. The landscape of analysis seems crowded with as many utopians as doomsayers, and much of what they have to say is rooted in exaggerating either the promise or the perils of the technology. When really, as with just about every tool or platform, the peril or promise hinge upon the intent of those using it.

The Mechanical Turk was (claimed to be) a chess-playing automaton that toured Europe in the 1770s. The machine would take on and defeat any comers in a live game. And while the notion of buying a ticket to go see some variation on Zoltar (1988’s Big) play chess against some series of randos lands in the mind of the present-day entertainment seeker as extraordinarily lame and boring, in its day, the Turk was a sensation. This turban-wearing wonder, wrought of wood, wire, and metal, toured the Capitals of the Continent, where crowds were hungry, apparently, to watch a crudely painted (and frankly quite racist, by today’s standards) simulated fella from the Mysterious East move very slightly in a herky-jerky way. The Turk would face any opponent, and remained undefeated in his public contests.

It goes without saying, of course, that the entire thing was a fraud. An artful fraud, yes, an ingenious fraud, without question. But a fraud nonetheless.

Style note: though the Mechanical Turk was without question an object, because that object was crafted in the form of a person, and because that person-shaped thing was constructed in a way that it appeared to be performing tasks, for clarity/simplicity, we’ll be referring to the Turk as “he/him”, as we would a person, and not “it.” Cheers, eds.

And, when thinking about LLMs, it's this element of the Turk that’s most salient and instructive. I’ve come across a couple bloggity/newslettery posts that draw parallels between the Turk and LLMs where he serves to illustrate the “Black Box Problem.” The Black Box Problem describes the persistent reality that as an LLM engages in its blistering-fast predictive work to arrive at its best guess for the next right word/phrase/sentence, we are not able to know/see, given the vastness of its inputs, what source material it’s drawing from, or to reproduce the “chain of cognition” pathway that leads it from our request to the output it provides. The Black Box Problem is a hugely complex and fascinating element of the LLM puzzle, but the Mechanical Turk is NOT a fitting image for describing it. Yes, there were unseen operations taking place inside the cabinet of the Turk; yes, there was an (intended) opacity to the means by which he arrived at his results. While this interpretation of the Turk as relates to LLMs is comprehensible (his workings were not visible, his functions not readily understandable, etc.,) it is not a defensible one - the entire reason for the Turk coming into being was to deceive his audience for the gain of those running the scam.

The DEFINING characteristic of the Turk was DECEPTION.

For all the cleverness and engineering sophistication contained within him, the Turk’s raison d’etre (Hey. Whoa. Whoa, whoa, whoa. Let’s keep the French to a minimum, shall we? - eds.) was to instill in his spectators the false belief that he (vs. some unseen operator) was a chess master. Which he very much was not. He was not simply unskilled in chess, he was unskilled in every way imaginable. He couldn’t move on his own. His sightless eyes were painted on. He did not strategize, did not respond to an opponent’s moves, could not see the chessboard in front of him. The intelligence underlying the Turk, the only agency at play, was that of his human operators - the host/presenter who lied about the miraculous clockworks inside the Turk’s cabinet that made him “move” and allowed him to “interpret the board and state of play,” and the (necessarily short and slender) human chess master wedged inside the cabinet who actually carried out the gameplay.

The Turk’s cabinet was fitted with two sets of hatch doors with a false wall inside, so that the Host Charlatan could open them up, magician-style, to let audience members inspect the inside and establish that there was no human operator in there. But, like a magician, these hatches would reveal one compartment at a time, permitting the Concealed Gameplay Charlatan to contort himself out of view when each was opened. So it’s THIS component of the Turk’s construction and operation that is MOST salient when relating him to the Large Language Models of the present day. The demonstrable intent to deceive, the misrepresentation of the Turk’s (nonexistent) capabilities that renders it an instructive episode of history. It is a useful caution to us today, especially those among us who wish to maintain a healthy skepticism about LLMs.

Most of us are familiar with the Latin phrase caveat emptor, or “buyer beware.” Which is sound advice that applies to a wide range of interactions. It is advice, however, that’s applicable mainly to material goods and services, advice that reminds us to measure the item we’re provided against the promises made about it.

We posit that Lector cave, which (according to the near consensus of several no doubt AI-powered online English-to-Latin translators,) is “reader beware” is a more germane and helpful caution to keep in our minds.

Next up: The Mechanical Turk, Part 2: Lying Liar Filled With Promise