LLM Mindset Myth #1: AI is an ANSWER MACHINE

We’ve grown accustomed to “search engine delivery” - we request information and expect to have it provided instantaneously. With the rise of digital assistants like Alexa and Siri, we verge on a Star Trek-ish assumption that we can simply call out “Computer!” then bark out any question and receive the immediate (correct, we assume) answer in an affectless robotic voice. Over time, a pitfall of such an assumptive approach arises: sapped of our capacity to engage critically, we lapse into unexamined, often baseless faith in the results. With AI, though, an approach like this represents a fundamental misunderstanding of how AI operates and what it is meant to do - AI is not intended to function in such a “transactional completion” model, there is no one-to-one ratio of input-to-output, of query to answer. AI, and in particular Large Language Model (LLM) AI, should be thought of as iterative, as almost Socratic in its pathway to good results - it nudges along toward the precise and applicable.

It is important to keep sight of the fact that AI on its own is not a source of information - it draws upon information. It is therefore limited in important ways: the biases and blindspots of the people selecting and inputting the data, the borders - of topic, of detail, etc. - on a given subset of knowledge (because no matter how large, it is always a subset.) As with any information source, the quality, comprehensiveness, and precision of the outputs provided by any AI system are the direct result of how complete and correct the data going into it is. To deploy a Large Language Model well requires a reorientation away from this Magic 8-Ball mindset in favor of a more reciprocal one. When provided with appropriate conceptual and rhetorical parameters, LLM AI is extraordinarily adept at helping us to parse problems and draft (not “declare”) solutions.

It is true that LLMs are mind boggling in their speed, and can draw upon vast troves of public (and, regrettably, private) information sources for material, but it lacks the inborn and sophisticated meaning-making so natural to humans. It is better to think of LLM AI as an intern with access (not actual comprehension in any functional sense) to all of human knowledge, but who is naive and lacks sound instincts. AI is not some opaque fog out of which emerges apt responses to our questions - it doesn’t know everything; it knows what we instruct it to know. An LLM is not a librarian, it’s an interface - or, more precisely, if the LLM is a librarian, it is rookie capable of a basic set of tasks, not the sophisticated mastery of the seasoned polymath: the LLM has no sense of what it does not know; an expert human librarian will know that a title is not in the collection, and can direct you to another institution that’s likely to have it - AI can’t do anything resembling this*.*

In a general sense, it is fitting to think of any Large Language Model not as some know-it-all colleague, as some of us seem to, but as as a machine. And like any machine - a drill press, a drawbridge, a cash register - it is well suited to performing the finite set of operations of its intended use. It’s perhaps better to conceive of an LLM as a QUESTION Machine, NOT an Answer Machine - the LLM excels at providing us with short-span bridges between bodies of knowledge, or in compiling an aggregated sense of what many fields or disciplines have to say about a subject. If your project is to design a tugboat, the LLM obviously cannot do that for you, but it can draw upon pertinent bodies of knowledge - fluid dynamics and engine requirements and maritime law and so forth - none of which will generate renderings of a tugboat. All of which can, however, provide a series of evaluative filters through which we may assess our evolving plan.

The LLM, in short, can be tremendously capable at the iterative interrogation of a project, where we provide it with applicable information. The LLM does not and cannot complete any complicated project for us, but is excellent at helping us ask about our own thinking about a complex objective: What conditions and factors might I have missed in devising this project? How might I refine and improve upon the current state of this project? It is this that separates the LLM from all previous machines - instead of firing rivets or spot welding, it can help us think. A Large Language Model, where we are clear on its intended use and capabilities, can introduce a kind of distributed cognition, a kind of multi-brain, into our processes - adding a kind of research assistant that does not generate novel ideas, but provides a useful, perhaps one day even indispensable, sounding board for tinkering with our own models and methods, for nudging our practices toward “best.”

LLM MINDSET MYTH #1A: “WHY CAN’T WE JUST RUN IT THROUGH CHATGPT?”

Well, you can, obviously. But will doing so provide you with anything resembling what you’re looking to attain? In terms of sharpening THE means of expressing ideas or refining your understanding of the problems you’re looking to solve? Not likely.

LLM platforms like ChatGPT are better understood as a series of switches than as a vast, sentient content generator. The LLM is NOT sentient, it does not GENERATE ANYTHING, not in the At the granular level, LLM systems function a lot more like auto-fill than human intelligence - it’s possible to have ChatGPT write you a sonnet, yes, in the sense that it will follow the rules of meter and rhyme and so forth, but it will have nothing like the depth of feeling or insight that a human writer can achieve. A person is able to compose thoughts and express them, in real time; the LLM guesses at what the next word or phrase is likeliest to be - with, it’s true, a speed that’s near incomprehensible, which no doubt contributes to our inflated sense about its capabilities. But this difference is crucially important to note - people naturally have a holistic approach to grasping and conveying concepts, the impressions we form and the language for making these known emerges more or less simultaneously; whereas even the most sophisticated LLM is inherently, unavoidably sequential and probabilistic.

The peril of this unmediated “run it through” approach is not that the results of doing so will be useless (though left to their own devices, the tendency of such platforms to produce gibberish is well documented,) it’s that they will be flat. If in the human brain, intelligence may be said to reside in complexities of structure, in the folds and wrinkles, unaided LLM’s “thinking” might fairly be called smooth-brained. It will tell us variations of what we already know - it will produce phrases adjacent to the expected. Or arrive at conclusions that are naive - not incorrect, exactly, but obvious, lacking nuance. The knowledge upon which LLMs will draw is vast but shallow, effectively the limitless inexpertise of the generalist. What LLM is exceptionally good at, however, is not in what we think of as cognition, but in providing us with cognitive assists - we should regard these systems as an enrichment tool, not as a generative one. Even for humans of the most arresting and singular intelligence, those transformative jolts of inspired, visionary thought are comparatively rare - but the prospect of them remains within reach. For ChatGPT and other LLM AI platforms, such leaps are not possible.

Say you are an architect hired by a client to build a home outside of Santa Fe. You will bring your knowledge to the project, certainly - all the years of information and insight you’ve gathered about materials and processes, esthetics and function. So far, LLM - provided access to facts about materials and building codes and architectural technique - will more or less be able to keep pace. What such platforms cannot do, however, is draw upon the flashes of inspiration that come to people, or the facility we have for interpreting the mind and desires of a client, or the intuitions that arise across the life of such a project that we heed or ignore. Consequently, if we submit a request to ChatGPT or any comparable LLM platform to “help design a house outside Santa Fe,” anything it tells us is likely to have a “received wisdom” feel, that might be broadly apt in principle, but is not germane or applicable to this project.

Reorienting ourselves to what LLMs actually ARE competent to provide such a project would be to rightly regard it as a component-level tool, an interlocutor adept at helping to navigate the dozens or hundreds of branches on the decision tree of conceiving of such a project. For instance, an LLM could harvest climate data for the exact GPS coordinates of the proposed site, and provide aspects to consider in the selection of building materials, or could match green building standards and water conservation practices to the proposed structure and landscaping. In other words, AI does not, cannot, design in the way that we can, in the sense of synthesizing information and inspiration to arrive at wholly novel conceptions of how to grapple with a particular set of problems. A Large Language Model, where it has been provided the right data sets, is very good at helping us bridge the gaps (leap synapses, find new relationships) between various bodies (points, areas) of abstract knowledge - in other words, it is a computational means of arriving at the unexpected, one where each step in attaining a “thought” bears some clear relational connection to each one before and after it. AI is not well suited to shaping an overall vision, but is highly useful in helping to render the many this-vs.-that distinctions that constitute the ingredients of vision.

Where a Large Language Model like ChatGPT can prove truly transformational, even revolutionary help to us, is where it serves not as our motorcycle, but as a sidecar - it does not generate power, it does not steer, nor decide on destination, but it can expand our capacity. We - as the drivers - must keep clear on our own cognitive capabilities and on the augmentations that the LLM component can provide. It is only when we do so, where we retain a solid “who’s good at what” grasp of the nature of the dynamic between us, that these platforms may realize their fullest potential as the fastest, smartest assistant we’ve ever had.