From Traditional AI to Language Models
Think about the last time you booked a trip. You checked a few flight options, compared two hotels, worked out whether the timing left room for a meeting, and gave up on one idea because it was too…
Think about the last time you booked a trip. You checked a few flight options, compared two hotels, worked out whether the timing left room for a meeting, and gave up on one idea because it was too expensive. Ordinary work. Nothing about it feels like a computing problem.
Hold onto that trip. We are going to come back to it in almost every chapter of this book, and by the end it will have turned into a small team of AI agents working on your behalf, with you approving the parts that cost money. But it starts here, with something much simpler: what kind of software could help with a job like that at all?
For most of computing history, the answer was none of it, really. Software was good at the parts of the trip that were already decided and bad at everything else. What changed is the subject of this chapter.
Four ways to build intelligent software
People often talk about AI as one thing that has been getting steadily better. It is more useful to see four distinct approaches, each of which still exists, and each of which is still the right choice for some problems. Understanding the differences will save you from the most common mistake in this field: reaching for the newest technology when an older one would do the job better.
Rules written by hand
The oldest approach, and still the most common in production software. A person works out the logic and writes it down: if the booking is within twenty-four hours, charge the late fee. The computer follows instructions exactly.
This has no learning in it at all. The system handles what it was told to handle and nothing else. ELIZA, written in 1966 and often described as the first conversational program, worked this way: it matched patterns in what you typed and replied from a script.
Rule-based systems have not gone away, and they should not. They remain the right answer whenever the logic is fully known, the inputs are predictable, and you need the same output every single time. A tax calculation does not want a language model. Neither does a payroll run.
Classical machine learning
Instead of writing the rules, you show the system examples and let it work out the pattern. Give it ten thousand past transactions labelled fraud or not fraud and it learns which combinations of features tend to go together.
This is where a great deal of working AI still lives: credit scoring, demand forecasting, recommendation, spam filtering. It is often the cheapest thing that works, it runs fast, and when it is built on a modest number of clear features you can usually explain its decisions. Some of these methods need labelled examples; others, such as clustering, find structure in data with no labels at all.
Deep learning
The same idea — learn from data — but using networks with many layers stacked on top of one another. Each layer builds on the representation the previous one produced, which lets the system handle messy input that classical methods struggle with: photographs, recorded speech, handwriting.
Deep learning is what made image recognition and speech transcription work well enough to ship. AlphaGo, which beat a world champion at Go in 2016, was built this way. The cost is appetite: these systems generally want far more data and far more computing power than classical machine learning, and they are much harder to explain.
Foundation models
The newest of the four, and the one this book is about. A foundation model is trained on an enormous and varied body of material — in the case of a language model, text — and can then perform many different tasks without being rebuilt for each one. The same model summarises a report, drafts an email, answers a question about Portuguese history and explains a piece of code.
A large language model is a foundation model for text. ChatGPT, released publicly in 2022, is what put one in front of a very large audience for the first time. Claude, Gemini and Llama are others you will have heard of.
Table 1-1: Four approaches, all still in use
| Approach | Where it is still the right choice |
|---|---|
| Rules written by hand | The logic is fully known and the answer must be identical every time |
| Classical machine learning | Clear features, plenty of examples, and you want speed and low cost |
| Deep learning | Messy input such as images, audio or handwriting |
| Foundation models | Language, open-ended requests, and many different tasks from one system |
Why language models changed things
Go back to the trip. A task-specific system could tell you which of two flights was cheaper. It could not read your message saying I need to be in Barcelona by Tuesday lunchtime but I would rather not fly at dawn and work out what you meant.
That is the shift. A language model takes an instruction written the way a person would write it, and does something sensible with it. And because it is general rather than task-specific, the same model can go on to handle the next request too, without anyone building a new system.
Figure 1 shows the difference in shape. On the left, one general model handling several kinds of language task. On the right, a narrow model doing one job.
What gives them their reach
- They learn from an enormous amount of material. Rather than following logic somebody wrote down, they pick up patterns, follow context and produce responses that fit the situation.
- One model serves many purposes. The same system powers assistants, writing tools, support desks and research aids without being rebuilt for each.
- They keep improving. Each generation has been more capable and more efficient with computing power than the one before.
- They are widely available. Organisations use them to handle routine work and inform decisions, and open projects such as Hugging Face have put the same technology within reach of individual developers.
Table 1-2: Language models at work
| Field | Typical use |
|---|---|
| Healthcare | Drafting clinical documentation for a clinician to review |
| Finance | Summarising filings, flagging anomalies, customer support |
| Education | Tutoring support and personalised explanation |
| Marketing | Copy, social posts, content recommendation |
| Legal | Contract review and tracking regulatory change |
Task-specific systems and general models, side by side
A question that comes up constantly: how is this different from the AI that came before?
The honest answer is that the difference is breadth, not intelligence. A task-specific system is built for one job and does it well. A foundation model is built for many, and covers a far wider range. Neither is simply better: a purpose-built system may be more predictable, more efficient or more precise on the narrow task it was made for, while a general model handles requests nobody anticipated.
Table 1-3: Task-specific systems compared with foundation models
| Task-specific system | Foundation model | |
|---|---|---|
| Built for | One job, defined in advance | Many jobs, decided at the moment of use |
| How it is made | Hand-written rules, or training on data prepared for that job | Trained on very large amounts of general material |
| Input | Usually structured: fields, forms, numbers | Ordinary language, and often images too |
| Adapting it | Rebuild or retrain | Change the instruction |
The same problem, two ways
A task-specific system. A bank flags any transaction over a set amount. The logic is fixed, so it raises a false alarm the first time a customer makes a large but entirely legitimate purchase, and it has nothing to say about why.
A language model. The same transaction is read in context. The system can weigh patterns that look like fraud against ordinary variation in how people spend, and it can produce an explanation for why a particular transaction was flagged. That explanation is what makes human review practical at scale — somebody has to look at the alerts, and they need to know what they are looking at.
Notice that the second version does not replace the first. Most real fraud systems run both: cheap deterministic checks catch the clear cases, and the expensive general model handles the rest.
Key takeaways
- Four approaches coexist: hand-written rules, classical machine learning, deep learning, and foundation models. Each is still the best answer to some problems.
- Only some of these need labelled data. Rules need none, and some machine learning finds structure without labels.
- A foundation model is general: one system, many tasks, no rebuild between them.
- A large language model is a foundation model for text.
- The advantage over task-specific systems is breadth and the ability to take instructions in ordinary language.
- Combining approaches usually beats choosing one.
All of which raises the obvious question. A system that takes an instruction in plain English and produces a sensible paragraph in reply is doing something genuinely strange. What is actually happening inside it?