Keeping Up With the Machine

The walk that opens this book happened shortly after GPT-4 appeared. I had used the generation before it, and the difference felt extraordinary — not a better version of the same tool but a different kind of tool. A research paper turned into a working program while I was out with the dog. A year earlier, nothing I had used could have done that.

That is my experience, not a measurement. Later generations have brought further large improvements, though in my experience they have been uneven: some capabilities leap, others creep, and a few seem to stall for a while. At times the progress has felt exponential. I am not claiming it is, or that it will continue at any particular rate. Chapter 08 set out my forecast and what would change my mind about it. The point here is simpler and, I think, beyond argument: the tool at the centre of machine coding changes fast enough that working assumptions you formed a year ago may already be wrong.

That makes keeping up part of the practice. Not following every announcement — nobody can, and trying is a good way to stop creating anything — but having a way to notice which changes matter to your work, find out whether they are real, and adopt them when they help. And because this is a book about directing machines, the most useful way to do that is to have a machine help you.

A tool that changes under your hands

Most of what programmers learn ages slowly. Years spent mastering a language or a framework keep paying off for years. A machine-coding workflow is different, because it rests on a moving combination of models, agents, tools, prices and environments, and each part moves on its own schedule.

The consequences are concrete. A technique you needed last year may no longer be necessary. A workaround you built for a weaker model — splitting every task into tiny pieces, repeating instructions it used to forget — can become a drag on a stronger one; Chapter 07 described exactly this, instructions that once helped and now get in the way. A model you relied on may become more expensive, change its behaviour, or be retired. An inexpensive model may cross the threshold Chapter 12 described and become good enough for work you used to send to an expensive one. A new agent runtime may make several of your manual steps unnecessary overnight.

It also works in the other direction. Not every release is an improvement for you, and switching has costs: re-learning a tool’s habits, re-testing your workflow, losing a setup that worked. So the skill is not “always use the newest”. Learning to work with AI includes learning when to change your working method — and when not to. That skill comes from repeated small experiments, not from loyalty to any one tool and not from the excitement of any one announcement.

Which machines to keep an eye on

Chapter 03 argued for allocating different intelligence to different work: strong reasoning for design and review, economical models for bounded implementation, and so on. That allocation tells you which machines are worth watching. You do not need to follow the whole field. You need to follow the slots in your own workflow.

For many people that means a capable primary model for difficult work, a cheaper one for routine tasks, perhaps a local or open-weight model for privacy, offline work or cost control, and a second, independent model for comparison and review. That is an example, not a prescription. Your slots depend on your work.

In my own experience, the gap between the best paid models and the free tiers has been large for difficult coding work. I report that as experience, not as a ratio, and it can change in a single release. Several distinctions matter more than the headline. Model capability is not the same as what a subscription entitles you to; usage limits, tool access and reliability differ between plans. A chat subscription usually does not give you API access for an application you build, and an application that calls a model will usually be charged separately. And some excellent models are free, open-weight or run on your own hardware.

The only reliable way to choose is to try candidates on a small set of your own representative tasks — the kind of work you actually do, many times. Leaderboards and launch posts tell you what someone else measured. The best model is not automatically the newest or the most expensive. It is the one that does your work reliably at a cost you can accept.

Follow the sources, not the excitement

There is far more information about AI than anyone can read, and its quality varies enormously. It helps to know what kind of claim each source can support.

Vendor information — release notes, model cards, API documentation, changelogs, deprecation notices — is the primary source for what a provider says has changed. It is where you find out that a model is being retired, that a price changed or that a new capability exists. It is still the vendor’s claim. It tells you what they say the model does, not what it will do on your work.

Research papers show where methods and capabilities are heading, often long before products catch up. But a paper is not a product. It may describe an idea, an experiment, or a result nobody has reproduced, and there may be no usable implementation at all. When you read one, ask which of those it is.

Hugging Face is one of the most useful single places to watch. Its daily papers page (huggingface.co/papers) surfaces new research that the community is discussing; its model hub (huggingface.co/models) is where open models appear, with their documentation; and both can be searched and queried programmatically, which matters for what follows. Keep four steps distinct: discovering that a paper exists, finding a model that implements it, reading that model’s documentation, and finding out whether it works for your task. Trending is a measure of attention, not of quality.

GitHub is where you find out whether an idea has an implementation you can use. Repositories, releases, examples and especially issue discussions reveal what works in practice and what breaks. You can watch selected repositories for new releases only, without following every commit. Stars and activity are signals, not guarantees.

Video demonstrations and commentators are often the fastest way to see a new tool in action, and some channels are excellent. I value them. But demonstrations are selected, edited, sometimes sponsored, and quickly out of date. Follow people whose judgement you have come to trust, and check anything consequential against primary sources and your own test.

Community discussion — forums and technical discussion sites — often reveals what announcements leave out: the rate limits, the regressions, the hidden costs. Treat it as leads to check, not facts.

You do not need many of these. A handful of well-chosen sources is worth more than a firehose.

You cannot read everything

The real problem is not finding information. It is filtering it. The question that matters is not “what is new?” but “what is new that could make a difference to what I am trying to create?”

That question has different answers for different people. A novelist working with AI cares about long-document editing, voice and handling manuscripts. A researcher cares about retrieval, reasoning and reproducibility. Someone running models locally cares about model size, memory, speed and how good the best open-weight models have become. Someone building mobile apps cares about on-device inference and battery life. Their briefings should look nothing alike.

That is why I will not give you my list of things to follow. It would be a list for my projects. Choose your topics from your own work, interests and constraints, and let a machine help you rank, group and explain what comes in. Just remember that a personalised ranking measures relevance to you, not importance in the world. Something your scout ranks first may be minor; something it filters out may matter enormously next year.

Build a machine to watch the machines

Here is the practical answer: build a small application that does the watching for you. I call mine, in its proposed form, an AI Frontier Scout. It is the second substantial thing this book asks you to build, after the arcade game, and it teaches something the game could not. The game showed that you can create software by speaking. The scout shows that you can create software that makes you better at working with AI — a tool that keeps paying back.

Its job is simple: help one person follow the changes relevant to their own interests, without reading hundreds of articles a week.

The first version needs very little:

• A few topics you choose — say, coding agents, local models and voice interfaces — that you can change later.
• Items collected from a small set of public sources, such as Hugging Face’s paper feed and the releases of repositories you care about.
• For each item: the title, the source, the link, the date it was published and the date it was collected — two different things, both worth keeping.
• Duplicates removed where practical.
• A short, readable page, with every item linking to its original.

That version needs no paid AI service at all. It collects and displays; you read. A simple GitHub repository can hold the collector, a scheduled GitHub Actions workflow can refresh it, and GitHub Pages can display the result, as with the game.

One design rule comes straight from GitHub’s own documentation: scheduled workflows can be delayed when the service is busy, and under enough load a scheduled run can be dropped without failing. So the page should always show when it last successfully updated, and the workflow should also be runnable by hand. A briefing that silently stopped updating three weeks ago is worse than none, because it looks current.

The AI layer comes second, as an enhancement. Once items are being collected, a capable model can read them and produce a personal briefing: for each item worth your attention, what appears to be new, why it might matter to you, which claims come from the source and which are the model’s interpretation, what remains unverified, whether it suggests an experiment, and the link to the original evidence. You can do this first by pasting the collected list into a conversational AI you already use. Automating it with an API is a later step, and the provider will usually charge for it separately.

Some rules hold throughout. Never put an API key in the code or on a public page; if you add one, keep it in the repository’s secret store. Never let the scout run code it finds in papers or repositories. Never let it upgrade your applications because something new was announced. Its job is to inform you, not to make high-impact changes on your behalf.

Try Machine Coding — Build Your Own AI Scout

This exercise has not been run as a demonstration for this book; it is proposed, and nothing below reports results. Speak each instruction to a coding assistant that can work with your GitHub repository, as in Chapter 13, and let the conversation go where it needs to.

Describe the idea.

I want to build a small application that keeps me informed about developments in AI. I don’t want to read hundreds of articles a week. Help me build something that collects useful updates and presents them simply. Before you build anything, ask me what I’m working on.

Choose the interests.

I’m mainly interested in AI coding agents, local models and voice interfaces. Make it easy for me to change these later. Start with a few reliable public sources, and show me where every item came from.

Build the application.

Create a simple page in my GitHub repository. Collect recent papers and relevant project releases from those sources. For each item show the title, source, publication date, the date it was collected, a short description and the original link. Remove duplicates. Keep the design clear.

Make it repeatable — and honest about it.

Make it refresh automatically on a reasonable schedule using GitHub Actions, and give me a way to run it manually. Show the time of the last successful update at the top of the page. Don’t add any paid services without asking me.

Add the thinking.

Take this week’s items and help me decide which could matter to my work. For each one, explain why, separate what the source actually claims from your interpretation, say what’s unverified, and suggest one small experiment I could try.

Make it yours.

Now ask me how much time I want to spend following AI each week, and what I’m building. Adjust the topics and the briefing so I only see what’s likely to be useful to me.

When you finish, you should have a page you can open once a week, items you can trace back to their sources, and a briefing that reflects your work rather than anyone else’s. Then change it. Someone following medical AI will want a very different scout from someone building games; that difference is the point.

From briefing to experiment

A briefing is only useful if it changes what you do. When something looks promising, a few plain questions decide whether it deserves your time. Would it improve one of my actual tasks? Would it reduce what I spend? Would it make something impractical practical? Would it make my workflow simpler? Is it reliable enough to replace what I use now? Can I try it without disrupting what already works?

The process that follows is short: discover, investigate, try, compare, then adopt or ignore. Discovery is the scout’s job. Investigation means reading the primary source, not the summary. Trying means a small experiment on a representative task, not a migration. Comparing means running the old and the new under reasonably equivalent conditions and looking at quality, cost, time, reliability, limits and the effort of switching — Chapter 07 described how to set such a comparison up so that you learn what actually caused any difference. Then decide, and write down a line about why. That line stops you from re-running the same experiment in three months, and it lets you notice when the answer has changed.

Two calibrations help. An improvement that looks spectacular in a demonstration may make little difference to your work. And a modest improvement in something you do hundreds of times can be worth more than any headline feature. Learn to tell those apart, and most announcements will sort themselves.

Your products move too

Everything above applies to your tools. It applies to anything you build on AI as well.

A product built around an AI capability lives on a moving foundation. A writing application may gain much better long-document editing when the models underneath improve. A research tool may become sharper with better retrieval. A voice interface may become fast and natural enough to change how people use it. A coding tool may become reliable enough to work unattended. Something that needed a server may become practical on a phone. Each of these is an opportunity. Many of them also bring incompatibilities, new costs, new failure modes or migration work, and a model your product depends on can change its behaviour or disappear.

That makes AI-enabled software more dynamic than much conventional software ever was. It does not mean rewriting your product every time a model improves. A personal tool can change whenever you feel like experimenting. A product other people rely on deserves conservative, tested changes, made when they serve its purpose and its users rather than because something new exists. Which kind you are building — Chapter 16’s question of purpose — decides how fast you should move.

Keep moving, keep creating

I am enthusiastic about this technology, and I think anyone working deeply with it needs to make some effort to keep up. But I want to give you permission not to follow everything.

A beginner building personal tools may need to revisit their setup only occasionally. Someone building AI infrastructure may need to follow developments closely. Someone running a stable service may rightly care more about reliability than novelty. A sustainable habit for many people might be a short weekly briefing, one promising item looked at properly, one small experiment when something warrants it, and an occasional review of the tools and subscriptions you are paying for. That is an example, not a schedule. Find the rhythm that keeps your methods improving without eating the time you meant to spend making things.

The machine should lower the cost of staying informed, not add a new administrative burden. And as these systems improve, more of the work of watching, investigating and comparing should move to them — which is the argument of this whole book applied to the book’s own subject.

Don’t spend your life trying to keep up with every machine. Build a machine that helps you recognise which changes are worth keeping up with.

What comes next

Keeping up means choosing which machines to work with. The next question is how to make the ones you choose genuinely useful to you — not a general assistant that answers you as it would answer anyone, but a working environment that knows your projects, keeps your decisions and follows your standards. That is the subject of Chapter 18, Make the Machine Yours.