AI Drift: Why Nobody Thinks Anymore

My husband asked me if he is getting smarter (or other people are just getting stupider). I laughed. But I knew what he meant. The brokerage had sent him a letter with his name and address wrong. Then again. And again. Four times. Each time, someone probably pasted the request into an AI, accepted the output, and passed it on. Nobody checked. Nobody thought.

I don't think we are getting stupider. It looks more like people are outsourcing thinking: from the intern to the CEO.

I know of many companies (and even schools) that get an enterprise Copilot account and call that the company's AI strategy. (Really!!!)

This makes the AI giants very profitable but the result?

Senior managers produce 20 pages of instructions for their staff and expect them to consume that and then provide a report. The junior staff put the 20 pages into an AI, get it to produce the report, and send it back to the senior manager.

Then guess what the senior manager does?

Call it AI drift: the compounding of small, unverified AI outputs as they pass from person to person, until the original intent is lost and no one can say who actually did the thinking. It keeps iterating and nobody thinks. Everybody just hopes someone else will.

Really? This is progress? This is a better use of resources?

I have a big problem with that.

Stop. Really. Stop.

Can someone please THINK!

Then we have the more intelligent ones. They use AI to code.

Yes, we can orchestrate multiple agents at one go. I know, because I can orchestrate 40 at one go. We become the manager or architect, deploy agents to fetch the information we require, do the vibe coding and voilà! We have a working app or program.

OMG, do you really believe this?

No. This is not how it is supposed to work.

An AI can give you a prototype. It can give you a cute app that is nice as a demo. Put it into production and it will crash. Then, you will discover the pain it has caused.

Production software isn't just code that runs. Someone still has to think about the architecture, security, data integrity, failure states, maintainability, scalability and all the strange things real users will eventually do to something you thought was idiot-proof.

We still need thinking humans to go in, understand the system, find the gaps, vet the code and architect the whole thing.

And. And the documentation.

Even humans forget the documentation. What about an AI that is being rewarded for giving you a working result? Six months later, nobody knows why something was built that way, what depends on what, or what will break if you change it.

If we don't document what was built, how are humans or robots supposed to maintain it?

AI is extraordinarily capable. It can generate code thousands of times faster than any human with no syntax error. But generating code and understanding a system are not the same thing.

Try asking an AI to debug a sufficiently complicated application that it created itself.

Unless you have actually done it, you may not realise how quickly it can start going around in circles. Ask it to refactor, and some functions are simply gone. It fixes one thing, then breaks another, patches that, yet introduces something else and, several iterations later, confidently tells you everything is working.

You test it.

Nope.

AI drift again.

There is an epistemic problem here too. When AI generates the code, reviews the code, fixes the code and then assures you that the code is correct, where did the independent verification go?

I have little doubt that eventually the technology will get better at this, but I suspect we are still some years away from AI being able to generate production code without the human.

No… I am not saying we should ditch this thing…

But an AI strategy is not giving every employee a Copilot/Claude/OpenAI licence and free chat so that the same employee who used to Google what present to buy his wife can now ask AI instead.

This is an expensive adoption. It isn't transformation.

Have we looked at the real opportunity?

Yes. To me, it is looking again at the business itself.

Where are the bottlenecks? Which processes exist in their present form only because they historically required human labour? What can be automated or redesigned completely with AI? Where can we increase capacity without increasing headcount? Can we create a product, service or revenue stream that wasn't economically possible before?

In other words, don't just put AI into the existing workflow.

Rethink the workflow because AI now exists.

That, I think, is where companies can get real value from AI even today.

And while we wait for the technology to catch up, to the point where an AI can architect, build, test and independently verify a production system without a human architect looking over its shoulder, there is something else we should perhaps be more worried about.

No doubt, the AI mind is going to keep getting better.

But if every improvement in the AI mind is accompanied by a little more atrophy in the human mind because we have outsourced yet another piece of our thinking to it, where does that eventually leave us?

If this is happening to a large population, then maybe my husband is really getting smarter. At least, he is getting quite prophetic after all.