I’ve started noticing this kind of …atrophy in people.
Mostly in others, but I’d be lying if I said I wasn’t worried I’d find it in myself if I looked hard enough.
It comes from using automation and machine learning systems, including, primarily, LLMs, for the wrong things.
Not wrong in a moral sense. I’m in no place to judge that.
Wrong as in using it as a crutch instead of a set of climbing picks.
Two examples. Bicycles and calculators.
Take someone who walks to work every morning. Call it 1.5km, 30 minutes each way, flat the whole trip so we can ignore hills.
Give that person a bicycle and the same journey takes ten minutes.
Machined bearings, a bougie (budget- and obsession- and whether-they’re-divorced-and-45-and-listen-to-too-much-Joe-Rogan-dependent) lightweight frame, and wheels mean momentum comes almost for free.
Walking taxes momentum heavily.
Stop putting one foot in front of the other and your forward motion dies. Or it continues, and your face loses an argument with a concrete gutter and you come away four teeth poorer.
Anyway. Momentum on a pushy barely gets taxed at all.
So our Lycra enthusiast now has a choice:
Hold the output constant and bank the saved effort: same trip, ten minutes, twenty minutes back in the day.
Or hold the effort constant and extend the output: same half hour of work, three or four times the distance.
The calculator is the same choice again.
Take a math test you can pass without a calculator. Addition, long division, a bit of algebra.
Every person has an upper bound on what they can score, set by ability and training.
Now hand them a scientific calculator. A good old Casio.
For the same effort, the same concentration, the same caloric spend, the amount of computation they can achieve is on a different scale entirely.
Not because the effort dropped; because the effort moved.
It’s no longer spent carrying the one and cross-checking arithmetic. It’s spent rearranging formulas, choosing which formula applies, and understanding what the problem is actually asking.
That’s the tool used properly. Same effort, different class of problem.
The obvious ladder version: no one would argue that a person with a screwdriver can’t assemble more IKEA furniture than a person with a knife, or the knife over the spoon.
Spoon-guy has no hope, and not just at the furniture.
Knife-guy isn’t massively better off, but are you reaaaally going to tell him? He’s holding a knife.
Screwdriver-guy is all big-brains and clearly the intellectual and a bit smug, annoyingly. Until:
Cordless Drill Guy comes in with the opposite of BDE and outdoes all of them combined.
But the caveat matters more than the ladder:
Screwing shit in is not the entirety of assembling flat pack furniture.
The drill accelerates one sub-task and the rest of the job remains. That’s not a hole in the argument, as much as it’s a constraint on it. The gain is real but partial, and the question of where you spend it still stands.
Spoon-Guy is still cooked though.
Moving on. Same thing, but invert it.
Give the bicycle to someone who only ever rides the same round trip and banks the twenty minutes. Give the calculator to someone sitting the same test they used to pass at 90 percent on raw effort.
They atrophy.
Not just the fitness or the math of it. The tolerance for discomfort goes.
The ability to apply sustained effort goes. Over time, I’d argue the base capability goes too.
The 1.5km walk and the no-calculator test just …stop being things they can do. Which is what I mean about people using LLMs wrong.
They’re automating the work they were already doing three years ago and spending the surplus on nothing, or worse, on posting about it.
There’s an efficiency gain in that, obviously. But it’s a winning position in a losing game.
Margins on “the same thing, faster” compress toward zero as everyone else gets the same tools. Same shape as asking for the industry standard.
The people who come out ahead will be the ones doing things that were simply not possible three years ago, or not possible without stupid amounts of funding.
If you’re not doing that, you’re leaving the upside on the table, yes, but you’re also rotting the one capability you had before the tools arrived.
This scales past the individual.
When the wheel was first discovered, even as log rolling, a civilisation that only ever used it to do the same shit it had always done would have fucked around with log rolling and found out via plague or famine.
Statistically, given enough time, the civilisation that never advances past certain thresholds dies sooner than the one that does.
Same mechanism at every scale: person, business, civilisation. Using the tool only to maintain the status quo doesn’t even achieve that, because the world keeps moving while you don’t.
My bias: I would much rather err toward fucking around and finding out too much than too little.
I think that’s broadly directionally right. Of course I do. That’s why it’s my bias.
But it carries a non-zero risk with a non-trivial consequence, that you go too hard and too fast and build something you cannot operate without frontier-level compute.
I’ve picked my side of that trade knowing the risk is real, not because I’ve argued it away.
And there’s a compounding risk sitting under this that I don’t hear discussed much. A lot of this is happening on subsidised compute, priced at some fraction of what it costs to deliver.
If model efficiencies don’t close that gap and inference costs correct upward, the person who atrophied their native ability and then loses affordable access to the crutch is… cooked.
They can’t do the old thing anymore and they can’t afford the new thing.
I don’t have a clean solution for that. I don’t think anyone does.
Other than: don’t be dumb. Which is really helpful inner-self-talk and really unhelpful advice.
Backups on backups
So, what I do have is a way of thinking about it, and it comes from working on aircraft.
Every serious institution that runs on complex systems, defence, NASA, airlines, keeps an analogue fallback. Never as capable as the primary.
If your avionics go down there are things you simply cannot do.
But the flight control system has backups on backups, and the pattern in those backups is consistent: each layer strips abstraction until the line from input to observable physics is as direct as the system allows.
And to be clear, I don’t know if I’m saying observable physics correctly there.
I’m talking about electrons and transistors and hydraulic system go sleep-sleep but plane still fly and car still turn.
I’m really bad at the level of big words I probably misuse, and the piss-take ELI5 version.
Gravity and levity, or something.
But, to try:
Pull back on the column, the surfaces move, the nose comes up.
Power steering dies in your car and the wheel gets heavy, but it still steers.
Plenty of people driving today have never steered a car without power assistance, but if you’re reading this I can (for now) assume you’re not one of them.
It’s still possible. It just costs more effort. (…noticing the theme, yet?)
When I worked on jets, before bomb disposal, every part that went onto an aircraft was tracked in a system called CAMM2, down to a buckle on an ejection seat harness.
Serial numbers, part numbers, compatibility, other things I can no longer remember, you name it.
When that system went down, and it did, certain things became impossible. You could not run certain reports or forecasting on paper.
But you could still work on the jet, because there were printouts. Massive, tedious sheets of paper, filled in with pen, never pencil, and a documented manual process for using them.
The printouts themselves ran on a cadence, daily at a minimum, and I’d assume closer to hourly depending on where you were operating and the threat level.
The intensity of the backup tracked the live risk, not a policy written once and filed.
In information space it’s the same move as the flight controls: the paper system can’t do what the database does, but “which serial number is on which airframe” is still a lookup a human can do with a pen.
The part I thought was stupid at the time and am only realising was literally lifesaving ~10 years later, is drills:
We ran drills on the paper system.
And the first time we did, it slowed everything down massively, because none of us knew how to do it well.
That’s exactly why the drill exists. A fallback nobody has rehearsed is documentation, not redundancy. The backup is the practised capability, not the paper and ink.
Not everything needs this.
Part of the fun of building right now is that you can experiment and iterate and fuck around and find out for a fraction of what it used to cost in time and cash.
If you’re testing, the likelihood of something breaking is near certain and the consequence is near zero, so building scaffolding around failure modes is a waste.
But the moment a thing touches money, safety, other people’s data, other systems that depend on it, or output that leaves the building, the maths changes. A sub-one-percent likelihood with a high consequence gets a backup, a check, and a rehearsal. I don’t care how unlikely it is.
The trap is that nothing really tracks the moment a toy becomes structural.
It happens one connection at a time. A client task starts routing through it. A scheduled job starts depending on it. Someone else’s data ends up inside it.
The original risk assessment was correct on the day it was made, even if that risk assessment was no risk assessment, because you were just fucking around.
That was correct at the time. Maybe it’s not anymore.
Aviation forces reassessment whenever operational context changes, and the printout cadence is the same principle running continuously.
The solo-builder equivalent is a standing rule: the moment anything external connects, the thing gets re-scored, regardless of how it started life.
Stop checking 4+4
Which brings me to critical thinking, a term I now hear used mostly by flogs mimicking each other on LinkedIn.
The standard advice is “don’t believe what the AI tells you, think for yourself.”
At some point, and we’re not there yet, that becomes bad advice taken literally.
Nobody types 4+4 into a scientific calculator and presses equals seven times asking “are you sure?”
That would be dumb. And it would also keep changing the answer if you just kept mashing equals.
Point: past a proven level of capability, that checking is pure waste.
And it’s not neutral waste. Verification comes out of the same budget as building. And ideating. And, you know, food.
Every unit of effort spent re-confirming what the system has already proven it can do is a unit not spent on the class of problem the tool was supposed to unlock.
Under-trusting a proven system and over-trusting an unproven one are the same error pointed in opposite directions.
The freed effort has a correct destination, and it isn’t more checking of the same kind.
Re-checking is a per-instance spend that produces nothing durable. An invariant is designed once and holds against every future execution.
Design tests that must pass through different paths and different mechanisms.
Run the same problem through a different model of calculator entirely and see if the answers converge.
Ask the model what it’s missing, what you’re missing, what you’re both missing.
Then don’t accept that adversarial pass blindly either. Which is all just “Don’t be dumb” in more words.
The other thing I will rant about:
Calibrate all of it against your own record, not a benchmark.
Not what the vendor says the model scores. Not YouTubers “one-shotting” a FlappyBird clone in 3D with GTA-style violence and bird-themed love interests that look like furries.
Pick your own internal metric. E.g:
I have run this class of problem through this system across this many projects, and my fuck me that didn’t go according to plan rate has moved from here to here.
It has to be internal, because if you’re doing this properly, the things you’re building don’t have a benchmark.
For your own financial and economic future, read the above again.
All you have is a throttle and a brake pedal.
Keep checking 4+4 and you waste the machine.
Never check anything and the jet falls out of the sky, because nobody caught the floating point error in the vibe-coded spreadsheet someone used as a calculator because their Casio hasn’t been touched in 15 years and still has penis drawings all over it.
The class that was never told no
For all of human history, efficiency gains have automated mechanical work, and the occupations automated out of economic relevance were, by self-selection or circumstance, rarely the people designing the replacements.
The intellectual class was never the one being replaced. They were the ones doing the replacing.
Even when smart people were displaced, the scribes after the press, the human computers at NASA (full disclosure, Gemini gave me that reference, and it was a good one), the decision came from a legible human authority they could see, argue with, or become.
That’s the coping machinery that’s missing now.
The knowledge class has never had to build it, because there was always a human standing there, backed by capital, saying no, this is the way forward. Mostly to great benefit.
Medicine, healthcare, economic efficiency.
But for the first time, people whose entire contribution has been intellect, degree, or pedigree are staring down automation with no human on the other side of it and no obvious path to becoming the replacer, because the replacer is a lab plus capital.
My honest read on the next five to ten years: the people who go furthest won’t be pure labourers and won’t be pure academics.
They’ll be the ones who have done both at some point, or can channel both.
The ability to do hard physical, practical work and improvise when it breaks, and the ability to step back and ask what’s being missed and whether there’s a better path entirely.
A lot of people are figuring this out live, in public, with no one to appeal to. That is an uncomfortable place to be.
And I think it should be, by design.
Confidence correlates inversely with distance from the frontier, and your own frontier. You can only be that certain when nothing you’re building is hard enough to break.
The people posting with the confidence of the tip of the spear, no room for doubt anywhere in it, are usually the furthest from it.
I learned that the hard way by listening to a lot of them for too long. Now, I tune in only ever if I need a cheap salve for paralysing impostor syndrome.
I’m still at the edge of what I can learn.
Still breaking things. Still making mistakes. That’s why I build safeguards.
The point? There isn’t one.
Go build something. Something cool.
That, and:
If you’re lecturing other people on critical thinking, you should probably think about critical action.
And if vice versa, then… vice versa.