Get the top HN stories in your inbox every day.
EdNutting
jkubicek
I’m going to defend Claude. If you give a robot the ability to delete production it’s going to delete production. This is 100% the user’s fault.
This problem has not yet been solved and will never be solved.
elashri
> If you give a robot the ability to delete production it’s going to delete production
If you give an intern the ability to delete production, it's going to delete production. But to be honest you can as well replace "intern" or "robot" by human in general. Deletion in production should have safety layers that anyone cannot accidentally do it, specially without the ability of rolling back.
belZaah
That’s a broken analogy. An intern and a llm have completely different failure modes. An intern has some understanding of their limits and the llm just doesn’t. The thing, that looks remarkably human, will make mistakes in ways no human would. That’s where the danger lies: we see the human-like thing be better at things difficult for humans and assume them to be better across the board. That is not the case.
Neywiny
I think the difference, though maybe I'm incorrect, is that when we have interns on our codebase they get restricted permissions. Can't push to prod, need pull requests with approvals and reviews, etc. Certainly can't delete backups. Whoever setup the robot's permissions did it wrong. Which is interesting because early on there were people complaining that these AIs refused to push to main, but now this stuff keeps happening.
anonzzzies
I had a senior tech lead delete production doing a late night quick fix. Especially in panic mode where sometimes processes are ignored, things are going to go wrong. Don't need interns for that, nor llms.
EdNutting
"Anything that can go wrong, will go wrong. Including deleting production."
I'm also waiting for the day we see a "Claude sold my production database on the darkweb" post haha.
spl757
You can't fix stupid.
groby_b
Best part: The guy's "training engineers to use AI in production environments".
And it's not all Claude Code - loved the part where he decided, mid disaster recovery, that that would be a good time to simplify his load balancers.
It's a case of just desserts.
semiquaver
The narrative incudes this:
> Claude was trying to talk me out of [reusing an existing AWS account for an unrelated project], saying I should keep it separate, but I wanted to save a bit
EdNutting
An expert would’ve at least taken a backup or checked existing backups weren’t going to be destroyed. Silently - without asking their manager - they just do defensive engineering as a good practice. Or they would’ve, at minimum, highlighted the suggestion, which doesn’t seem to have happened in this case. As someone who recently did a short term contract to capture manually created AWS infrastructure into CDK, I can tell you this was one of my first moves!
So, Claude as a tool: sure, this is user error. Claude could be improved by making it suggest defensive steps and making it push harder for the user to do them first, but it’s still down to the user. I’ve repeatedly encountered this issue that Claude doesn’t plan for engineering - it just plans to code - even with Claude.md and skills and such.
Claude as a replacement for engineers? Well, yeah, the marketing is just that: marketing.
tapoxi
The user's bio is literally "Teaching engineers to build production AI systems"
It would be funny if these LinkedIn/Twitter influencers weren't so widespread.
akouri
Well, he definitely taught me what not to do
QuantumGood
Marketing "protect your business from harm by Claude internally" seems to be a growth industry.
fragmede
And they said Claude was gonna take our jobs.
tomwphillips
I'm not sure a staging environment would have caught it.
I often find Claude makes changes that _look_ reasonable, but it's only when I really dig in (e.g. when refactoring) that I realise there's insidious problems.
I can imagine the author making the changes in a staging environment, seeing that it _appears_ to be ok, then blowing up production anyway.
(AI aside, staging is a lie: https://www.tomwphillips.co.uk/2026/01/staging-is-a-wasteful...).
gos9
Considering engineers have made similar mistakes I’m not so sure that’s a great razor, haha
overfeed
Usually junior engineers accidentally drop dbs.
Lacking backups and staging/test environments is organizational failure: everyone who is between senior and the CTO is to blame for not fixing it post-haste.
EdNutting
Usually engineers who have not recently been trained on well documented examples of what to do and what not to do and the consequences ;)
(Yes, I chose the word "trained" intentionally)
ethbr1
So what I hear is after this makes the training set, Claude Code might get a promotion from junior to level 1?
weikju
Itd be nice if our computer programs were more deterministic, that's what we use computers for. Not to repeat failure modes of humans.
tayo42
> And to top it off, Claude (the supposed expert tool) didn't repeatedly output "Are you insane?
It did though acoording to the article and he ignored it.
The Ai can only work with what you tell it.
EdNutting
The difference is, an expert engineer would flat-out refuse to do these things and would keep pushing back. Claude may sometimes attempt _one time_ to warn someone, and then (after consent fatigue means they're just blindly clicking "yes"), it ploughs right ahead without further complaint.
tayo42
Do you really want the Ai to not do the things you tell it?
It only knows what you tell it, if you tell it risky operations are OK, what do you expect?
SunshineTheCat
Putting yourself in a situation where this could happen is kinda insane, right? Could be something I'm missing.
I can't think of any specific example where I would let any agent touch a production environment, the least of which, data. AI aside, doing any major changes makes sense to do in a dev/staging/preview environment first.
Not really sure what the lesson would be here. Don't punch yourself in the face repeatedly?
levkk
As the tool gets better, people trust it more. It's like Tesla's self-driving: "almost" works, and that's good enough for people to take their hands off the wheel, for better or for worse.
The "almost" part of automation is the issue + the marketing attached to it of course, to make it a product people want to buy. This is the expected outcome and is already priced in.
sofixa
Exactly, Waymo were talking about this a few year back, they found that building it up gradually will not work, because people would stop paying attention when it's "almost" there, until it isn't and it crashes. So they set out on having their automation good enough to operate on its own without a human driver before starting to deploy it.
nine_k
I would say the opposite here. The perpetrator has rejected multiple Claude's warnings about bad consequences, and multiple Claude's suggestions to act in safer ways. It reminds me of an impatient boss who demands that an engineer stopped all this nonsense talk about safety, and just did the damn thing quick and dirty.
Those guys who blew up the Chernobyl NPP also had to deliberately disable multiple safety check systems which would have prevented the catastrophe. Well, you get what you ask for.
fragmede
I view it more as "I crashed my car, I should have been wearing my seat belt, wear yours!"
Source: had codex delete my entire project folder including .git. Thankfully I had a backup.
insane_dreamer
But that's the "promise" of AI (that management believes), isn't it? That it can replace an engineer because it's as good or better -- so why wouldn't you allow it to touch your production database? (I agree with you, just pointing out the difference between what's being sold and reality.)
jeanlucas
Yep, you're not insane, they were amateur.
nativeit
I wonder if Iran is considered a “production environment”?
happytoexplain
Why are you writing in this defensive manner? The post isn't an anti-AI screed, it's a "I screwed up, here's what I did and how to avoid it."
You say "Not really sure what the lesson would be here", but the entire contents of the blogpost is a lesson. He's writing about what he changed to not make the same mistake.
There is a total mismatch between what's written and how you're responding. We don't normally call people idiots for trying to help others avoid their mistakes.
The culture war around AI is obliterating discourse. Absolutely everything is forced through the lens of pro-AI or anti-AI, even when it's a completely neutral, "I deleted my data, here's what I changed to avoid doing it again", where the tool in question just happens to be AI.
abustamam
I didn't take it to be defensive. A bit tongue in cheek, but not defensive. I think the person you're responding to has a good point though. AI or not, you probably shouldn't futz around with prod before doing so in a lower env. Guardrails for both AI and humans are important.
stavarotti
If you found this post helpful, follow me for more content like this.
I publish a weekly newsletter where I share practical insights on data and AI.
It focuses on projects I'm working on + interesting tools and resources I've recently tried: https://alexeyondata.substack.com
It's hard to take the author seriously when this immediately follows the post. I can only conclude that this post was for the views not anything to learn from or be concerned about.
inhumantsar
what's truly incredible is that this person is selling bootcamps.
the things they "didn't realize" or "didn't know" are basics. they're things you would know if you spent any time at all with terraform or AWS.
all the remediations are table stakes. things you should at least know about before using terraform. things you would learn by skimming the docs (or at least asking Claude about best practices).
even ignoring the technical aspects, a tiny amount of consideration at any point in that process would have made it clear to any competent person that they should stop and question their assumptions.
I mean, shit happens. good engineers take down prod all the time. but damn man, to miss those basics entirely while selling courses on engineering is just astounding.
the grifter mentality is probably so deeply engrained that I'm willing to bet that they never once thought "I'm totally qualified to sell courses", let alone question the thought.
undefined
6thbit
why would you repost that here?
are you secretly OP trying to get substack hits?
fragmede
It's hard to take you take seriously. His blog has a generic "read more" footer and that's a demerit worth mentioning? What do serious people in your world that write blogs do? Not want people to read their other content? In what world do you live in that writers (serious or not) don't want you to read their other work?
LelouBil
It's not a generic footer, it's their reply directly to their tweet about the incident.
I agree with the person you are replying to, writing a tweet like :
"How I misused AI and caused an outage"
and replying to this very tweet saying
"Here's a blog where I write insights about AI"
Obviously do not make me want to read the blog.
mnky9800n
Some people seem think that self promotion is wrong and work should stand on its own merits. I don’t think this way. It’s important to think about engaging and attracting eyes to your ideas. If you don’t why bother sharing them?
fragmede
Self promotion being wrong has never met reverse psychology. Ego hacking is a thing.
xmodem
An engineer recklessly ran untrusted code directly in a production environment. And then told on himself on Twitter.
petcat
From the article, it sounds like that engineer did a lot of other reckless things even before handing the tasks over to the AI agent to continue the recklessness with even more abandon.
This is a case study in "if you don't know what you're doing, the answer is not just to hand it over to some AI bot to do it for you."
The answer is to hire a professional. That is if you care about your data, or even just your reputation.
VWWHFSfQ
> before handing the tasks over to the AI agent to continue the recklessness with even more abandon
Which is a funny outcome of this because apparently the AI agent (Claude) tried to talk him out of doing some of the crazy stuff he wanted to do! Not only did he make bad decisions before invoking the AI, he even ignored and overruled the agent when it was flagging problems with the approach.
abustamam
Yeah I always considered AI to be an accelerator. If you don't know what you're doing and would break stuff without Ai, AI will just accelerate that.
EdNutting
"To err is human; To really foul things up requires a computer."
Extended with: "To really foul things up quickly, requires an AI tool."
Ancalagon
This is literally what major company execs want engineers and eventually their agents to do.
semiquaver
I’m not going to “defend” the LLM here but this:
> I forgot to use the state file, as it was on my old computer
subscribed
And the single terraform wiped the snapshots too?
I'd say skill issue.
mizzao
AI is like having a chainsaw when you only had a bow saw before. You can cut down the tree 10x faster or you can cut off your foot completely.
NicuCalcea
Quite funny that that author followed up with this tweet:
> If you found this post helpful, follow me for more content like this.
> I publish a weekly newsletter where I share practical insights on data and AI.
tomcatfish
Despite multiple comments blaming the AI agent, I think it's the backups that are the problem here, right? With backups, almost any destructive action can be rolled back, whether it's from a dumb robot, a mistaken junior, or a sleep-deprived senior. Without, you're sort of running the clock waiting for disaster.
forgotaccount3
Yes, backups are great but a 'dumb robot' or a 'mistaken junior' shouldn't have access to prod.
And a sleep-deprived senior? Even then. They shouldn't have access to destructive effects on prod.
Maybe the senior can get broader access in a time-limited scope if senior management temporarily escalates the developers access to address a pressing production issue, but at that point the person addressing the issue shouldn't be fighting to stay awake nor lulled into a false sense of security as during day to day operations.
Otherwise it's only the release pipeline that should have permissions to take destructive actions on production and those actions should be released as part of a peer reviewed set of changes through the pipeline.
JoBrad
If a sleep-deprived senior shouldn’t have access to prod, I think we have big problems, frankly.
fragmede
Which, if you're Google-sized, you have follow-the-sun rotations, in order to avoid that problem. But what about the rest of the class?
charcircuit
But smart robots like Claude should and will have access to production. There has to be something figured out on how to make sure operation remains smooth. The argument of don't do that will not be a viable position to hold long term. Keeping a human in the loop is not necessary.
b112
It is absolutely necessary. Point in fact, most DEVs don't have access to PROD either. Specialists do.
Clause, maybe, is a junior DEV.
Not a release engineer.
abustamam
Should and will are pretty large assumptions given the the post we're commenting on!
> will not be a viable position to hold long term
Why not? We've literally done it without robots, smart or dumb, for years.
QuercusMax
Nobody, not even a "smart robot" should have unfettered read-write production access without guardrails. Read-only? Sure - that's a totally different story.
Read-write production access without even the equivalent of "sudo" is just insane and asking for trouble.
esseph
> Keeping a human in the loop is not necessary.
You don't work in anything considered Safety Critical, do you?
hobs
You need to care about your Recovery Time (how long does it take to get back up again?) and your Recovery Point(how long since your backup was taken?) and it gets Much Worse when you start distributing state around your various cloud systems - oh did that queue already get that message? how do we re-send that? etc
happytoexplain
They are two orthogonal issues. One doesn't make the other irrelevant.
tomcatfish
I agree that a second issue doesn't erase the first, but also I've got enough work experience to know that a system which can be brought down by 1 person no matter the tooling they use is a system not destined to last for long.
Joel_Mckay
Zero workmanship was always worth nothing.
It usually takes about 10 months for folks to have a moment of clarity. Or for the true believer they often double down on the obvious mistakes. =3
clouedoc
100% agree. Everyone should always backup their production database somewhere where's it's not trivial to delete.
import
Well apparently guy were running tf from his computer and ask claude to apply changes while not providing state file, and “blaming” claude for the catastrophic result?
01284a7e
I'm cool with blogging about your mess-ups, sort of. Is "I'm incompetent" a good content strategy though? Yeah, you're going to get a lot of traffic to that post, but what are you signaling? Your product is a thousand bucks a year. I would not go near it.
hahajk
I'm absolutely loving the genre of "chatbot informing user it messed up real bad":
> CRITICAL: Everything was destroyed. Your production database is GONE. Let me check if there are any backups:
> ...
> No snapshots found. The database is completely lost.
sva_
It's the still kind of uplifting tone that gets me. Like the task has finally been completed
INTPenis
And this guy is selling tech courses.
I'm no AI advocate, I have been using it for 6 months now, it's a very powerful tool, and powerful tools need to be respected. Clearly this guy has no respect for their infrastructure.
The screenshot he has, "Let me check if there are backups", a typical example of how lazy people use AI.
tdsanchez
That’s why you tell CC to do a ‘terraform plan’ to verify it’s not wrecking critical infrastructure and NEVER vibe-code infrastructure.
Get the top HN stories in your inbox every day.
No staging environment?
No prior attempt to follow best practices (e.g. deletion protection in production)? Nor manual gating of production changes?
No attempt to review Claude's actions before performing them?
No management of Terraform state file?
No offline backups?
And to top it off, Claude (the supposed expert tool) didn't repeatedly output "Are you insane? No, I'm not working on that." - Clearly Claude wasn't particularly expert else, like any principal engineer, it would've refused and suggested sensible steps first.
(If you, dear reader of this comment, are going to defend Claude, first you need to declare whether you view it as just another development tool, or as a replacement for engineers. If the former, then yeah, this is user error and I agree with you - tools have limits and Claude isn't as good as the hyped-up claims - clearly it failed to output the obvious gating questions. If the latter, then you cannot defend Claude's failure to act like a senior engineer in this situation.)