Where's the shovelware? Why AI coding claims don't add up

These kinds of "clients" have always been around. If they want to tell me how long they think it should take, my answer has always been that they should do it themselves. If they say something like, "it shouldn't take too long," I said "send me what you need it to do." Then I look at that and ask "what if A, B or C happens? What if a user does X?" Make a list about 15 bullet points long of edge cases they hadn't thought of showing, the flaws in their business logic. (This, btw, is what I'd do if I were instructing an LLM as well). Then it's, "well, how long will that take?"

And my answer - like the best of car mechanics who work on custom rides - is: I don't know how long until I actually get in there, but minimum 5x what you think, and my rate is $300/hr. Is it worth it to you to do it right?

Usually the answer is no. When it's yes, I have a free hand. And having a few clients who pay well is worth a lot more than having a few dozen who think they know everything and are too cheap to pay for it anyway.

I have fully vibe coded several apps at this point (none professionally, but stuff I actively use in my life). One thing I don't think everyone understands is that it still takes time. I need to do countless rounds of testing, describing the exact problem I find, rinse and repeat again and again for hours and hours. I happen to enjoy this way of working and I do think it's faster than writing the code myself, but it's not fast.

If it's 5hs, why not distract his own engineers?

That's pretty much nothing ... to me that line indicates a whole lot of other possible things.

[flagged]

I think a kind of AI complacency has set in. Companies are just in chill mode right now, laying off people here and there while waiting for AI to get good enough to do actual work.

First thing I thought of was Benioff saying he cut thousands of customer support roles because AI can do it better then turning around and giving lackluster earnings report with revised down guidance and the stock tanks

I have never, ever seen SVPs, CEOs, and PMs completely misunderstand a technology before. And I agree with you, I think it's more of an excuse to trim fat--actual productivity is unlikely to go up (it hasn't at our Fortune 500 company)

> we are heading towards a big recession

Who is we? One country heading into a recession is hardly enough to nudge the trend of "all things code"

The whole 'startup/scaleup' culture (which have become the industry titans of today) - is insane and have been insane for as long as it has been a thing - the culture of either 'just grow and figure out how to monetize'(Social media, food delivery etc.) or 'we're selling you this technology that doesn't exist yet but going to be insanely valuable in the future (AGI, self driving)' or 'we're selling shovels to the first 2' (cloud providers, Nvidia) has been insane.

I'd argue that compared to a decade. 15 years ago, relatively little value has been created. If you sat down in front of a 15 yo computer, or tried to solve a technical challenge with the tooling of 15-10 years ago, I don't think you'd get a significantly worse result.

Yet in this time the US has doubled its GDP, most of it owning to the top, to the tech professionals and financiers who benefited from this.

And some of this money went into assets with constrained supplies, such as housing, marking up the prices adjusted for inflation, making average people that much worse off.

While I do feel society is making progress, it's been a slow and steady march. Which in relative terms means slowing. Of I gave you $10 every week, by week 2, you'd double your wealth, by the end of the year, you almost didn't notice the increase.

Technology accumulation is the same, I'd argue it's even worse, since building on top of existing stuff has a cost proportional to the complexity for a fixed unit of gain (and features get proportionally less valuable as you implement the most important ones first).

Sorry got distracted from my main point - what happens when people stop believing that these improvements are meanigful or that technology that was priced in to produce 100x the value will come at all (and more importantly the company you're invested in will be able to caputre it)?

> That's insane. Who the hell pulls a number out of their ass and declares it the new reality?

Product and Sales?

VXUS

Not investing advice; the bottom 490 companies in the S&P500 are nominally flat since 2022 and down against inflation, GPUs and AI hype are holding everything together at the moment.

> In simpler terms, 35% of the US stock market is held up by five or six companies buying GPUs. If NVIDIA's growth story stumbles, it will reverberate through the rest of the Magnificent 7, making them rely on their own AI trade stories.

https://www.wheresyoured.at/the-haters-gui/

> Capex spending for AI contributed more to growth in the U.S. economy in the past two quarters than all of consumer spending, says Neil Dutta, head of economic research at Renaissance Macro Research, citing data from the Bureau of Economic Analysis.

https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-...

> Two Nvidia customers made up 39% of Nvidia’s revenue in its July quarter, the company revealed in a financial filing on Wednesday, raising concerns about the concentration of the chipmaker’s clientele.

https://www.cnbc.com/2025/08/28/nvidias-top-two-mystery-cust...

> He's the one who will get fired.

I wouldn’t count on it.

Here's a new keyboard. I've cut all your estimations by five percent; surely you can type much faster with this.

> That's insane. Who the hell pulls a number out of their ass and declares it the new reality?

Chatgpt.

Them managers have always been pulling a number out of their ass.

The CEO doesn't care. He'll fail upwards. Be able to spin it as increasing shareholder value by cutting costs and bounce before the chickens come home to roost.

I agree with you and I'm being 0% facetious.

I think making stakeholders have to engage with these models is the most critical point for people having deadlines or expectations based on them.

Let Claude run incident response for a few weeks. I'll gladly pause pagerduty for myself.

I think Chuck Prince's "As long as the music is playing, you've got to get up and dance. We're still dancing." from the GFC https://www.reuters.com/article/markets/funds/ex-citi-ceo-de... is the more relevant famous line here.

Do not forgive them. We already have a description for them:

"A bunch of mindless jerks who'll be the first against the wall when the revolution comes."

Oh, they for sure know what they're doing.

Counterpoint: fuck them, they know exactly what they do (try to extract more work for the exact same pay out of their subordinates)

also customer service; I was at my dental office today and there were 3 people handling checkin/checkout. I'm quite confident 80% of their workload could be automated away to where you would just need a single person to handle edge cases. That's where we're going to see a lot of entry-level jobs go away, in many domains.

I actually built something (a time tracking tool that helps developers log their time consistently on jira and harvest) that most developers in my company use in under a week.

I have backend development background so I was able to review the BE code and fix some bugs. But I did not bother learning Jira and Harvest API specs at all, AI (cursor+sonnet 4) figured it out all.

I would not be able to write the front-end of this. It is JS based and updates the UI based on real-time http requests (forgot the name of this technology, the new ajax that is) and I do not have time to learn it but again, I was able to tweak what AI generated and make it work.

Not only AI helped me do something in much shorter than it would take, it enabled me do something that otherwise would not be possible.

I'd rather see those SVPs, PMs, and line managers be turned into AI.

Yeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with.

If I have to manually verify every answer, I may as well read the docs myself.

Personally, I don't trust LLMs to write code for me, generally speaking. That said, as of late I've been very pleased with the whole "shoehorn your problem to fit code examples found online" thing these LLMs do, in the very special case of massaging unix scripts, and where the "code examples found online" part seems to mostly amount to making fairly canonical reference to features documented in man pages that are plastered all over the web and haven't changed much in decades.

For questions that I know should have a straightforward answer, I think it beats searching Stackoverflow. Sure, I'll typically end up having to rewrite most of the script from scratch; however, if I give it a crude starting point of a half-functional script I've already got going, pairing that with very clear instructions on how I'd like it extended is usually enough to get it to write a proof of concept demonstration that contains enough insightful suggestions for me to spend some time reading about features in man pages I hadn't yet thought to use.

The biggest problem maybe is a propensity for these models to stick in every last fancy feature under the sun. It's fun to read about a GNU extension to awk that makes my script a couple lines shorter, but at best I'll take this as an educational aside than something I'd accept at the expense of portability.

Before accepting an answer I’ve started asking “is there a simpler more straight forward way of achieving that?” Ans most of the time it changes the whole thing it just wrote lol

No, you've got it backwards. If anything, people are getting tired of comments like yours.

Just recently I was having trouble getting something to work with a large well documented framework library. Turned out the solution I needed was to switch to a similar but different API. But that's not what Claude told me. Instead it wanted me to override and rewrite a bunch of core library code. Fortunately I was able to recognize that the suggested solution was almost certainly bad and did some more digging to find the right answer, but I could easily see nightmarish code that solves immediate problems in terrible ways piling up fast in a vibe coded project.

I do find LLMs useful at times when working in unfamiliar areas, but there are a lot of pitfalls and newly created risks that come with it. I mostly work on large existing code bases and LLMs have very much been a mildly useful tool, still nice to have, but hardly the 100x productivity booster a lot of people are claiming.

This keeps happening to me. I keep coming across big files written during my Cursor hype period from 3 months ago and finding huge non DRY chunks and genuinely useless nonsense. Yes, I should have reviewed better, but it's a lot to wade through and it ostensibly "worked," as in, the UI looked as it should.

Today I asked Claude how to ignore Typescript type checking in some vendored js files in my project. It churned on this and ended up turning off type checking on all js files in my project and proudly declaring it a great success because the errors were gone. Hurray. If I knew nothing about my project then I would be none the wiser.

Nail polish remover

I think it was a typo and should have been "Stop using a hammer on screws," suggesting tool / application mismatch.

PHP of course!

https://blog.codinghorror.com/the-php-singularity/

This glass bottle

And they were frequently in public repos that were updated with people filing issues if necessary.

An MCP server called Context7 excels at providing up to date api/library documentation for LLMs.

Maybe the world has run out of interesting websites to create. That they are created faster doesn't necessarily imply they'll be created more frequently.

Exactly. It quickly builds a lot of technical debt that must be paid down, especially for people writing code in areas they aren't deep in.

For tight tasks it can be super helpful -- like for me, an AI/Data Science guy, setting up a basic reverse proxy. But I do so with a ton of scrutiny -- pushing it, searching on Kagi or docs to at least confirm the code, etc. This is helpful because I don't have a mental map about reverse proxy -- so it can help fill in gaps but only with a lot of reticence.

That type of use really doesn't justify the billion dollar valuations of any companies, IMO.

Even scaffolding a new project is not easy work, especially a new stack or new versions of existing tools. For example, I have never been able to create a Vue 3 project with Vite and Tailwind setup correctly. I tried top SOTA models. Maybe my prompting skills are not good, but everytime it fails to set up a project correctly. Everytime it gives me some old configurations that's not relevant anymore.

The reason LLMs suck in plenty of brownfield projects is because those codebases likely either implemented frameworks in a proprietary way, maybe did not rely on any public framework at all, or were in general done in an esoteric way and therefor few (if any) similar codebases exist within the LLMs training data. Which is problematic because LLMs aren't capable of reasoning or learning they're literally just predicting the next most likely token in a chain similarly to how autocomplete works. Without you supplying additional context and explicitly defining guardrails for preforming common tasks the LLM has no frame of reference for working with your codebase.

I've had great success with GPT5 in existing projects because its agent mode is very good (the best I've seen so far) at analyzing the existing codebase and then writing code that feels like it fits in already (without prompt engineering on my part). I still agree that AI is particularly good on fresh projects though.

I actually completely disagree with this, and IMO it works best with projects that are templated with AI development in mind. Lots of documentation and comments, working tests, etc.

You want as much context as possible _right in the code_.

You don't need sotware engineering to build successful software, until you do.

In my experience you don't need to know a whole lot about LLM's to work them. You need to know that everything they spit out is potential garbage, and if you can't tell the good from the garbage then whatever you're using them for is going to be terrible. In terms of software terrible is fine for quite a lot of systems. One of the first things I build out of university in the previous millennium is still in production today and it's horrible. It's inefficient, horribly outdated since it hasn't been updated ever. It runs 10 times a day and at least 1 of them will need to automatically restart itself because it failed. It's done it's job without the need for human intervention for the past many decades though. I know because one of my old colleagues still works there. It could've been improved, but the inefficiency cost over all those years is probably worth about two human hours, and it would likely take quite a while to change it. A lot of software is like that, though a lot of it doesn't live for so long. LLM's can absolutely blast that sort of thing. It's when the inefficiency cost isn't less than a few human hours that LLM's become a liability if you don't know how to do the engineering.

I use LLM's to write a lot of the infrastructure as code we use today. I can do that because I know exactly how that should be engineered. What the LLM can do that I can't, is that it can spit out the k8s yaml for an ingress point with 200 lines of port settings in a couple of seconds. I've yet to have it fail, probably because those configurations are basically all the same depending on the service. What a LLM can't do, however, is write the entire yaml config.

Similarily it can build you a virtual network with subnets in bicep based on a couple of lines of text with address prefixes. At the sametime it couldn't build you a reasonable vnet with subnets if you asked it to do it from scractch. That doesn't mean it can't build you one that works though, it's just that you're likely going to claim 65534 ip addresses for a service which uses three.

The first one.

I'm not talking about generating a few lines instead of importing left-pad. In recent PRs I've had:

- Implementing a scheduler from scratch (hundreds of lines), when there are many many libraries for this in Go.

- Implementing some complex configuration store that is safe for concurrent access , using generics, reflection, and a whole other host of stuff (additionally hundreds of lines plus more for tests).

While I can't say any of the code is bad, it is effectively like importing a library which your team now owns, but worse in that no one really understands it or supports it.

Lastly, I could find libraries that are well supported, documented, and active for each of these use-cases fairly quickly.

Yes, for leftpad-like libraries it's fine, but does your URL or email validation function really handle all valid and invalid cases correctly now and into the future, for example?

i've seen this fairly often with internal libraries as well - a recent AI-assisted PR i reviewed included a complete reimplementation of our metrics collector interface.

suspect this happened because the reimplementation contained a number of standard/expected methods that we didn't have in our existing interface (because we didn't need them), so it was considered 'different' enough. but none of the code actually used those methods (because we didn't need them), so all this PR did was add a few hundred lines of cognitive overhead.

> Too often people pull huge libraries for 50 lines of code.

I used to be one of those people. It just made sense to me when I was (I still am to some extent) more naïve than I am today. But then I also used to think "it makes sense for everyone to eat together at a community kitchen of some sort instead of cooking at home because it saves everyone time and money" but that's another tangent for another day. The reason I bring it up is I used to think if it is shared functionality and it is a small enough domain, there is no need for everyone to spend time to implement the same idea a hundred times. It will save time and effort if we pool it together into one repository of a small library.

Except reality is never that simple. Just like that community kitchen, if everyone decided to eat the same nutritious meal together, we would definitely save time and money but people don't like living in what is basically an open air prison.

I do think the discovery piece is hugely valuable. I’m fairly capable with grep and ag, but asking Claude where something is in my codebase is very handy.

Yes and this is the placid version where lazy programmers elect to lighten their cognitive load by farming out to AI.

An insidious version is AGI replacing human cognition.

To replace human thought is to replace a biological ability which progresses on evolutionary timescales - not a Moore's law approximate curve. The issue in your skull will quite literally be as useful as a cow's for solving problems... think about that.

Automating labor in the 20th century disrupts society and we've see its consequences. Replacing cognition entirely: driving, writing, decision making, and communication; yields far worse outcomes than transitioning the population from food production to knowledge work.

If not our bodies and not our minds, then what do we have? (Note: Altman's universal basic income ought to trip every dystopian alarm bell).

Whether adopted passivity or foisted actively - cognition is what makes us human. Let's not let Claude Code be the nexus for something worse.

The author of the article had an interesting solution to this. Flip a coin to see who implements the feature.

Heads you code. Tails you review.

Maybe if you have a business where the need for software is a constant, so it’s great to get it for 90% off. (It’s not clear what business that is in 2025, maybe a small plumbing contractor?)

But if your business is making software it’s hard to argue you only need a constant amount of software. I’ve certainly never worked at a software company where the to-do list was constant or shrinking!

At the end of the day economy is king. Nothing else matters. But economy is not something you can plan and predict (shout out to my communist friends), it is a chaotic system full of emergent elements, chance-based actors and third-order effects and so it usually takes years for trends and patterns to emerge. All I'm going to say is that unless AI keeps improving exponentially (and there's definitely an argument to be made that it's not already) there is going to be a hell to pay a few years down the road.

I use Grok, Claude and Gemini every day, these "tools" are very useful to me (in the sense of how google and wikipedia changed the game) and I watch the LLM space closely, but what I'm seeing in terms of relative improvement is far removed from all the promises of the CEOs of these companies... Like, Grok 4 was supposed to be "close to AGI" but compared to Grok 3 it's just a small incremental improvement and the same goes for others...

Or CEOs.

I disagree that layoffs are because of AI-mediated productivity improvements.

The layoffs are primarily due to over-hiring during the pandemic and even earlier during the zero-interest-rate period.

AI is used as a convenient excuse to execute layoffs without appearing in a bad position to the eyes of investors. Whether any code is actually generated by AI or not is irrelevant (and since it’s hard to tell either way, nobody will be able to prove anything and the narrative will keep being adjusted as necessary).

Bootstrapping is a lot easier when you have your family's or someone else's money to start a business and then fall back on if it doesn't pan out.

The reason people take jobs comes down to economics, not "creativity".

The reason there were so many layoffs is because cheap money dried up.

Nothing to do with AI.

Interest rates are still relatively high.

The point was that "programming" is far more than just "writing code".

isn't this likely to be the case when a field is developing quickly and there are a large number of people who have different opinions on the subject?

e.g. I liked GitHub Copilot but didn't find it to be a game changer. I tried Cursor this year and started to see how AI can be today.

Indeed. The LLMs have been pretty useful for greenfield projects & one off scripts for a while, but GPT-5 was the first time I've found a model to be quite helpful on large-scale legacy code (>1M LOC).

> was probably faster

That sure doesn't sound like 10x

> That was 5 months ago, which is 6 years in 10x time.

That's some pretty bad math.

But yes, it isn't making software get made 10x faster. Feel free to blow that straw man down (or hype influencer, same thing.)

I have linked my github above. I don't know how that fares in the bigger scope of things, but I went from 0 opensource to hundreds of tools and frameworks and libraries. Putting a number on "productivity" makes no sense to me, I would have no idea what that means.

I generate between 10-100k lines of code per day these days. But is that a measure of productivity? Not really...

The author’s main point is that there hasn’t been an uptick in total code shipped, as you would expect if people are 10x-ing their productivity. Whether folks admit to using AI in their workflow is irrelevant.

>Do you also expect I mention that I used Intellisense and syntax highlighting too?

No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. But, I guess that's not a priority anymore if you're willing to sacrifice $2400 a year to Anthropic.

Not sure what you mean? This was a demo in a live session that took about 30 minutes, including ui ideation (see pngs). It’s a reasonably well featured app and the code is fairly minimal. I wouldn’t be able to write something like that in 30 minutes by hand.