AI should elevate your thinking, not replace it

I think if manual coding becomes "outdated" then there will just be no demand for junior engineers to manually code. People will probably still learn to code manually, just as there are folks who will still build their own furniture. There may just not be a business demand for it.

What that means 20-30 years from now when the seniors of today retire if there are no juniors right now is yet to be seen. People say that AI will probably have advanced far enough that it won't be a problem. But let's say somehow AI stagnantes, then I would guess that AI-generated code that is too difficult to debug will be treated as legacy and there'll be demand for manual coding again.

Companies that aren't able to afford the rewrite or maintenance will probably go out of business.

It's an interesting time we live in for sure.

> Socrates wrote about what was being lost…

Dr. Steven Skultety & Dr. Gad Saad discussed this in a recent video / podcast.

This link is time stamped to the topic https://youtu.be/7mcQf9E3YRo?t=1058

I'd say that by purging stuff from the brain we are losing thinking itself. Thinking is manipulating ideas and concepts in your head, assembling and linking. The fewer things there is, the more primitive the result. You cannot juggle without object to juggle, connecting the dots result in trivial patterns when you have just a couple of dots.

Yeah but where comparison with philosophy falls short is - if we lost some ways of thinking, it was gradual and most didn't notice.

Software code is on the other hand extremely formal, and either it works perfectly as intended, it works crappily and keeps breaking in various edge cases or just doesn't work (last 2 are just variants of same dysfunctionality, technically its binary state). There is no scenario where broken code somehow ends up working and delivering, or maybe 1 in trillion, sometimes.

Also the change is so fast that the failure is immediately obvious to everybody, its not gradual change of thinking over few decades/generations.

LLMs are getting impressive, but anybody claiming there is no massive long term harm to getting to what we call now proper seniority is... don't know, delusional, junior who never walked that long and hard-won path, doing PR for llms at all costs or some other similar type. Or simply has some narrow use case working great for them long term which definitely can't be transferred on whole industry, like 1-man indie game dev.

Socrates was histories first Luddite. He opened Pandora’s box. I wish him and Plato would be radically rejected as the garbage trash they are (basically just a defense of hierarchy and dialects)

Quoting my boy Max Stirner who also fking hated these guys

“This war is opened by Socrates, and not until the dying day of the old world does it end in peace.“ - The Ego and its Own, Max Stirner

I'd argue that the engineers of 20 years ago were better than the engineers of today because they were significantly more resource constrained and for example, would never use a 300mb javascript library for a profile page.

Understanding something and learning something are not the same things.

>Generally it’s possible to do that in English much more tersely than it is in code.

I really doubt this is the case for any decent amount of complexity. English is ambiguous. Programming languages exist because we don't want this ambiguity from natural languages. A sufficiently descriptive spec written in English is barely indistinguishable from just code. We've come full circle.

> It's not by writing syntax that you get there.

Writing syntax is still an important part of the experience. It is valuable because it requires you to spend time immersed in the nuts and bolts that hold software together. I'd compare it to cooking, if you have an assistant or a machine do everything and you never actually touch a knife or stir a pot, you'll lose your touch. But there is also something valuable about covering more ground and the additional experience that brings.

Why are you focusing on syntax so much? There's more to that when writing code.

That's why students learn how to write pseudo-code before picking up a programming language. Learning how to think through implementing a solution to a problem is extremely important. It's exactly this experience that helps engineers grow their scope and understand bigger, more complex system.

There's also the tactical components of using programming languages. The only way to know when to use one type of data structure over another, or to debug tricky language-specific behavior is _to actually have used that language._

And it's exactly this knowledge that's being threatened by LLMs given how they are implemented today.

You can lead a horse to water, but you can't make it drink.

> We don't hire juniors and throw them boilerplate and tiny bugs while expecting them to learn along the way ad hoc through some pair programming and the occasional deep end.

Is that generally the case though? I'm about two years into my first job in the industry and that's exactly my experience, and certainly frustrating...

In fairness, 9 women can't make 9 babies in a month either

Hah, right, I mixed it up!

Put differently: you get good at what you actually do, not what you think you're doing.

If you're not coding anymore, but using AI tools, you're developing skills in using those AI tools, and your code abilities will atrophy unless exercised elsewhere.

I’ve also seen along those lines “there is no compression algorithm for experience” - a nice summary of the hn posts from today.

... or by textbooks, Stack Overflow, senior engineers, code review. How many engineers today got their start by building Minecraft mods or even MySpace?

I do think that these pieces sometimes smuggle in a nostalgic picture of how engineers "really" learn which has only ever been partly true.

More worrisome is that the speech which that came from went on to prophetically observe that for each extension of human capability afforded by technology, there was a matching amputation in human skill/facility --- heretofore, computers have largely fit in with Steve Jobs' vision of them as "bicycles of the mind", making human effort more efficient --- the cognitive engine of LLMs looks to be dumbing down human reasoning to a least common denominator/mean:

https://publichealthpolicyjournal.com/mit-study-finds-artifi...

You're right. ive struggled to understand what exactly this means, in large part perhaps it's so often misused?

I think it means something like we're trapped in the constraints of the medium. Tweets say more about the environment of twitter than whatever message happened to be sent.

but i think im off on that, ill look this person up and find out!

Could it be that what we called flow state was actually a sort of high level thinking time afforded by doing low level routine work?

For instance I'm the old world, if you wanted to change an interface, you might have to edit 5 or 6 files to add your new function in the implementations. This is pretty routine and you won't need to concentrate that much if you're used to it, so you can spend that low-effort time thinking about the bigger picture.

Its the "unsettling little ways", right. So you can't skip whole paragraphs, you literally have to read everything. And sometimes its worded in ways I don't understand at all (due to missing implications that the LLM conveniently omitted), so I have to re-ask it about that point as well. For every major feature or work-unit it takes up to 2 or 3 hours.

I figured out some patterns in the way it behaves and could put more guard-rails in place so they hopefully won't bite me in the future (spelled out decision trees with specific triggers, standing orders, etc.), but some I can't categorize right now.

When the end result has problems and needs to be reworked.

You can't figure this out instantly except when you'd review everything the LLM produces, which I am not. So the round trip time is pretty long, but I can trace it back to the intent now because I commit every architecture decision in an ADRs, which I pour most of my energy into. These are part of the repo.

Using these ADRs helped a lot because most of the assumptions of the LLM get surfaced early on, and you restrict the implementation leeway.

Do they? I haven't experienced models deviating from a spec in a very long time. If anything I feel they are being too conservative and have started to ask to confirm too much.

Thanks for sharing your process! It's helpful and refreshing to hear from someone about how they engage with AI when writing, and where / when the detection tools may fail.

(We obviously live in a more nuanced world than most social media interactions might make you think :P)

> On a lighter note, decades ago, in middle school, we had an exercise to summarize a book we read.

My first experience with plagiarism was in first grade, when we were told to write a book report about a subject during our library time. I diligently took my book on the musk ox and copied three pages word-for-word into my notebook as my report. I can't remember when or how we learned this wasn't "right", but I still think back on that and laugh.

Sorry but it's very obvious you used an LLM for more than just suggestions. Ironic given the point of the article.

Taking you at your word, your A.I. revision process nonetheless seems to have yielded content which may as well have been generated at the start for how difficult it is to get through it.

    The valuable engineer is the one who sees the hidden constraint before it causes an outage. The one who notices that the team is solving the wrong problem. The one who reduces a vague debate into crisp tradeoffs. The one who identifies the missing abstraction. The one who can debug reality, not just read code. The one who can create clarity where everyone else sees noise.

    "The software engineers who will be most valuable in the future are not the ones who do everything themselves. They are the ones who refuse to spend time on work that A.I. can do for them, while still understanding everything that is done on their behalf."

    "The danger is not that A.I. will make people lazy in some vague moral sense. It is that it makes it easy to simulate competence without building competence."

    "In that world, the engineer is not replaced by A.I. The engineer becomes more leveraged because they are operating above the level of raw output."

    "The ability to explain why something works, not just that it appears to work."

    "That process is not optional. It is how engineers acquire and elevate their competency."

    "The support system may make you look functional, but it does not make you capable."

    "The challenge is not merely adopting A.I. tools. It is protecting the conditions under which real thinking, learning, and craftsmanship continue to thrive."

    "They will need interview loops that test reasoning, not just polished answers."

    "The organizations that handle this well will not be the ones that simply push A.I. adoption hardest. They will be the ones that learn to separate leverage from dependency, acceleration from imitation, and genuine capability from convincing output."

Don't do it man. just stop. you loose something of yourself when you turn to AI

One of the many things that has been strange to me is how often people will label written thoughts as AI slop when the "signs" are just normal phrases. Sure, that's a tired expression, and I 100% agree we should be critical of writing that seems to embolden pointless trite expressions. But people have written in that way for years before LLMs.

I find it very interesting that we only now have more widespread discourse around the quality of prose and rhetoric now that LLMs have become ubiquitous.

Humanity's doom would be failing to recognize how amazing advancements in one area don't necessarily translate to another – not the opposite.

That's why philosophy (in the original sense of "striving for wisdom") is important. We're thinking animals after all, and can be more than dumb consumers, victims of our instincts.

The risk is the next generation, who will not have lived in a pre-AI world, won't know the difference to compare.

It feels like an addiction. Normal coding requires sustained attention, you can sense how deep you are in the progress and when you're too tired to continue, but with LLMs the next feature always feels like another prompt away, having sessions go well into the early morning/late-night. You rationalize you can quit, that you've been reading the source and each diff enough to "understand" the codebase. But the truth is when the rate limit runs out, you'll be absolutely helpless, crawling back for extra-usage, until you finally see the total bill at the end of the month.

I did the same. 2 was more of a curiosity, to see how quickly will it paint itself into corner. Maybe not there yet, but close enough that I consider taking over even for the side project.

It should probably be "would initially struggle to be as efficient without them."

This is not a binary.

> ... nobody in the field of "Software Engineering" is actually an Engineer if we go by a strict definitions.

This is a pet peeve of mine, so while I understand what you mean, I will challenge you to come up with a strict definition that excludes software engineering!

And since I've had this discussion before, I'll pre-emptively hazard a guess that the argument boils down to "rigor", and point out that a) economic feasibility is a key part of engineering, b) the level of rigor applied to any project is a function of economics, and c) the economics of software projects is a very wide range.

Put another way, statistically most devs work on projects where the blast radius of failure is some minor inconvenience to like, 5 users. We really don't need rigor there, so I can see where you're coming from. But on the other extreme like aviation software, an appropriately extreme level of rigor is applied.

Engineers are accredited in the US too. But there is an "industrial exemption" that allows you to work as an engineer without a license for certain kinds of employers. You just can't offer engineering services to the public without a license. This is more important in some fields than in others.

Where I work, there are plenty of non licensed engineers, but we pay a 3rd party agency for regulatory approval. The people who work for that agency are licensed engineers. Their expertise is knowing the regulations backwards and forwards.

Here's what I think is happening within industry. More and more work done by people with engineering job titles consists of organizing and arranging things, fitting things together, troubleshooting, dealing with vendors, etc. The reason is the complexity of products. As the number of "things" in a product increases by O(n), the number of relationships increases by O(n^2), so the majority of work has to do with relationships. A small fraction of engineers engages in traditional quantitative engineering. In my observation, the average age of those people is around 60, with a few in their 70s.

I started my career as a machine designer (mechanical engineering), designing some machines for FMCG factories.

It wasn't that much different from SWE - mostly looking up catalogs, connecting certain pre-made pieces together with custom parts and lots of testing of the final plan to make sure there are no collisions and every movement is constrained properly.

95% of the time no load or sizing calculations were necessary - we just oversized everything based on tacit knowledge (the greybeards reviewing the plans) since these machines were not mass produced and choosing somewhat bigger parts was not expensive given that these machines would operate and produce value 24/7 for years.

(I hope the analogy to software engineering is visible!)

What I'm saying is that the level of "engineering rigor" heavily depends on the field where engineers are operating within. Even certain SWE fields (healthcare, finance, aviation etc.) have more regulation and require more rigor than others.

as an actual engineer i just feel sad. i should probably feel happy but i like solving problems. fml i have becomea luddite.

i think you accidentally overlooked accredited engineers who happen to be writing software

The concept of engineer predates the accreditation systems you’re referring to by centuries.

Local AI models are already more than capable enough writing code that surpasses the ability of any bad or even mediocre engineer. That is not something we need to worry about.

In a way, this is less of a cost issue than the fact that some/many engineers do not seem to be willing or able to host things themselves anymore and will happily outsource every part of their stack to managed services, be it CDN, hosting, databases, etc. I don't know why that's not more alarming than the LLMs.

Self hosting at a reasonable scale is much cheaper than people think. I am running clusters of DGX Spark machines with BiFrost load balancers in our company and for client projects. They work flawlessly!

128 GB unified memory, Nvidia chip and ARM CPU for just around 3k€ net. They easily push ~400 input and ~100 output tokens per second per device on say gpt-oss-120b. With two devices in a cluster, thats enough performance for >20 concurrent RAG users or >3 "AI augmented" developers.

And they don't even pull that much power.

Slack, GitHub, Figma, AWS, etc

Lots of people use firebase, supabase etc.

Many people's jobs are centered around using Salesforce

It all makes me uncomfortable- I want to be able to work without internet. But it's getting more difficult to do it

> IDEs are free. Libraries are free. Languages are free. This is becoming more like an internet subscription where you’re at the mercy of Anthropic the same way you may be at the mercy of Comcast.

Ollama/llamafile/vllm/llama.cpp are free. Qwen/kimi/deepseek are free. Pi.dev/OpenCode are free. If you're using a SaaS AI subscription that's fine, but that's hardly the only option.

Not all IDEs are free. Not all LLMs are subscriptions.

You are right that there are a large number of top AI people who are very concerned with the ramifications of AI. I would say there are two core issues.

The first is that these people have often been indistinguishable from the ambitious and power hungry people I was decrying. Sam Altman was able to blend in for a long time by copying the rhetoric of AI safety types. When there is this much money to be made and power to be amassed, it's not hard to pretend to care.

The second is that I have often been disappointed by what the AI safety folks are concerned about. There has been a huge amount of talk about existential risk and not nearly as much about, say, the impact on children if education is outsourced to AI. The obsession with science fiction led to some very out there scenarios that may or may not still happen, but have nothing to do with the very real impacts of AI on people's lives right now. I believe that even the well intentioned have been too detached from humanity as a whole.

I guess I appreciate the explicit realpolitik of it all, but I'm not sure I buy your argument. The US is the world's current dominant global empire, and unlike other leftists, I don't believe this is inherently a bad thing. But I don't think it's necessarily a good thing either. It's just the reality.

I also think your whole unipolar vs. multipolar framework is ahistorical. It's always been more secure living within the cosmopolitan center of an empire, but there's never just been one empire.

I just think your view of history is too simplistic to be accurate or interesting. If/when the US declined as a global power, the results would be entirely unpredictable. They wouldn't adhere to the kind of formula you're describing.

This is an interesting perspective I haven't heard before. Do you have links to anyone who has articulated this further?

All this stuff is proving to me one thing: we don’t need to get better at producing stuff faster. We need to get better at producing the right stuff. Right stuff meaning getting better at project selection.

I guarantee a firm that is really good at project selection with hand written code will annihilate a firm that is full of agentic engineers. And frankly I hope the outcome is that all those who went all in on agentic coding go out of business because they got beaten by disciplined and visionary leaders who understand this subtle and nuanced point.

> I don't know how to do something so nobody else knows either

Ironically making the case for the thesis of the piece - what happens when you let A.I. do all the thinking instead of exercising competent judgement. Disclaimers and leaving real thought to others do not make it much better. Pangram is confidently wrong here.

> This requires logic, which LLMs lack.

What? I've heard many takes on what AI lacks, but never this one. We had ChatGPT being able to solve an Erdős problem on its own yesterday [0]; how could you explain that if it cannot do logic?

[0] https://news.ycombinator.com/item?id=47903126

Ever since the standard office setup went from offices or cubicles to bullpens and hot desks there is less and less time to think, and all of that is a management decision to ship things as fast as possible

Grade inflation and schools passing kids who should fail to game metrics and keep collecting student loans is a problem. I wouldnt consider hiring anybody from my alma mater who didnt score a sandard deviation or higher on the tests.

You don't need a 4.0 to graduate. And even if you got one, a lot of grades are composed of tests, not projects. You can just memorize your way through things if you were dedicated enough.

It's not really that hard to get a degree in engineering if your only goal is the degree itself.

Half of my graduating class could barely program.

I've seen it happen multiple times. Engineering degrees are no different than a vast majority of degrees in that if you are good at the read and regurgitate cycle, you can make it through. Not only can you make it through, but you can do it with a very respectable GPA. They come out with a large dictionary of keywords in their arsenal, but no idea how to put them into practice. Some are able to put it into practice and tie it all together. As they see practical examples of those keywords in the real world, it starts falling like dominoes, and at an accelerating rate. For some, it never goes much beyond keywords. The dominoes fall, but it is slow, and they stop falling for extended periods of time for them. Not many mature engineering organizations can tolerate that sort of progression rate. They usually don't last very long at any one place, until they find a company where they can blend into the background due to a combination of company culture, and low complexity systems being worked on.

The practice of software engineering is not what they teach in university.

I would say that today's graduates are IMO a bit better than a few decades ago but there are still many graduating who are just not good at writing computer software and don't really have the aptitude for that (or maybe the interest in getting good). That's what happens when the pipeline of people coming in are people who want to make money and the institution is mostly a degree factory.

OP should have put "engineers" in double quotes. Many software developers like to describe themselves as engineers although they don't have an actual engineering degree. A lot of software development resembles plumbing more than engineering, so most devs don't really need an engineering degree anyway, but they should be more honest about what they're actually doing and not try to elevate themselves with fancy titles.

You are, of course, right that the idea that someone could finish a serious engineering degree without being able to think is ridiculous.

I don't know but I can point at more than half of the people that I work with that can't think, and every time they try to, takes a whole group of people that can think to undo their mess, they all have degrees and I don't.

So what does that tell me?

Better yet, for about 30% having the LLM slop it would have yielded better outcomes, but having them slop something nets terrible slop. But at least I can reshape because even the LLM wont do something that stupid.

A degree is passing the test. Not all degree programs get into more advanced topics nor do they necessarily require that someone is able to work through how to solve a problem that they haven't seen before.

--

A lot of students (and developers out there too) are able to pass follow instructions and pass the test.

A smaller portion of them are able to divide up a task into the "this is what I need to do to accomplish that task".

Even fewer of them are able to work through the process of identifying the cause of a problem they haven't seen before and work through to figure out what the solution for that problem is.

--

... There are also a lot of people out there that aren't even able to fall into the first group without copying and pasting from another source. I've seen the "stack sort" at work https://xkcd.com/1185/ https://gkoberger.github.io/stacksort/ professionally. People copying and pasting from Stack Overflow (back in the day) without understanding what they're writing.

Now, they do it with AI. Take the contents of the Jira description, paste it into some text box, submit the new code as a PR, take the feedback from the PR and paste it back into the box and repeat that a few times. I've seen PRs with "you're absolutely correct, here are the updates you requested" be sent back to me for review again.

This is not a new thing. AI didn't cause it, but AI is exacerbating the issue with professional programming by having the people who are not much more than some meat between one text box and another (yes, I'm being a bit harsh there) and the people who need instructions but don't understand design to be more "productive" while overwhelming the more senior developers.

... And this also becomes a set of permanent training wheels on developers who might be able to learn more if they had to do it. That applies at all levels. One needs to practice without training wheels and learn from mistakes to get better.