Ask HN: What to do about ‘Good at programming Bad at Leetcode’

I can't answer for the parent, but I've done similar style interviews. Because the candidate is not familiar with your codebase, you must be the expert. Your job is to let them drive as much as possible, while you "unstick" them when they trip over what they can't possibly know. It is a collaboration, which is part of what makes this a good interview style IMO. Sometimes a candidate will offer an otherwise good solution, but it won't work well within your codebase -- in this case you explain why, and then you iterate.

Yeah, usually we would give a little bit of context, and then look at the sprint tickets in Jira, and take one. The candidate was not expected to be familiar with the codebase or product at all beforehand, and they weren't expected to become an expert on it during the course of the interview. But coming up to speed quickly is one of the skills we liked to see, since that was part of the job.

They should be able to 'hang', in the sense of being familiar with how we approach problem solving as programmers. They should ask questions when needed, and answer questions as they arose. The goal was to just figure out what level they were at in general, and what it would be like to work with them. They weren't expected to come up with the best possible solution, because usually there wasn't a single right solution.

We would ask them questions like "well, you saw the ticket, how do you think we should approach this?" or even just natural things like "this function we just wrote isn't working, what's going on here?". The point is that they are working on real problems — literally real problems — and not abstract CS exercises.

Since we mostly did pair programming every day, the other thing you'd get a sense of is how easy someone is to work with. Not everybody had experience pairing, which is to be expected. You found out quickly who took to it, and who wanted to be a lone wolf. We didn't not hire someone because they weren't good at pairing, but we factored it in. Some people just did not communicate very well, which is a problem.

(This also gave them a good sense of what it would be like to work with us, and sometimes people decided not to take an offer because they didn't like the way we worked: good outcome for everybody)

They would usually do two rounds of this, both on the same day. Morning session, then an afternoon session. We offered to take them to lunch in between, which exposed them to more of the team, and let us talk to them in a more casual setting. Part of the interview would also be a private talk with one of the founders, and an HR person (in a different session).

The people the candidate paired with would provide input like: "I would recommend hiring this person" or not, but didn't make the decision themselves.

Here, I should admit that we did have a set of artificial problems we'd give candidates if we were hiring for a skillset that did not have a real project to test them on. We did both web and mobile contracts at this (consulting) company, so if we were hiring, e.g., an Android dev but didn't have an Android project for them to work on, we'd have them check out a repository and work on a fake ticket instead. Usually this was with a developer beside them, but I can remember times when the candidate had to do it solo.

> The folks who don’t do this are trying to automate hiring.

Or they are trying to emulate Google and similar companies without understanding why they do it.

Sure, great for metrics, terrible for actually doing the job.

Leetcode can never compete with or allow a candidate to demonstrate real multi-dimensional or multi-systems thinking. You may be able to write some bass ackwards arcane algorithm and prove you’re “31337” but can you make that work at scale with a data store that you have no information about? How’s it going to get the data to operate on in the first place? Will your data retrieval query cripple a prod database? Read from replica? How far behind can that data be without making your fancy algorithm useless? How will you parallelize that work set? In-request or background job? Multi-threaded or single thread? What about changing state in multi threaded contexts? Database indexes? Schema design? How will you handle failures in your algorithm? How will you handle malformed data? And most importantly, does what you’re doing benefit the user, or just your epeen?

Leetcode is bullshit. It’ll never let somebody show you they’re good at anything other than solving bullshit problems. And you’ll lose everybody who CAN work within the larger context by chasing candidates who’s chief skill is solving bullshit problems.

Agreed, but I see the impulse to standardize as the source of a lot of the problems people have — by people, I mean both good engineers, and the companies who can't hire them.

sure, let's spend time solving Leetcode problems instead of understanding how HTTP works.

> but coding them from scratch, from memory, within a fixed time? Pretty much never

This is definitely true, but having to do it quickly and on-the-spot is one way of measuring just how comfortable you are with it. You might never have to do it in an hour, but being able to is a proxy, however weak and indirect, for your competence, and guess what, I only have an hour to talk to you.

I love and hate leetcode interviews. I enjoy the challenge, and I hate being put on the spot and knowing that poor performance on one specific problem might result in an unwarranted thumbs down. I sincerely hope any interviewer whose chair I'm sitting in treats them the way I do, which is that it isn't important that you come up with the optimal solution; it's important that you can think through the problem and come up with any solution, however inefficient, and then honestly evaluate this solution (including big O) and point out how it could be improved. You probably don't have time to implement that improvement, but you've demonstrated competence in both thinking and coding along the way.

Edit to add: When I'm preparing to interview, I practice leetcode-style problems, but not so much to memorize all the questions and their answers as much to just practice my quickness for thinking and coding on that type of thing. It feels kinda silly because, as you point out, that's pretty much not a part of the job at all, but it's very very helpful to exercise your mental muscles for that sort of thing so you don't stumble over the simple mechanics of the problem. It feels absolutely great when you get a question you've never heard before but you've put yourself in the mental state to solve it and you kill it. And like it or not, I've personally found that people who enjoy this type of challenge are often the exact types of coders I want to work with.

Not OP but here are a couple reasons:

* It doesn’t scale well to huge organisations.

* It’s not standardised “repeatable” in the sense that you cant generate datasets from this that let you correlate interviews with subsequent outcomes and improve hiring metrics company wide

* It requires expertise in pair programming, which most companies don’t have

Basically, OP’s hiring process is an excellent example of “doing things that don’t scale” to gain an asymmetric advantage as a small organisation.

They don’t need to derive metrics, they can invest the time, they have a team who are skilled at pair programming - so they can have this hiring process and other people can’t.

> They're looking for people who will devote their life to the company and can quickly learn new skills

This doesn't reflect my lived experience in SV. People I know who learned leetcode skills and got into big companies usually work less than people who passed more practical tests and get into smaller companies.

I'd say big companies are trying to hire employees who are a "good" mix of:

1. Smart

2. Conscientious / willing to work hard on the right things

3. Existing CS knowledge you know well enough to explain and apply.

For some vague handwavy definition of "good"

(2) is probably worth expanding a bit. Many people are willing to work very hard on the wrong thing, this extends to engineering. As an example, a common failure pattern you might see is someone constantly struggling with how React works and what they really need to do is sit down and read the ~15 pages of documentation. But they never do, and just keep putting in 10 hour days with subpar output.

I've met some legit geniuses (think Putnam winner) for whom basically no studying was required to pass these interviews. Companies paying top dollar are happy to have them. For people like who are less smart and need to dedicate ~100-200 hours of focused studying and practice, companies paying top dollar are happy to take our mix of smarts and willingness to do that work. But once in the company I haven't noticed any expectation to "devote my life" to it.

> an quickly learn new skills. That's exactly what leetcode tests for

I don’t know where people get this idea. Leetcode used to test programming proficiency

I feel the only reason companies continue to take leetcode to absurd levels is because (a) they think harder questions would get them better engineers and (b) other companies are doing it

> Really the answer is either commit to the grind or go for a job at a non-tech company.

There is quite a world outside these mammoths Google/Facebook/Amazon/(MS, depending on how their IBMization is going these days) and their minions/wannabes. Personally I would not work for any of these even if they paid me in gold bars.

Yup exactly, that’s the whole thing. They want people who will devote their lives to meaningless tasks

> devote their life

Sounds like they are trying to breed a cult...

I appreciate the cynicism but isn't it more likely that they use it just because everybody uses it?

Exactly! 10 years ago, I was a strong proponent of leetcode style questions instead of using college GPAs to hire engineers.

It’s the ‘Cracking’ idea that has thrown this whole process off. People have started over optimising for the metric

> All companies paying over a certain threshold seem to be taking this to ridiculous levels.

They can afford it. And assuming they are a tech company, they need that in key roles. You cannot write things that scale without solid knowledge and intuition in designing and analyzing algorithms, something that can be learned and furthered through experience.

Now of course it's true that not every job in a larger tech company is like that, so there would be some room to adjust the interview based on the exact role. And there actually is. But overall, if you have a massive amount of people applying every week, why take chances, why not take someone from the pool who can demonstrate the skill and allow for some horizontal movement within the company, into a tech-heavier team?

> And please do share your ideas for what people should do or what finally convinced you that you weren’t bad at this

For me, back then, it was something that was fun for me. Part of why I got into computer science in the first place. I would be totally lying if I said I didn't study for the most relevant of my interviews, beyond what I would have done without them.

But I distinctly remember for example freshing up some graph algorithm skills not by doing leetcode examples (I don't think "leetcode" was already a thing back then), but by arbitrarily taking the Debian package database and do fun self-made exercises like: Let's find the largest strongly connected component in the debian package library, i.e. the largest set of packages where installing one will always install all of the others.

> Leet code is very contrived. However, learning leet code, then applying it in an interview shows you have the aptitude to learn quickly.

I feel very strongly that this premise is flawed. It shows that you have aptitude for a very specific type of learning.

I lead an engineering org at a small startup. We teach our engineers to dive deeply into the customer value prop to find ways to deliver the simplest technical solution to their problem. You don't need leetcode skills when you can identify simpler, more accessible solutions to a customer's need.

The reality is outside of very specialized technical businesses (or business components), most engineers jobs are not to build technically advanced solutions.

This argument can be used to justify forcing candidates to learn how to do double-entry bookkeeping (or insert other skill that you won't use on the job) just to see if they can learn. Yes, it checks if they can learn, yes, it is somewhat related to the area, but maybe there's a better way?

I've been 10 years in the industry. If recruiter tells me I need to prepare for an interview in ways other than looking back at the projects I've done, then the interview is not testing for what is important on the job.

> However, learning leet code, then applying it in an interview shows you have the aptitude to learn quickly.

That’s not why they were introduced. Leetcode using tools like hackerrank were introduced because college GPAs were very bad indicators of programming proficiency and a lot of people also did not have a CS degree.

The questions weren’t this bad before.

And these competitive programmers aren’t necessarily fast at learning things either.

It also shows you have the time to prepare. Not everyone has that luxury - think working parents with some other challenging happenings thrown in and you have an advantaged and disadvantaged class of people for these interviews

The algorithm test is standardized. If you make up your own weird test, candidates won’t play ball. Personally I’ve turned down companies that had a strange interview process that required extra preparation. It’s not worth it when I’m trying to churn through a dozen interviews (without my current employer noticing). The leetcode prep process is unpleasant, but since I have to do it anyway, adding one more company doesn’t take any extra studying.

Much better advice than “get used to ‘em” as alternatives to the status quo do in fact exist.

If you learn all this and you can't solve LFU cache in 45 minutes, go back and study the parts about heapsort, red-black trees, and B trees.

If you learn all this and you can't solve "integer to words", go back and do some more practical programming. Build your own blog in Django or something.

If you learn all this and you can't solve "next permutation", well, yeah sometimes interviews ask you dumb theoretical stuff that you will never use in real life. Them's the breaks.

I've started ignoring companies that hire using leetcode or whiteboard algorithm questions.

Even if you make it through the interview process, you're going to be working for people who cargo cult their business practices and surrounded by coworkers who are willing to spend vast amounts of energy on stupid things to accomplish their goals.

It's a massively negative indicator for company culture imo.

I can't remember the last time I actually enjoyed using Google's software or their libraries, I don't know why I'd work for a mini-Google.

No that's not true, the vast majority of companies/positions do not do this.

Most companies that pay very well and are competitive do it, and the reason is because we get so many crappy applicants that it remains to this day the least risky, cost effective way to filter out bad developers.

Blaming it on CTCI is like blaming it on needle manufacturers for people shooting up with heroin. The issue is more fundamental. CTCI is just giving a methodology for people to get through the interview process.

The issue is that SV cargo cults extremely hard. A lot of startups copy the big companies (and often the founders of small companies are from big companies - so they take what they know with them). Then everyone is like, "SV is so cool and full of money! Let's copy everything they do to emulate their success."

That's all it is. I'm not a fan of the author (for personal reasons) of CTCI but I don't think she or the existence of the book is the issue.

I’m not seeing everyone else doing it. I wish they were because I prefer those interviews over something that pretends to emulate real work. I want every company I interview with to do leetcode and most of them don’t.

Just went through a round of interviews in the past 6 months (~20 phone screens, 6 in person loops, 3 offers). Not all of the companies do it. If you don't like the process, then politely decline the interview. (you probably don't want to work there anyways).

The worse to me are the interviews that ask you technology specific commands (for ops positions, especially k8s) and/or specific language constructs.

Come on, there's enough, far more than enough, out there which is not like this.. just change your expectations.

[Disclaimer: Two-time Googler here, obviously speaking only for myself.]

Sure it does. Collaborative software development is a low-pass filter, not a high-pass one. If you put 100 people on a team and tell them to develop a product, the product quality will track the least controversial ideas, not the best. With more people, the set of uncontroversial ideas gets progressively smaller, and trends toward the features that nobody really cares about because those are the only ones that don't provoke strong emotional reactions. The simple fact that you added more people is what made the product shitty, not that the people were incompetent or poorly-skilled.

I've been in design discussions where we've had a room full of incredibly accomplished people, folks who've written major open-source projects or launched consumer products with billions of users, and the eventual decision was more brain-dead than anything that any one individual in that room could've come up with. And everybody knew it too, but you don't want to piss off highly intelligent and accomplished people, so you make the decision that everyone can kinda live with rather than the one that will wow users.

The lesson here should be "Don't hire unless you absolutely need to", not "Google hires stupid people."

> if that's the case, it doesn't explain why Google's software and product quality has been declining steadily.

Why do you assume that poor products are poor products chiefly because of code?

User opinion is one of many success metrics that Google tracks but it isn't the most important one. Up until the recent recession their stock performance indicated that their product quality has been doing just fine.

> it's the sense that you'll need at least 1-2 months of studying to get every subsequent job

I think it doesn't have to be this way if you focus on the fundamentals when you prepare, as it's easier to refresh a reasoning that clicked the first time than a complex algorithm you understood only superficially. And these fundamentals are enough for leetcode problems of medium difficulty.

2 months of studying every 2 years for job switch is 8.3% overhead. Do leetcode jobs pay 8.3% more after-tax than non-leetcode jobs?

> (since you won't be touching leetcode-like problems in the course of your actual job)

That is not universally true.

A lot of those people in India and China commit suicide, have mental breakdowns, or end up living generally unsatisfying lives. Their system is entirely unenviable.

There's a difference between 'entitlement' and recognizing that a system is flawed such that it doesn't necessarily select for the best candidates, only those who are willing to play the game hardest, which doesn't necessarily correlate with the best on the job performance.

The zero sum burnout games many Chinese and Indian students play are also a good example of such a system, and I think it's a positive feature of American culture that it's objected to.

OP here.

I’m in India. People spend 2-3 years grinding leetcode questions too. So you can imagine how difficult the tests have become over the past few years and how pointless this whole test is.

If the grinding is not going to translate into a business advantage, then it is pointless.

They're useful as a second piece of evidence if, like me, you did poorly at classwork because of undiagnosed ADHD.

I guess you just swear off ever doing an algorithms problem? This doesn’t seem comparable at all - there are feasible circumstances where you need to know this stuff.

> I’ve spent the last year or so reinventing the hiring process at my tech company to expressly avoid algorithmic puzzles and leetcode nonsense, and I was able to do this because I patently refused to accept the status quo.

I think companies that break out of the leetcode habit are going to do well. There's a lot of great talent out there that's being passed over by these tests.

If the job is to solve leetcode problems, by all means, test the candidates for that. If the job is to solve real life problems, there's almost certainly a better measure.

I will never judge someone based on that kind of problem. I believe the data that results is just bad.

Edit: I should add that leetcode is useful for developing programming and problem-solving skill. I've done tons of problems, myself, and have fun solving them... and learn from them. So I do recommend it for a challenge. But not for interviews.

Yeah, I've worked in tech 6 years now and never had to do Leetcode to get by in interviews. That being said, I never interviewed at FAANG. But many teams set a bar easier to pass than leetcode that still ensures good engineering skills

> now is a fantastic time for the tech industry to adopt change because your talent recruiters are probably struggling to find and retain new developers

Unfortunately, this will likely change much sooner and more rapidly then most of us expect. The music is stopping and chairs are about to be removed. It was great while it lasted, but all good things come to an end.

Would you be able to share the kinds of questions you ask and have had success with? :)

FWIW, happy you're doing this! A better process would make so many peoples' lives better.

The ONLY people who should "get used to 'em" are new/recent grads looking for a junior-ish IC job. For them, it's just a fact of life: they had to learn history & biology to get their CS degree; they have to learn leetcode to get a job.

Interviewers should make a better process, as you're doing.

So what'd you come up with?

For what it’s worth, it was like this in my interviews at Microsoft in the late 80s.

The point is that leetcode is an instance of the class interview_riddles; the class has persisted for a half century.

That bias is generally true for any education-based employment that requires a significant upfront investment.

Memorizing is meant in a different context.

Ultimately the problems boil down into a few "types". So you are memorizing the types.

https://dev.to/fahimulhaq/14-patterns-to-ace-any-coding-inte... https://neetcode.io/

I have used hackerrank/leetcode previously just sporadically to learn a new programming language. I used it for the first time to help me get another job earlier in the year.

However, I have continued to solve 1 problem a night because what I realised is after 15+ years in the industry, I don't have a good grasp of some concepts anymore.

I'm slowly getting better at identify types of problems and apply what could be the most efficient data structure/algorithm in solving the problem. Once I solve the problem I go have a look at the solutions to see a more efficient way of solving it and try to store it away in my memory banks for another day.

The thing I don't like is when interviewer asks follow ups such as "Can you do the same solution but using O(1) space complexity". Unless the job has a restriction on running on limited hardware resources. I feel like it's unjustified asking such follow up questions because most of the time that is a very specific solution for the problem - or maybe I'm just not good enough to come up with such a solution yet.

> anyone who can memorize the solutions to all those is some kind of genius savant and should be hired anyway

But that's the problem. Why should they be hired?

The actual job will not consist of regurgitating those memorized solutions.

I'm impressed by someone who can memorize 2000+ solutions, same as I'm impressed by the acrobats at Cirque du Soleil, but I don't want to hire from either group for a software engineering job because that's not what the job is about.