Get the top HN stories in your inbox every day.
cletus
throwaway7783
I'm not sure I understand. "The solution is to have dummy versions of services and interfaces that have minimal correct behavior".
That's mocks in a nutshell. What other way would you use mocks?
cletus
Imagine you're testing a service to creates, queries and deletes users. A fake version of that service might just be a wrapper on a HashMap keyed by ID. It might have several fields like some personal info, a hashed password, an email address, whether you're verified and so on.
Imagine one of your tests is if the user deletes their account. What pattern of calls should it make? You don't really care other than the record being deleted (or marked as deleted, depending on retention policy) after you're done.
In the mock world you might mock out calls like deleteUserByID and make suer it's called.
In the fake world, you simply check that the user record is deleted (or marked as such) after the test. You don't really care about what sequence of calls made that happen.
That may sound trivial but it gets less trivial the more complex your example is. Imagine instead you want to clear out all users who are marked for deletion. If you think about the SQL for that you might do a DELETE ... WHERE call so your API call might look like that. But if the logic is more complicated? Where if there's a change where EU and NA users have different retention periods or logging requirements so they're suddenly handled differently?
In a mokcing world you would have to change all your expected mocks. In fact, implementing this change might require fixing a ton of tests you don't care about at all and aren't really being broken by the change regardless.
In a fake world, you're testing what the data looks like after you're done, not the specific steps it took to get there.
Now those are pretty simple examples because there's not much to do the arguments used and no return values to speak of. Your code might branch differently based on those values, which then changes what calls to expects and with what values.
You're testing implementation details in a really time-consuming yet brittle way.
throwaway7783
I am unsure I follow this. I'm generally mocking the things that are dependencies for the thing I'm really testing.
If the dependencies are proper interfaces, I don't care if it's a fake or a mock, as long as the interface is called with the correct parameters. Precisely because I don't want to test the implementation details. The assumption (correctly so) is that the interface provides a contract I can rely on.
In you example, the brittleness simply moves from mocks to data setup for the fake.
Jach
The general term I prefer is test double. See https://martinfowler.com/bliki/TestDouble.html for how one might distinguish dummies, fakes, stubs, spies, and mocks.
Of course getting overly pedantic leads to its own issues, much like the distinctions between types of tests.
At my last Java job I used to commonly say things like "mocks are a smell", and avoided Mockito like GP, though it was occasionally useful. PowerMock was also sometimes used because it lets you get into the innards of anything without changing any code, but much more rarely. Ideally you don't need a test double at all.
phanimahesh
There are different kinds of mocks.
Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by.
The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build.
Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.
OrangeMusic
The second way is usually referring to as "fakes", which are not a type of mocks but a (better) alternative to mocks.
throwaway7783
Why is check if XYZ is called with return value ABC bad, as long as XYZ is an interface method?
Why is a minimally correct fake any better than a mock in this context?
Mocks are not really about order of calls unless you are talking about different return values on different invocations. A fake simply moves the cheese to setting up data correctly, as your tests and logic change.
Not a huge difference either way.
ahepp
Mocking is testing how an interface is used, rather than testing an implementation. That's why it requires some kind of library support. Otherwise you'd just on the hook for providing your own simple implementations of your dependencies.
yearolinuxdsktp
Heavy mocks usage comes from dogmatically following the flawed “most tests should be unit tests” prescription of the “testing pyramid,” as well as a strict adherence to not testing more than one class at a time. This necessitates heavy mocking, which is fragile, terrible to refactor, leads to lots of low-value tests. Sadly, AI these days will generate tons of those unit tests in the hands of those who don’t know better. All in all leading to the same false sense of security and killing development speed.
fatso83
I get what you are saying, but you can have your cake and eat it too. Fast, comprehensive tests that cover most of your codebase. Test through the domain, employ Fakes at the boundaries.
https://asgaut.com/use-of-fakes-for-domain-driven-design-and...
LgWoodenBadger
“The solution is to have dummy versions of services and interfaces that have minimal correct behavior”
If you aren’t doing this with mocks then you’re doing mocks wrong.
bccdee
Martin Fowler draws a useful distinction between mocks, fakes, and stubs¹. Fakes contain some amount of internal logic, e.g. a remote key-value store can be faked with a hashmap. Stubs are a bit dumber—they have no internal logic & just return pre-defined values. Mocks, though, are rigged to assert that certain calls were made with certain parameters. You write something like `myMock.Expect("sum").Args(1, 2).Returns(3)`, and then when you call `myMock.AssertExpectations()`, the test fails unless you called `myMock.sum(1, 2)` somewhere.
People often use the word "mock" to describe all of these things interchangeably², and mocking frameworks can be useful for writing stubs or fakes. However, I think it's important to distinguish between them, because tests that use mocks (as distinct from stubs and fakes) are tightly coupled to implementation, which makes them very fragile. Stubs are fine, and fakes are fine when stubs aren't enough, but mocks are just a bad idea.
[1]: https://martinfowler.com/articles/mocksArentStubs.html
[2]: The generic term Fowler prefers is "test double."
cowsandmilk
In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production.
Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.
sfn42
I use test containers and similar methods to test against a "real" db, but I also use mocks. For example to mock the response of a third party api, can't very well spin that up in a test container. Nother example is simply time stamps. Can't really test time related stuff without mocking a timestamp provider.
It is a hassle a lot of the time, but I see it as a necessary evil.
pdpi
I'd go a bit farther — "mock" is basically the name for those dummy versions.
That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.
rgoulter
> there is a massive difference between writing mocks and using a mocking library like Mockito
How to reconcile the differences in this discussion?
The comment at the root of the thread said "my experience with mocks is they were over-specified and lead to fragile services, even for fresh codebases. Using a 'fake' version of the service is better". The reply then said "if mocking doesn't provide a fake, it's not 'mocking'".
I'm wary of blanket sentiments like "if you ended up with a bad result, you weren't mocking". -- Is it the case that libraries like mockito are mostly used badly, but that correct use of them provides a good way of implementing robust 'fake services'?
saghm
I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.
jchw
The difference, IMO, between a mock and a proper "test" implementation is that traditionally a mock only exists to test interface boundaries, and the "implementation" is meant to be as much of a noop as possible. That's why the default behavior of almost any "automock" is to implement an interface by doing nothing and returning nothing (or perhaps default-initialized values) and provide tools for just tacking assertions onto it. If it was a proper implementation that just happened to be in-memory, it wouldn't really be a "mock", in my opinion.
For example, let's say you want to test that some handler is properly adding data to a cache. IMO the traditional mock approach that is supported by mocking libraries is to go take your RedisCache implementation and create a dummy that does nothing, then add assertions that say, the `set` method gets called with some set of arguments. You can add return values to the mock too, but I think this is mainly meant to be in service of just making the code run and not actually implementing anything.
Meanwhile, you could always make a minimal "test" implementation (I think these are sometimes called "fakes", traditionally, though I think this nomenclature is even more confusing) of your Cache interface that actually does behave like an in-memory cache, then your test could assert as to its contents. Doing this doesn't require a "mocking" library, and in this case, what you're making is not really a "mock" - it is, in fact, a full implementation of the interface, that you could use outside of tests (e.g. in a development server.) I think this can be a pretty good middle ground in some scenarios, especially since it plays along well with in-process tools like fake clocks/timers in languages like Go and JavaScript.
Despite the pitfalls, I mostly prefer to just use the actual implementations where possible, and for this I like testcontainers. Most webserver projects I write/work on naturally require a container runtime for development for other reasons, and testcontainers is glue that can use that existing container runtime setup (be it Docker or Podman) to pretty rapidly bootstrap test or dev service dependencies on-demand. With a little bit of manual effort, you can make it so that your normal test runner (e.g. `go test ./...`) can run tests normally, and automatically skip anything that requires a real service dependency in the event that there is no Docker socket available. (Though obviously, in a real setup, you'd also want a way to force the tests to be enabled, so that you can hopefully avoid an oopsie where CI isn't actually running your tests due to a regression.)
zem
my time at google likewise led me to the conclusion that fakes were better than mocks in pretty much every case (though I was working in c++ and python, not java).
edit: of course google was an unusual case because you had access to all the source code. I daresay there are cases where only a mock will work because you can't satisfy type signatures with a fake.
ebiester
Mockito, in every case I had to use it, was a last resort because a third party library didnt lend itself to mocking, or you were bringing legacy code under test and using it long enough to refactor it out.
It should never be the first tool. But when you need it, it’s very useful.
rurban
I dont use dummy services and I dont use mocking. I'm writing simulators to test things for HW or big services which are not available for testing.
Simulators need to be complete for their use cases or they cannot be used for testing.
t-writescode
Wow, there's a lot of anger in some of these posts.
I've been using Mockito for about 4 years, all in Kotlin. I always found it to be "plenty good" for like 99% of the cases I needed it; and things more complicated or confusing or messy were usually my fault (poor separation of concerns, etc).
I regularly found it quite helpful in both its spy() and mock() functionality.
I never found it meaningfully more or less useful than MockK, though I have heard MockK is the "one that's better for Kotlin". It's mostly just vocabulary changes for me, the user.
I'm going to have to monitor Mockito's future and see if I'll need to swap to MockK at some point if Mockito becomes unmaintained.
gerdesj
"Wow, there's a lot of anger in some of these posts."
If I was OP I'd retire happy knowing that a very thankless job is well done! Given what it does: the more outrage the better. Projects like Mockito call out the lazy and indolent for whom they are and the hissing and spitting in return can simply be laughed at.
10 years this bloke has given his time and effort to help people. He states: nearly a third of his life.
I'll raise a glass and say "was hale" or perhaps wassail as an Englander might.
WD-42
Is it anger? The agent thing maybe. The other two points seem to boil down to:
1. Kotlin is a hack and 2. Rust is more fun.
Pretty understandable why one would simply want to move on to greener pastures.
vips7L
I'm not sure Kotlin is a hack, but it's growth is directly related to Java not being able to deliver on simple features that would help the day to day lives of developers. So I also can't blame anyone for moving on from the language. I get frustrated with it all the time, especially when Kotlin shows that some feature could work perfectly fine.
Here are some examples that have hit the graveyard: It's been 2 years since exception handling in switch was proposed [0], 3 years since null-restricted types were proposed [1], 4 years since string templates [2], 8 years since concise method bodies [3], and 11 years for JSON parsing [4].
[0] https://inside.java/2023/12/15/switch-case-effect/
[1] https://openjdk.org/jeps/8303099
[2] https://openjdk.org/jeps/430
gf000
Sure there are pain points in Java, but I definitely wouldn't want the language to introduce 15 new features each having only a tiny benefit.
Based on your other comment you prefer "fat" languages like kotlin and C# - and that's fair. I find languages with less, but more powerful features far more elegant and while Java has its fair share of historic warts, modern additions are made in a very smart way.
E.g. switch expressions seamlessly support sum and product types, meanwhile kotlin's `when` really is just syntactic sugar.
All in all, with too many features you have to support and understand the complete matrix of their interactions as well, and that gets complicated quickly. And Kotlin is growing towards that.
MBCook
Why did Kotlin prevent those features?
Larrikin
Kotlin is a thoughtful language that is a joy to write.
Java was stagnant and ripe for being kicked off the top.
Scala was the hack that showed why Java was starting to suck but Scala suffered from no direction. Every single idea a PhD ever had was implemented in the language with no thought other than it seems cool to be able to do that too. Which is why all the Scala codebases fell apart, because you could write anything, anyway you want, and it was impossible to maintain without extremely strict guidelines on what parts of the language you were allowed to use. Also the build times were atrocious.
Kotlin designers evaluated language features across the ecosystem and chose what made sense to make Java better when it came out and now has easily surpassed Java. They are still thoughtful in what they choose to add to the language and it is now very powerful. However they smartly imitate Python by having clear guidelines on what choices you should be making when writing code so that other developers can easily dive in.
lisbbb
I liked Groovy, but Kotlin was the better successor. Scala was beyond ridiculous as far as learning curve--crazy language. I always thought Java sucked and I spent far too much time working with Java (because money) and having to deal with framework on top of framework as well as all those idiotic design patterns that seemed so important only for me to realize years later that it was all basically a waste because OOP itself is a waste. I was writing better code in Python and Go five years ago without needing those all-important design patterns (okay, a few still translate). I just didn't know it at the time how horrible the situation was in Java. In hindsight, everything about Java was someone's money grab that I was dealing with. Mockito did help me a lot through those times, but I wish I had never gone through them at all!
krzyk
> Scala was the hack
> Kotlin designers evaluated language
Well, I like scala, and most of the time I understand it, but I can't say the same with Kotlin. I would say quite the opposite is true.
gf000
> Scala was the hack
Well, I absolutely disagree with this take. Scala actually builds on top of a couple of its powerful primitives. Especially Scala 3 is a beautiful language.
jeremyjh
In retrospect - from an outside point of view - it seems they should have just declined to support Kotlin. There are other options for Kotlin, and it sounds like it makes sense for it to have something developed for that from the beginning. But probably they had no idea what they were getting into when it was first proposed.
emodendroket
At the end of the day I think there's nothing wrong with the tool itself. The problem is that mocking and spies make it easy to not bother properly isolating the effects of a function for testing and then you end up having a test where 95% of it is setting up an elaborate array of mocks to create the condition you wish to test, which are completely incomprehensible to the next person trying to read it.
lisbbb
I used it mainly because the code I inherited was untestable as written. I made it testable via those methods. Then it got refactored.
emodendroket
Well sure you sometimes haven’t got much choice. But I’m talking about people write new code this way
exabrial
Mock is effective when developers keep applications at 4-5 layers deep (and honestly, you don't need more than that 97% of the time: initiators, controllers, services, transports, and cross cutting concerns).
The problem is, engineers love to solve problems, and the funnest types of problems are hypothetical ones!
Yep, I was guiltily of taking DI and writing a spider web of code. It's not the tools fault, it was my attitude. I _wanted_ something hard to work on, so I created something hard to work on. Nowadays, my team and I work on a 4-5 layer deep limit and our code base is tight, consistent, and has near 99% test coverage naturally. We use mock testing for testing the single class, and of course integration test for testing requirements. Not everyone will do it this way, and that's fine, but the most important thing is actually just creating a plan and sticking to it so as to be consistent.
In the end, don't blame the tool, when you (or your coworkers) simply lack discipline.
senko
For those, like me, who haven't heard of it: Mockito is the "most popular mocking framework for Java".
ronnier
It’s taken years off of my life dealing with the test mess people have made with it.
nsxwolf
Absolutely the worst. 1000 line test setups that shatter into pieces the instant you try to make the simplest change to a function. Makes refactoring an absolute nightmare.
pjc50
What's specifically bad about Mockito here? Poor defaults for mocks?
mangodrunk
Which is absurd that people use mocks considering the tests are supposed to help with refactoring but because of the mocks they can’t make a change without breaking the test.
the_arun
I wish there are tests written to cover functionalities (& race conditions) instead of covering lines/branches.
schumpeter
It also translates to “small booger”, in Spanish, which always made me question who thought the name was a good idea over there.
Freak_NL
Why? Every name you pick is likely to be weird in one language or another. Mockito does one thing well as a name, and that is hinting strongly at what it is (a mocking library).
kace91
>is likely to be weird in one language or another.
But this name is weird in the specific language it’s imitating (both the -ito termination for diminutives and the drink on which I assumed the name is based are Spanish).
lisbbb
Wasn't it a riff on "Mojito" which was a popular drink at the time?
marinesebastian
Actually, no. "small booger" would be _moquito_ in spanish.
awesome_dude
Look, as an English only speaker I don't care - I'm still stuck at "Haw haw, small booger library!"
schumpeter
Fair. The spelling is off, but the pronunciation is the same.
bhawks
| My personal take is that folks involved with the change severely underestimated the societal impact that it had. The fact that proper build support is non-existent to this day shows that agents are not a priority. That's okay if it isn't a priority, but when it was communicated with Mockito I perceived it as "Mockito is holding the JVM ecosystem back by using dynamic attachment, please switch immediately and figure it out on your own".
Id like to hear the platform team's perspective on this. As it stands, it is a pretty sad state of affairs that such a prominent library in the ecosystem was made out to be the scapegoat for the adoption of a platform change. It is not a healthy thing to treat the library maintainer community like this.
clanky
The JDK team has advertised these changes for years. People who want to do wild & crazy dynamic stuff in their testing infrastructure still can, with a tiny amount of added setup. This is the correct tradeoff, vs having the JVM pessimistically unable to apply all sorts of optimizations because they never know when their users might have inadvertently opted into runtime shenanigans.
1a527dd5
Running open source _anything_ looks exhausting from the outside.
I don't know why owners/maintainers show grace like this.
For me; I would say I'm done and hand it over or just archive the repo.
Hope Tim finds a measure of sanity after he steps down.
hcfman
I think showing grace was the right way to do things. It preserves the value of all the effort he put into it. When you work on things that long they are part of you. Like your baby, you don't toss them aside carelessly, it's painful to do that I'm sure.
Well done for Tim for being one of those guys that worked so hard for others to benefit. Tim, you've done something you can be deeply proud of forever, no matter how it ended.
dingi
A lot of OSS burnout comes from a broken assumption: that publishing code creates an obligation.
Historically, open source meant "here's code, use it if it helps, fix it if it breaks." No support contracts, no timelines, no moral duty. GitHub-era norms quietly inverted that into unpaid service work, with entitlement enforced socially ("be nice", "maintainers owe users").
Intrinsic motivation is the only sustainable fuel here. Once you start optimizing for users, stars, adoption, or goodwill, pressure accumulates and burnout is inevitable. When you build purely because the work itself is satisfying, stopping is always allowed, and that's what keeps projects healthy.
Hard boundaries aren't hostility; they're corrective. Fewer projects would exist if more maintainers adopted them, but the ones that remain would be stronger, and companies would be forced to fund or own their forks honestly.
Open source doesn't need more friendliness. It needs less obligation
krackers
>Mockito 5 shipped a breaking change where its main artifact is now an agent. That's because starting JVM 22, the previous so-called "dynamic attachment of agents" is put behind a flag
Wouldn't this hold back enterprise adoption, the same way breaking changes meant that Java 8 was widely used for a long time?
philipwhiuk
JVM 22 stuff isn't going to be mainline at enterprise for years. Java 25 is the first LTS release that includes it and it's only just been released.
Most places are just about getting rid of 8 for 17.
lisbbb
I know!!!!!! I've been yelling about this for years how impossible to uplift JVM versions is because of how products were built! It's so crazy. Lots of obfuscated jars went unsupported after like Java 7.
nitwit005
It just means your test runs need to add the flag.
undefined
exabrial
TimvdLippe: You've done an incredible job. You have incredible ideas and vision. Just wanted to say thank you.
lisbbb
Hey, fwiw, thank you for all of your hard work! Mockito and Powermock helped me be a big hero at one company I worked for 2011-2014. Before I arrived as a tech lead there, they had ZERO tests. The QA cycle was weeks, even months long. I instituted a whole new regime of unit and intergration testing for all the applications I was responsible for. Within one release cycle, the bug counts fell to nearly zero and the QA cycle basically became a verification step because there were no more bugs. The only way I was able to pull that off was using mocking and some clever powermock hacks because the code was otherwise untestable and thus un-refactorable. So, thanks!
EdwardDiego
Pretty sure Powermock is dead in the water now, it did some pretty gnarly things (like patching `Object`) that flat out didn't work in later versions of Java.
I (ironically enough) spent some time replacing usages of it in Kafka test code with Mockito because of this, IIRC there was only one situation where Mockito couldn't easily replace Powermock and I'm pretty sure it was mocking out a private static method, so the solution was to refactor the code under test so that you didn't need to mock a private static method to test it.
dave_sid
Mockito is fine if you know how to write tests. You can write bad tests with Mockito or many other frameworks if that’s what you do. You can even write bad tests while programming in Rust from high up in your ivory tower. You can write good tests also.
mbfg
IMO mockito has a relatively good use experience. If you use MockitoExtension, especially, you write code that is relatively maintainable, and easy to mutate. The problem without MockitoExtension is you can throw all kinds of junk in there and it just sits there doing nothing, without you knowing it.
Spy's on the other hand, are a pain in the neck. I think they should be good in theory, but in practice they are difficult, the debuggers really don't work correctly, setting breakpoints, stepping, etc, is just broken in many cases. Would love to know if this is just a difficult bug, or something that is baked into spying.
Get the top HN stories in your inbox every day.
My second project at Google basically killed mocking for me and I've basically never done it since. Two things happened.
The first was that I worked on a rewrite of something (using GWT no less; it was more than a decade ago) and they decided to have a lot of test coverage and test requirements. That's fine but they way it was mandated and implemented, everybody just testing their service and DIed a bunch of mocks in.
The results were entirely predictable. The entire system was incredibly brittle and a service that existed for only 8 weeks behaved like legacy code. You could spend half a day fixing mocks in tests for a 30 minute change just because you switched backend services, changed the order of calls or just ended up calling a given service more times than expected. It was horrible and a complete waste of time.
Even the DI aspect of this was horrible because everything used Guice andd there wer emodules that installed modules that installed modules and modifying those to return mocks in a test environment was a massive effort that typically resulted in having a different environment (and injector) for test code vs production code so what are you actually testing?
The second was that about this time the Java engineers at the company went on a massive boondoggle to decide on whether to use (and mandate) EasyMock vs Mockito. This was additionally a waste of time. Regardless of the relative merits of either, there's really not that much difference. At no point is it worth completely changing your mocking framework in existing code. Who knows how many engineering man-yars were wasted on this.
Mocking encourages bad habits and a false sense of security. The solution is to have dummy versions of services and interfaces that have minimal correct behavior. So you might have a dummy Identity service that does simple lookups on an ID for permissions or metadata. If that's not what you're testing and you just need it to run a test, doing that with a mock is just wrong on so many levels.
I've basically never used mocks since, so much so that I find anyone who is strongly in favor of mocks or has strong opinions on mocking frameworks to be a huge red flag.