The costs of microservices (2020)

This is why I prefer the old term "service oriented architecture" over "microservices". "Microservices" implies any time a service gets too big, you split it apart and introduce a network dependency even if it introduces more problems than it solves.

It's a pretty common issue. If you have 2-3 services, it's pretty easy to manage. And if you have 1000, you likely have the infra to manage them and get the full benefit.

But if you have 20 engineers and 60 services, you're likely in a world of pain. That's not microservices, it's distributed monolith and it's the one model that doesn't work (but everyone seems to do)

I don't want to get too tied up in the terminology, but "microservices-first" does not seem to be the problem the post is describing:

One way to mitigate the growing pains of a monolithic backend is to split it into a set of independently deployable services that communicate via APIs. The APIs decouple the services from each other by creating boundaries that are hard to violate, unlike the ones between components running in the same process

Exactly, just use well-factored services of any size and smack anyone saying "micro..." with a wet towel for they are just parroting some barely-profitable silicon valley money sinks.

For some reason, most of the people I've worked with recently are either fully into monoliths or lots of fine grained, interdependent microservices.

They don't seem to understand there's a useful middleground of adding fewer, larger data services, etc. It's like SOA isn't a hot topic so people aren't aware of it.

Actually, you are wrong. Microservices are surely not about defaulting to new microservices, but to capture a specific context into one service. There is no rule about how big a context is. A context can contain other context's. There can be technical reasons to split deployments into different microservices, but that's not the norm. What you describe is what happens, when people get microservices wrong.

In the end, i like the viewpoint that microservices are a deployment pattern, not so much an architecture pattern. Usually, you can draw a component diagram (containing an OrderService and a DeliveryService, etc.) and without technical details (execution environment, protocols), you couldn't tell if it's describing multiple microservices or multiple components in one service.

Exactly my thoughts

Yes, but the choice to add a new programming language to your company's profile has to be taken with care and due diligence; you should make sure to have a number of developers that know the new language, offer training, incorporate it into your hiring, etc. It's an added node to your dependency graph, which can quickly become unmanageable.

You should always look into existing languages first. There's a lot of "I rewrote this thing into $language for a 100x performance boost" posts, in a lot of cases the comments are like "Yeah but if you rewrote it in the original language you could make it a lot faster too".

For a cool visualisation of this problem, check out https://boringtechnology.club/

These are almost never pragmatic decisions. Giving teams independence over the stack usually results in resume-driven development, and now your JS developers are forced to maintain a Go server because some jock thought it was a cool thing to do .

Due diligence in these cases is rare.

Because of dependency issues like he mentioned. If I am using Library A which depends on version 1 of Library C and I need to start using Library B which depends on version 2 of Library C then I have a clear problem because most popular programming languages don't support referencing multiple different versions of the same library.

Those dependencies might cross language boundaries, or interfere with your other dependencies running in the same process.

More moving part. A network call can fail in more ways than a function call. Also something no one has mentioned until now, the more "services" you have the more of a pain in the arse it is to get a development environment running.

Still trying to unlearn that one. Turns out, most decisions are cheap to revert or backtrack on, while delaying them until Last Responsible Moment often ends in shooting past that moment.

That's technical, and not architectural. I'm _all about_ technical solutions to lack of discipline, and in fact I think technical and process solutions are the only immediate way to create cultural solutions (which are the long-term ones). I'd even consider minor increases to architectural complexity for that purpose justifiable - it's a real problem, and trading to solve it is reasonable.

But architectural complexity has outsized long-term cost, and service-orientation in particular has a _lot_ of it. And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined.

No, not at all. CI/CD blocking pull requests is in place because large systems have large test suites and challenging dependencies which mean that individual developers literally can't run every test on their local machine and can often break things without realising it. It's not about organisational discipline, it's about ensuring correctness.

The other problem is that these self-imposed roadblocks are so engrained in the modern SDLC that developers literally cannot imagine a world where they do not exist. I got _reamed_ by some "senior" engineers for merging a small PR without an approval recently. And we're not some megacorp, we're a 12 person engineering startup! We can make our own rules! We don't even have any customers...

You're certainly misunderstanding me. Microservices are definitely justifiable in plenty of cases, and _services_ even more often. But they _need to be technically justified_ - that's the point I'm making.

The majority of SOA adoption in small-to-medium tech companies is driven by the wrong type of pain, by technical leaders that can see that if they had their domains already split out into services, their problems would not exist, but don't understand that reaching that point involves _solving their problems first_.

_Services_ are obviously a good idea (nobody is arguing something like PostgreSQL or Redis or DNS or what have you should all run in the same process as the web server).

_Microservices_ attract the criticism. It seems to assume something about the optimal size of services ("micro") that probably isn't optimal for all kinds of service you can think of.

> I find it odd that there is this widespread meme—on HN, not in the industry—that microservices are never justified.

Many HN patrons are actually working where the rubber meets the road.

DNS is a poor comparison. Pretty much everything, related to your application or not, needs DNS. On the other hand, the only thing WNGMAN[0]may or may not do, is help with finding the user’s DOB.

https://m.youtube.com/watch?v=y8OnoxKotPQ

> I find it odd that there is this widespread meme—on HN, not in the industry—that microservices are never justified.

There's a few things in play IMO.

One is lack of definition -- what's a "microservice" anyhow? Netflix popularized the idea of microservices literally being a few hundred lines of code maintained by a single developer, and some people believe that's what a microservice is. Others are more lax and see microservices as being maintained by small (4-10 person) development teams.

Another is that most people have not worked at a place where microservices were done well, because they were implemented by CTOs and "software architects" with no experience at companies with 10 developers. There are a lot of problems that come from doing microservices poorly, particularly around building distributed monoliths and operational overhead. It's definitely preferable to have a poorly-built monolith than poorly-built microservice architectures.

I've been at 4 companies that did microservices (in my definition, which is essentially one service per dev team). Three were a great development experience and dev/deploy velocity was excellent. One was a total clusterfuck.

DNS resolution is genuninely reusable, though. Perhaps that's the test: is this something that could concievably be used by others, as a product in itself, or is it tied very heavily to the business and the rest of the "microservices"?

Remember this is how AWS was born, as a set of "microservices" which could start being sold to external customers, like "storage".

"I find it odd that there is this widespread meme—on HN, not in the industry—that microservices are never justified"

I think the problem is the word "micro". At my company I see a lot of projects that are run by three devs that and have 13 microservices. They are easy to develop but the maintenance overhead is enormous. And they never get shared between projects so you have 5 services that do basically the same.

I rarely see anyone claiming that microservices are never justified. I think the general attitude toward them is due to the amount of Resume Driven Development that happens in the real world.

> Team velocity is empowered via microservices and controlling their own data stores.

Bologna. Choosing clear abstractions is an enabler of focus, but that doesn’t necessarily imply those abstractions are a network call away.

Found the author of CISCO / Java Spring documentation.

I honestly expected people used passive-aggressive corpo-slang only for work / ironically. But this reads intentionally obfuscated.

But, to answer to the substance: you for some reason assumed that whoever designed first solution was an idiot and whoever designed the second was, at least clairvoyant.

The problem you describe isn't a result of inevitable design decisions. You just described a situation where someone screwed up doing something and didn't screw up doing something else. And that led you to believe that whatever that something else is, it's easier to design.

The reality of the situation is, unfortunately, the reverse. Upgrading microservices is much harder than replacing components in monolithic systems because it's easier to discover all users of the feature. There are, in general, fewer components in monolithic systems, so less things will need to change. Deployment of a monolithic system will be much more likely to discover problems created by incorrectly implemented upgrade.

In my experience of dealing with both worlds, microservices tend to create a maze-like system where nobody can be sure if any change will not adversely affect some other part of the system due to the distributed and highly fragmented nature of such systems. So, your ideas about upgrades are uncorroborated by practice. If you want to be able to update with more ease, you should choose a smaller, more cohesive system.

Microservices don't magically fix shared schema headaches. Getting abstraction and APIs right is the solution regardless of whether it's in-memory or over the network.

Instead of microservices, you could add a static analysis build step that checks if code in packages is calling private or protected interfaces in different packages. That would also help enforce service boundaries without introducing the network as the boundary.

Or how about "we want to update a 3rd party library due to a critical security issue, but we can't because that library is used in 500 different parts of the code and no way in hell can we stop development across the entire org for multiple weeks".

With microservices, you deploy the updates to public facing services first, write any learnings down, pass it along to the next most vulnerable tier.

Heck on multiple occasions I've been part of projects where just updating the build system for a monolithic codebase was a year+ long effort involving dozens of engineers trying to work around commits from everyone else.

Compare this to a microservice model where you just declare all new services get the new build tools. If the new tools come with a large enough carrot (e.g. new version of typescript) teams may very well update their own build tooling to the latest stuff without anyone even asking them to!

As with so many software solutions, the success of microservices is predicated upon having sufficient prognostication about how the system will be used to recognize where the cut-points are.

When I hear success stories like that, I have to ask "Is there some inherent benefit to the abstraction or did you get lucky in picking your cleave-points?"

This applies to performance optimizations which leave the interface untouched, but there are other scenarios, for example:

- Performance optimizations which can't be realized without changing the interface model. For example, FOO_TABLE should actually be BAR and BAZ with different update patterns to allow for efficient caching and querying.

- Domain model updates, adding/updating/removing new properties or entities.

This kind of update will still require the 32 consumers to upgrade. The API-based approach has benefits in terms of the migration process and backwards-compatibility though (a HTTP API is much easier to version than a DB schema, although you can also do this on the DB level by only allowing queries on views).

> Team realizes their table wont scale, but their data is provided via API. They plan and execute the migration next sprint.

... followed by howls of anguish from the rest of the business when it turns out they were relying on reports generated from a data warehouse which incorporated a copy of that MySQL database and was being populated by an undocumented, not-in-version-control cron script running on a PC under a long-departed team member's desk.

(I'm not saying this is good, but it's not an unlikely scenario.)

Team size is probably the most important factor that should influence the choice about microservices. Unfortunately, there was a period when it looked like every project and every team had to adopt them or be declared a dinosaur.

Also, DRY is not about repeated code. This drives me crazy. Ruby developers love to make code worse by trying to "DRY it up".

You are correct, but static typing does make it a lot easier. Working with Rider feels like working with an IDE that fully understands the code, at least structurally. Working with PyCharm feels like working with an IDE that makes intelligent guesses.

> Network calls are a powerful thing to introduce. It means that you have an impassable boundary, one that is actually physically enforced - your two services have to treat each other as if they are isolated.

That is not too true at all. I've seen "microservice" setups where one microservice depends on the state within another microservice. And even cases where service A calls into service B which calls back into service A, relying on the state from the initial call being present.

Isolation is good, but microservices are neither necessary nor sufficient to enforce it.

The reality of this situation is that the tool everyone is using to build microservices is Kubernetes. It imposes a huge tax on communication between services. So your aspiration as to improving performance fly out of the window.

On top of this, you need to consider that most of the software you are going to write will be based on existing components. Many of these have no desire to communicate over network, and your micro- or w/e size services will have to cave in to their demands. Simple example: want to use Docker? -- say hello to UNIX sockets. Other components may require communication through shared memory, filesystem, and so on.

Finally, isolation is not a feature of microservices, especially if the emphasis is on micro. You have to be able to control the size and where you want to draw the boundary. If you committed upfront to having your units be as small as possible -- well, you might have function-level isolation, but you won't have class- or module- or program-level isolation, to put it in more understandable terms. This is where your comparison between the actors model and microservices breaks: first doesn't prescribe the size.

It is trivial to tightly couple two services. They don't have to treat each other as isolated at all. The same people who create tightly coupled code within a single service are likely going to create tightly coupled services.

It's also something else that breaks. A lot more than function calls.

I think Robert C. Martin has already described that fairly well [1] [2].

[1] https://www.youtube.com/watch?v=N7agCpAYp1Q

[2] https://en.wikipedia.org/wiki/Acyclic_dependencies_principle

No, I made it all up. I wish I had time and interest to formalize it.

It’s called engineering. /s

Not sure why you’re downvoted - but you’re right. We heavily use microservices but we have a well defined stack. Python/gunicorn/flask/mongodb with k8s. We run these on Kafka or rest api. We even runs jobs and corn jobs in k8s.

Functional decomp is left to different teams. But the libraries for logging, a&a, various utilities etc are common.

No microservices that don’t meet the stack unless they’re already developed/open source - eg open telemetry collectors.

Edit: I think the article is a path to a book written by the author. It’s more of an advertisement than an actual assessment. At least that’s my take on it.

> Most places I've worked with microservices had 2 or 3 approved languages for this reason (and others) and exceptions could be made by leadership if a team could show they had no other options.

This works well if you have knowledge redundancy in your organization, i.e., multiple teams that are experienced in each programming language. This way, if one or more developers experienced in language 'A' quit, you can easily replace them by rearranging developers from other teams.

In small companies, this flexibility of allowing multiple languages can result in a situation in which developers moving to other jobs or companies will leave a significant gap that can only be filled with recruiting (then onboarding), which takes much more time and will significantly impact the product development plan.

More often than not, the choice between Microservices and Monoliths is more of a business decision than a technical one to make.

I think it's fair to say microservices increase the need for governance, whether manual or automated systems. When you start having more than 1 thing, you create the "how do I keep things consistent and what level of consistency do I want" problem

In the department I work there's a lot of microservices, about 5-6 so 5-6 teams. But everything is quarkus/spring java and nothing else.

[dead]

If the monolithic application is written in a language with sufficient encapsulation and good tooling around multi-module projects, then you can indeed have well known and encapsulated interfaces within the monolith. Within the monolith itself you can create a DAG of enforced interfaces and dependencies that is logically identical to a set of services from different codebases. There are well known design issues in monoliths that can undermine this approach (the biggest one that comes to mind is favoring composition over inheritance, because that's how encapsulation can be most easily broken as messages flow across a single-application interfaces, but then I'd also throw in enforced immutability, and separating data from logic).

It takes effort to keep a monolithic application set up this way, but IMHO the effort is far less than moving to and maintaining microservices. I think a problem is that there's very popular ecosystems that don't have particularly good tooling around this approach, Python being a major example--it can be done, but it's not smooth.

To me the time when you pull the trigger on a different service+codebase should not be code complexity, because that can be best managed in one repo. It is when you need a different platform in your ecosystem (say your API is in Java and you need Python services so they can use X Y or Z packages as dependencies), or when you have enough people involved that multiple teams benefit from owning their own soup-to-nuts code-to-deployment ecosystem, or when, to your point, you have a chunk of code that the team doesn't or can't own/maintain and wants to slowly branch functionality away from.

I disagree. The older the code is, the more hands it's been through, which generally means it's stronger, hardened code. Each `if` statement added to the method is an if statement to catch a specific bug or esoteric business requirement. If a rewrite happens, all that context is lost and the software is generally worse for it.

That being said, I agree that maintaining legacy systems is far from fun.

> The best code is the easiest to throw away and rewrite, because let's face it, the older the code is, the more hands it's been through, the worse it is, but more importantly the less motivated anyone is in maintaining it.

The more testing that's been done and the more stable it should be.

The argument for new can be flipped because new doesn't mean better and old doesn't mean hell.

This assumes a clear interface. Which assumes that you get the interfaces right - but what's the chance of that if the code needs rewriting?

Most substantial rewrites crosses module boundaries. In micro services changing the module boundary is harder than in a monolith, since it can be done in a single commit/deploy.

My first gig out of school was a .net monolith with ~14 million lines of code; it's the best dev environment I've ever experienced, even as a newcomer who didn't have a mental map of the system. All the code was right there, all I had to do was embrace "go to definition" to find the answers to like 95% of my questions. I spend the majority of my time debugging distrubuted issues across microservices these days; I miss the simplicity of my monolith years :(

Not so much a ball of yarn but more like the cabling coming out of a network cabinet. It can be bad and a big mess if you let it, but most professionals can organize things in such a way that maintenance isn’t that hard.

I've noticed that a lot of industry practices demonstrate their value in unexpected ways. Code tests, for instance, train you to think of every piece of code you write as having at minimum two integrations, and that makes developers who write unit tests better at separating concerns. Even if they were to stop writing tests altogether, they would still go on to write better code.

Microservices are a bit like that. They make it extremely difficult to insert cross cutting concerns into a code base. Conditioning yourself to think of how to work within these boundaries means you are going to write monolithic applications that are far easier to understand and maintain.

If an organization can't figure out how to factor out clearly-defined contracts within a single codebase and maintain that over time, adding a network hop and multiple codebases into that will not make it any easier.

"because you have a clearly defined contract for how that service needs to operate"

But if you need to change the contract the changes span the service and all of its clients.

> you just need to know each juncture where that strand weaves into other strands.

No I don't, that's what computers are for. It's why static analysis is good. Instead of knowing what calls what, you say, "yo, static analysis tool, what calls this?".

Absolutely. This is how we operated when we were at the peak of our tech showmanship phase. We had ~12 different services, all .NET 4.x the exact same way.

This sounds like it could work, but then you start to wonder about how you'd get common code into those 12 services. Our answer at the time was nugets, but we operate in a sensitive domain with proprietary code, so public nuget services were a no go. So, we stood up our own goddamn nuget server just so we could distribute our own stuff to ourselves in the most complicated way possible.

Even if you are using all the same language and patterns everywhere, it still will not spare you from all of the accidental complexity that otherwise becomes essential if you break the solution into arbitrary piles.

Ideally you would use the same language if it had distributed support. Use Erlang or Elixir for example and you have everything you need for IPC out of the box. Might take a little bit more effort if you're on Kubernetes.

One of my problems with microservices isn't really the services themselves, but the insane amount of tooling that creeps in: GRPC, Kafka, custom JSON APIs, protobufs, etc. etc. and a lot of them exist in some form to communicate between services.

You need a CTO with a backbone to say no to the senior people hell bent on padding their resumes and scratching an itch though

Too often devs weaponise micro services and say it’s one of the key reasons for microservices is to allow free language choice !

Maybe logging should be a microservice....

"There might be a good reason to have independent deployable, testable and releaseable units"

Of course this is the bottom line. But everything you define in the sentence can be achieved with a proper pipeline and repository architecture based on a monolith as well. For example teams could use a branch setup where they own their own team branches capable of merging to master and deploying. Each team could then define their own testing strategy and Definition of Done on their "team master".

Having the ability to release independently is actually a social problem, not a technical one. But the symptom of that social misalignment often shows up as a technical problem (dropping release KPIs, etc.)

So changing from a monolith to microservice will most likely only fight the symptom, not the root cause.

This is why I saw microservices are "closer to the metal", i.e. they depend more on the physical characteristics of their environment than non-microservices.

A function call in a monolith can:

  * segfault
  * be called with the wrong number of parameters
  * call the wrong function
  * go through but never return
  * substitute itself with another function

These issues can* be abstracted as well, but the advantage of scaling (being close to the metal) is the disadvantage as well (being very close to the hardware abstractions that let you scale).

E.g., there is no difference between running a microservice on a scaling computer than guarantees memory access across a cluster with interrupts, etc (some millions of cpus and terabytes of memory), and running it on a bunch of instances, except the hardware. The former is exotic and abstracts the hardware, the later does not and so all these "low level" errors surface with great frequency.

I was offering that simplification to compare monolith single host to multi-service single host. No network hops.

But yes, you proceed to explain (some of) the complexities of RPC over the network. Security concerns make this even worse. I could go on..

Engineering is hard

Microservices not being able to talk to each other the network basically never comes up.

What you are saying is outright ridiculous.

That is all true but most languages have semi-sane libraries with semi-sane defaults that handle most of that. Sure you need some config and tuning but it's not completely uncharted waters

Honestly we just need better tooling. Where is my micro-service-fleet creator, which handles built-in all these failure modes of RPC calls?

My first Q was re: monolith on one host to many services on one host.

And yes, when going off-host, you'll have these issues. One should not employ network hops unnecessarily. Engineering is hard. Doesn't make it not worth doing.

Absolutely true, but also: Usually, your business transactions happen in a business context, which happens to be in a microservice. It can be a sign of bad design if you happen to have a lot of those transactional problems.

You will have distributed transactions with a distributed microservice setup, but most transactions will still be be contained within a single microservice (and thus be atomic and not distributed).

Yep.

There are situations where microservices genuinely add net value. In discussions like this one, people make valid points for microservices and for monoliths. Often, words like “large” and “many” are used without providing a sense of scale.

Hers is a heuristic. It’s not a hard rule. There are likely good counter examples. It does sketch a boundary for the for/against equation for microservices. Would love to hear feedback about whether “100” is that number or a different heuristic would be more accurate.

Engineering departments with fewer than 100 engineers should favor monoliths and seriously question efforts to embrace microservices. Above that, there’s an increasing chance the microservices could provide value, but teams should stick to a monolith as long as possible.

That's why I included "/libs", for people like you :)

To be clear, for me they are libraries, but realistically people use them as frameworks, as in "standard" ways to think, "frame" and implement something.

But because I'm not dying on any hill specially front-end ones, I'll take a right then a left and go on my way.

"desktop class web app" is a subset of "modern web-app". If you need frameworks, use them.

In my experience the bozos were absolutely not the newbies. Maybe you work in a job that is dedicated to engineering only, but what happens is that often in a company of non-engineers, some kinda reorg happens where a person ends up on your team who never studied programming in their life, with the assumption that the person is a go-getter who will be able to pick all this stuff up. The experiment is never called a failure when they consistently fail to learn. 2 years later the same underwhelming "engineer" will still be there getting other people to do their work while desperately trying to introduce bugs into production.

Acculturating new developers is one of the main tasks of an organization. I don't think it's very difficult to communicate that some company uses language[s] X[, Y and Z] only.

I work on low code cloud ETL tools. We provide the flexibility for the customer to do stupid things. This means we have extremely high variance in resource utilization.

An on demand button press can start a processes that runs for multiple days, and this is expected. A job can do 100k API requests or read/transform/write millions of records from a database, this is also expected. Out of memory errors happen often and are expected. It's not our bad code, its the customer's bad code.

Since jobs are run as microservices on isolated machines, this is all fine. A customer(or multiple at once) can set up something badly, run out of resources, and fail or go really slow and nobody is effected but them.

Its not automatic but it has the potential for more isolation by definition.

If your service has memory leak, crash it only takes down the service. It is still up to your system to handle such a failure gracefully. If such a service is a critical dependency then your system fails. But if it is not then your service can still partially function.

If your monolith has memory leak, or crash it takes down the whole monolith.

But not all sequences. It depends on your dependencies. Some services are critical for some processes. In the monoloth design its a critical dependency for all processes.