Get the top HN stories in your inbox every day.
buro9
sascha_sl
Or simply use a proper CDN that doesn't pretend to eat all the cost for a flat fee but then sometimes does not. BunnyCDN has an amazing volume tier at half a cent per GB.
buro9
Oh exactly that.
Or if caching is your biggest priority then Fastly or Akamai will shine too.
But if you're balancing all considerations and want the cheap "good enough" caching with the DDoS protection, free TLS certs, and unmetered (assuming you aren't imgur or something)... then Cloudflare does a great job at being good enough. And for those sharp edges... drop in a proxy of your own, or layer your CDNs.
_xnmw
I don't understand, what is the advantage of Cloudflare over Fastly or Akamai if caching is not your biggest priority? Does Cloudflare have better DDoS protection, or something else?
KerryJones
In this scenario are you saying
AWS/Azure > BunnyCDN > Cloudflare?
Or just straight AWS/Azure > Cloudflare?
reitzensteinm
Will BunnyCDN reliably keep an 18gb file in cache without hitting origin? I use and like Bunny, but relying on that to not get a massive bill in the mail scares the shit out of me.
PaywallBuster
they also have storage feature, so they could
jimbobimbo
Azure has its own CDN. If one wants to do Cloudfare -> CDN -> Azure Storage, then at least let it be Azure CDN in the middle, not another cloud provider in the mix. ¯\_(ツ)_/¯
z3t4
Or simply run everything on your own server. All those middlemen are going to kill any latency improvements you get from anycast edge servers.
martindbp
I've switched to Backlaze B2, which has a bandwidth alliance with Cloudflare. Even without it, B2 egress is something like 1/5th of S3, so may be worth thinking about.
rawtxapp
If you use argo caching on Cloudflare, it should reduce origin server load even more. Essentially, instead of going directly to your origin, cloudflare endpoint will first reach to it's root node to see if it's cached there and only that node is allowed to communicate with your origin. I see like ~95% cache hits with that turned on.
XCSme
> Azure|Amazon > Hetzner|Linode > Cloudflare
Why not directly Hetzner|Linode > Cloudflare?
nightpool
Because Hetzner and Linode VPSs have fixed disk sizes, while Azure and AWS have basically infinite storage. You use your cheap commodity VPS as a cache, not a source-of-truth.
manquer
Many of them have managed object storage services as well. OVH[1] and Linode[2], scaleway[3] have them, that should scale for most use cases and are S3 compatible APIs
Also Azure and Linode, Scaleway Backblaze and others are part of Cloudflare bandwidth alliance [4] so there shouldn't be egress fees between the two.
It is really only AWS which is a problem, you don't need this setup with any other provider.
[1] https://www.ovhcloud.com/en/public-cloud/object-storage/
[2] https://www.linode.com/products/object-storage/
XCSme
You can use block storage for scalable disk size: https://www.linode.com/products/block-storage/
remram
If your cache is much smaller than the data, it will be ineffective, unless you think everyone keeps downloading the same tiny subset of files. That last assumption works for web content (e.g. newest articles see more hits) but probably not for data.
nostrebored
So that you incur as much downtime risk as possible, obviously.
I hate these 'cloud economics' optimizations that people tend to try.
ipaddr
The risk that your service becomes faang popular and you suddenly need unlimited everything and need it immediately?
It is possible but highly unlikely. The more likely scenerio is you just continue overpay like a lot of others waiting for the moment. If that moment happens you realize with the sudden popularity your store inventory is sold out so you couldn't profit off of the extra traffic anyhow.
Sebb767
There's a clear trade-off between downtime risk and cost explosion risk. For a hobby/non-profit project, risking the downtime to possibly save 7k€ plus surely saving the surcharge of "scalability" is definitely worth it.
ajmurmann
The best setup will forever remain Heroku free instance tier with a free Pingdom account providing traffic to keep it from getting shutdown
zrail
Another option if Linode's included bandwidth + overages is too much is a dedicated box from Reliable Site. I'm not a customer nor am I affiliated with them at all, I just occasionally check in on their low end prices and noticed that they've started included an unmetered 1Gbps port with every host.
(search HN and reddit for that URL, you'll see they've been around and recommended for a really long time).
edub
If you're going to have an intermediary proxy that you run, for AWS perhaps use Lightsail. It is price competitive, and includes more bandwidth than Linode/DigitalOcean/Vultr for the price.
klohto
You are not allowed to use Lightsail once you use more professional services on AWS atleast per ToS
mappu
Do you have a more detailed citation for that? At $DAYJOB we seem to be using Lightsail (for non-cache purposes) along with some "real AWS" resources without a problem,
edub
Interesting. In this example where the parent comment discusses using a proxy from AWS to Linode/Hetzner to Cloudflare, then I'd go with someone in the Bandwidth Alliance, which would include Linode and Vultr.
ddlutz
Why not use the CDN of the cloud provider you are on? Azure Storage > Azure CDN
tills13
Reducing CloudFlare to a CDN is a disservice. They have some amazing services like Bot Management and Workers that make them very appealing. The CDN is just a nice bonus.
pojzon
Because its order pf magnitude more expensive like anything on the cloud really..
bastawhiz
Azure CDN offers almost no discount on egress over Azure storage directly. The same is the case with Amazon's equivalent services.
canucker2016
Or Troy Hunt can ping his Cloudflare contacts and see if he can get access to Cloudflare R2 Storage.
see https://blog.cloudflare.com/introducing-r2-object-storage/
From the Cloudflare blog, it seems R2 would've handled this exact situation - auto-migration of cloud S3-like-storage objects - download from cloud-storage just once and cache in R2 for Cloudflare to serve.
lewisl9029
Has anyone gotten access to R2 yet? I signed up but haven't heard back myself.
Would love to find out if you can write to any/every region and have things replicate, or if you have to write to a single region. BunnyCDN's edge storage solution looked interesting until I found out it only supported writes to a single region.
Hoping R2 might be my savior here, otherwise will probably have to roll my own active-active minio cluster, which I'm not looking forward to maintaining. Other suggestions welcome!
Dave3of5
Ah the old cloud provider switcheroo. Yip this is the way they make money. They make it easy to setup some gigantic hugely scalable website then hit you with a gigantic scaled up bill. AWS would do this as well.
Team I'm in at the moment is in the early stages of cloud adoption but the company in total has fell hook line and sinker for AWS. When I mentioned the cost there is always an excuse.
The main one being that you don't have to hire sysadmins anymore as that's taken care now by AWS. Ah yes but they have actually been replaced with a "DevOps" team plus just our department now spend > £1 million per year to AWS in hosting costs. A 20% reduction in those fees could pay for a few sysadmin(s).
The next one is that no other vendor would be able to supply the kit. You know StackOverflow is able to run on a single webserver (https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...). Plus many of the other providers have loads of instances available.
I mean I'm not against cloud it's just not the cheapest option if you choose one of the big 3 providers. I use a company called scaleway (https://www.scaleway.com/en/) they have all the essential cloud services you need and everything else you can run yourself in docker or k8s.
Kneecaps07
There's an argument to be made for quality of life for your employees. As someone who has transitioned from on-prem server management to mainly cloud work, my job happiness has skyrocketed. I haven't set foot in a data center in three years and I do not miss it one bit.
Dealing with hardware failures, hardware vendors, confusing licensing, having to know SKUs, racking new cabinets, swapping hard drives, patching servers - it's all awful work. When you go cloud only, you can be more productive instead of dealing with some of that nonsense work.
drdaeman
I always was a software developer first, but in the old days I spent enough time in the server rooms doing all sorts of sysadmin work, and those days I dabble in devops.
And, honestly, I miss the old days. Today, $cloud has some weird spasms where you suddenly get an influx of connection timeouts or tasks waiting for aeons to get scheduled and you just can't log in to a switch or a machine and figure out what the exact hell is going on. You just watch the evergreen $cloud status page, maybe file some tickets and pray someone bothers to investigate, or maybe live with those random hiccups "sorry $boss, everything is 100% good on our side, it's $cloud misbehaving today", adding more resilience -> complexity -> unreliability in the name of reliability to the system. Either way, with the clouds I feel handicapped, lacking the ability to diagnose things when they go wrong.
I don't miss those three days we spent fighting a kernel panic. Was about a decade ago - we outgrew the hardware and had to get a new one with a badass-at-the-time 10GB SFP+ NIC that worked nice for the first few weeks but then its driver suddenly decided to throw some tantrums on almost a hourly basis. I don't even remember the details - a lot of time flew since then, but thankfully we found some patch somewhere in the depths of LKML and the server was a perfect clockwork ever since. That wasn't fun, but that was an one-in-many years incident.
Either way, I do feel that in the ancient ages hardware and software used to be so much more simple and reliable. Like, today people start with those multi-node high-availability all-the-buzzwords Kubernetes-in-the-cloud monstrosities that still fail now and then (because there are so many moving parts shit's just bound to fail at incredible rate), and in the good old days people somehow managed to have a couple of servers in the rack - some proper, some just desktop towers sitting by - and with some duct tape and elbow grease those ran without incidents for years and years.
Have I turned old and sour? Or maybe it's just the nostalgia about the youth, and I've forgotten or diminished most the issues while warmly remembering all the good moments?
pojzon
Cloud popped up mostly due to ease of use. Its a lot easier to hire cloudops engineer with somehow enough knowledge to deploy something on the cloud than someone who will be managing a datacenter and have it running.
The later ppl still do what they did, they just work for Cloud Providers making probably quite a bit more than they did previously.
IMHO its a win win situation for everybody. Less skilled engineers can be “productive” and former sysadmins have huge salaries.
Symbiote
In between your two extremes are colocation (no managing buildings, power, cooling, racks, security, optionally network), dedicated servers (no managing/installing servers, disks, warranties) and basic VMs.
agsamek
We do colocation and we have to deal with HD and ram failures from time to time. Replacement of the hardware part is managed by the provider, but discovery and software requieres our involvment.
I just wonder what happens if a ram or hd failure hits a cloud provider node. Is the architecture on average really able to come over such failures without help and intervention.
kortilla
This reads like a software engineer being happy work caters lunch so he/she didn’t have to cook for the whole team anymore. Didn’t anyone discuss maybe hiring a cook?
fragmede
Yes but soon then you're running a kitchen and then a cafe and catering business, as well as a software startup. Which, given how many startups had in-office lunch/food pre-covid is maybe not a bad way to think of that.
Dave3of5
I think this depends. For OPS people no longer having to physically go into a DC I agree but you've now pushed a bunch of work developers especially now will have a harder time as they used to make code and there was someone who sorted infrastructure now the devs themselves are kept up all nights with AWS stuff going up and down.
If cloud improved QOL for ALL employees I'd agree but I think it just shifts work around and costs more.
goodpoint
> Dealing with hardware failures ... it's all awful work
I've met plenty of datacenter technicians that loved they work and the opportunities for growth it provided.
Some companies really know how to manage a datacenter with minimum pain. Some don't.
BlueTemplar
It's not like all those jobs have been taken over by automation - someone still has to take care of these cloud servers ?
Handytinge
> Dealing with hardware failures, hardware vendors, confusing licensing, having to know SKUs, racking new cabinets, swapping hard drives, patching servers - it's all awful work.
Each to their own, but I think you'll find there's a fairly significant portion of sysadmins who love that work!
3pt14159
I can see both sides. If you're a startup that needs to be able to scale quickly if product market fit is achieved, the cloud really saves your bacon. Or is your ten person team really going to figure out how to get Postgres to reliably run with billions of records, with encrypted backups, etc?
It's basically a form of permanent debt. Faster product market fit, higher long term infrastructure costs until you have enough breathing room to start pulling it into your own datacenter. At that point you have some negotiating leverage with the cloud provider.
On the other hand, if you're not looking for explosive growth man oh man is DigitalOcean or anyone of a number good providers of good old VPSes / Cloud-lite.
capableweb
I keep hearing this argument against using your own infrastructure again and again, and I'm not sure how true it is.
I've worked with teams on both sides, and everyone is gonna have to deal with figuring out how to run at scale, it's just different ways of achieving that.
I've worked with teams that manage their own infrastructure with dedicated servers, and not having to think about scaling for a long time as the one beefy server could just take whatever load you threw at it.
I've also worked with teams who don't manage their own infrastructure and thought they were ready to scale without issues, but once the scale actually happened, it turned out there was more things to consider than just the amount of servers you run, race-conditions were everywhere but no one thought about that.
Definitely a case of "right tool for the right job", but I don't think it's as easy as "Self-managed: harder to scale, PaaS/Cloud: easy peazy to scale".
Dave3of5
Yeah agreed I haven't worked with Google scale companies but I've always found scaling issue to to development related not infrastructure related. So examples would be a bad db query that takes the system down, overly chatting webserver that issues too many queries to the backend, pulling large datasets into the webapp causing exhaustion of memory ...etc. AWS / Azure can't be these issues they have to be fixed in your code.
There is definitely a place for AWS/Azure and their offering of services is fantastic but they are not a silver bullet for scaling your website to millions of active user.
On another point though the vast majority of websites you'll ever build won't have that level of active users. It's a good problem to have though as it means your site is doing really well.
martinald
I don't disagree; but I think the cloud (AWS/Azure/GCP) have sort of shielded people from how cheap/powerful the underlying hardware has became.
For ~100eur/month on hertzner you can get a 16core Zen3, 128GB RAM with 8TB of NVMe SSD.
Unless your stack is horrendously badly optimised you can serve SO MUCH traffic off that - definitely billions of postgres records without breaking a sweat.
So the scale argument somewhat disappears - if anything, people end up adding much more complexity to the product to get round the high hardware costs of the cloud (complex caching systems for example, instead of just throwing loads of hardware at the problem).
jjav
> I don't disagree; but I think the cloud (AWS/Azure/GCP) have sort of shielded people from how cheap/powerful the underlying hardware has became.
I guess I shouldn't be surprised, but I do find myself often surprised to realize that for a younger generation of developers they have never experienced hosting on bare metal. So they have not been exposed to costs & benefits vs. the cloud approach and feel that no local machine could ever be as fast as AWS. Even though in reality even a pedestrian server is immensely faster and cheaper than any AWS offering.
Now, sure, there are tradeoffs in ease of scaling up and other considerations, but it's good to keep and eye on the actualy tradeoffs you're making and how much it's costing.
Ostrogodsky
> For ~100eur/month on hertzner you can get a 16core Zen3, 128GB RAM with 8TB of NVMe SSD.
What option is that? The closest I see is the CCX41, but that is 40% more expensive, 140 Eur/month, half the RAM (64 GB) and ~4% of the disk space (360 GB)
Dave3of5
> Or is your ten person team really going to figure out how to get Postgres to reliably run with billions of records, with encrypted backups, etc?
Actually AWS won't help you here. I have literally been on a 2 day training course or aurora with AWS and the explanation of how to scale was actually just the same as any traditional non-cloud explanation. Correct usage of indexes, partitioning data, optimising queries (especially any non trivial query output by an ORM) and read replicas.
In terms of explosive growth if you're talking about something like google or tiktok again slapping it all in AWS will not automatically just work. There is a lot of engineering that you'll need to get to their level.
I also think you haven't really looked at the SO link I sent through with thoughtful engineering they have huge user base with a tiny footprint.
> DigitalOcean or anyone of a number good providers of good old VPSes / Cloud-lite
Not sure why you are dunking on DO here they are a fully fledged cloud provider with much the same stuff you would need. You can also run up a huge bill on DO as well.
Bedon292
There are two parts to this. You are correct that RDS doesn't help you with picking the index strategy, or optimizing queries. I don't see that as running the DB though, that is how you interact with it once its running. What it does do it help you reliably run the DB server itself.
Without any effort you can stand up a redundant, high availability deployment. With all of the data encrypted at rest. And configure nightly backups, which are stored on redundant storage in multiple physical locations and also encrypted. You can then restore those backups into a working system with the click of a button. Oh, and minor version patches happen automatically with no downtime. And you can click a button to do major version updates.
The last time I did analysis on it, which was a while ago, all of those features cost us less than 8 hours of my time each year. It would probably take more than 8 hours of my time each year just to handle security patches on the systems. Let alone the amount of engineering that it would take to get a system as redundant and reliable as a DB in RDS. I will happily pay them to take all of that off my plate so I can focus on other things, like optimizing the queries.
ignoramous
> I can see both sides. If you're a startup that needs to be able to scale quickly if product market fit is achieved, the cloud really saves your bacon.
Depends on the team size of the said startup [0]. In my opinion, tech-shops are better off using new-age cloud providers like fly.io / glitch.com / render.com / railway.app / replit.com / deno.com / workers.dev etc [1].
[0] https://tailscale.com/blog/modules-monoliths-and-microservic...
fiddlerwoaroof
> is your ten person team really going to figure out how to get Postgres to reliably run with billions of records, with encrypted backups, etc?
Most of the problems here will be DBA problems like understanding query plans and such. Even with AWS RDB, I’ve had to upload various setting files to tweak tunables to get things working.
mcbain
That stackoverflow infra blog post is out of date. They use more than a single webserver now. For example: https://stackexchange.com/performance
dijit
Now they have 9.
They still serve a lot more traffic than I do and I have hundreds of instances; thousands of containers.
nightpool
You have thousands of containers? Physician, heal thyself.
andrewxdiamond
Most importantly, SO is extremely read-heavy, write-lite, and cache-friendly.
A similar “scale” e-commerce site would be significantly more load, have more dynamic data, and just be overall harder to run.
Dave3of5
Looks like they have actually reduces their footprint. It not that they do run on a single webserver it's that they can run on one.
chasd00
> Looks like they have actually reduces their footprint.
i don't remember who said it but a quote i really like is "it's not finished when there's nothing left to add, it's finished when there's nothing left to take away"
traceroute66
See also Let's Encrypt: https://letsencrypt.org/2021/01/21/next-gen-database-servers...
nova22033
our department now spend > £1 million per year to AWS in hosting costs. A 20% reduction in those fees could pay for a few sysadmin(s).
You can hire a "few" sysadmins for 200k/year?
Dave3of5
In the UK/Europe yes:
https://uk.indeed.com/jobs?q=System%20Administrator&vjk=5149...
Probably not at FAANG level salaries but I doubt there are many sysadmins working for FAANG companies anymore.
DevOps btw are more expensive and infact in the UK DevOps can be higher paid that a developer. I suspect most of the DevOps working for this company are on £65k+. According to:
https://ifs.org.uk/tools_and_resources/where_do_you_fit_in
That puts those earners in the top 3% or from that website:
" In the below graph, the alternatively shaded sections represent the different decile groups. As you can see, you are in the 10th decile group.
In conclusion, Your income is so high that you lie beyond the far right hand side of the chart. "
mattbee
£200k / year, in the UK? That's about 2-5 depending on experience.
sparselogic
A 20% reduction would result in ~£800k/yr.
Arnavion
They're saying that if the AWS costs decreased by 20%, they could use the now freed-up money, ie 200k, to pay sysadmins.
InefficientRed
> > £1 million per year
I'm curious about your workload. I tend to only use cloud for workloads where it's either (1) by far the only feasible option (e.g. need GPUs for short periods of time), or else (2) basically free.
> I mean I'm not against cloud it's just not the cheapest option
This is certainly true for most workloads. It's also true that buying is better than renting, but here I am living in a rented apartment.
The logic from on high might be something like "if demand is uncertain and capex is risky, why buy when you can rent?"
cyberCleve
Ouch. If Troy Hunt of all people can make this mistake, it can happen to anybody. HIBP is an awesome service funded totally by donations, so it's too bad this happened. Of course Microsoft is happy to hide behind their confusing pricing model and let customers overpay for Azure without alerting them.
j1elo
> If Troy Hunt of all people can make this mistake, it can happen to anybody.
Exactly this. As a low-level / embedded / non-cloud stuff dev, I've been getting up to speed through all the cloud-ification of the industry, but I'm still scared (not literally ofc) of running most things on my own on any big cloud provider (smaller ones seem more manageable).
I'm reading this and seems like being a customer of cloud services is like walking a dangeous path filled with gotchas and caveats, just jumping from cover to cover while hiding from danger, and hoping you're safe and didn't mess it up so far, "fingers crossed".
Like this tiny detail that he didn't realize was critical, so I would fall on it too plus on another 500s small papercuts: "oh I set cache up, so I hope all is well". "Yeah, no you aren't, I guess you didn't think of this detail about maximum cached file size! Gotcha, Game Over!"
Yeah cloud providers should have clearer communitacions and etc etc... but the fact of today is that they don't. So I'd never sleep well feelin 100% confident that I had covered and taken into account every minuscule detail and possible scenario that could end up being a disaster.
qw
> I reached out to a friend at Cloudflare and shortly thereafter, the penny dropped
Another advantage is his big network that he can ask for help. There's also a chance that his blog post will reach the right person in Azure and he'll get a reduced bill.
As someone who doesn't have the same network or the "fame", I am concerned about what would have happened to me in that situation.
chasd00
Remember when no-sql came out and everyone was rushing to it because "rdbms don't scale"? I'm beginning to feel the same way towards "cloud" in the Azure or AWS sense. You can go really really far with standard issue VMs from linode or digital ocean and so on. I wonder how many are overpaying for Cloud services so far above and beyond what their actual needs are.
sydthrowaway
How are you teaching yourself?
j1elo
Due to my work, I'm involved in some WebRTC stuff and have been getting in touch with deployments on AWS, CloudFormation, that kind of stuff.
tpetry
Every cloud makes this mistake easy! You have to manually activate billing alerts for everyone because they want you to spend more snd more each month.
I am still waiting for a cloud without these dark patterns. But that will never happen because it‘s leaving a big amount of money on the table by not being hostile.
Dave3of5
Also the billing alerts is just that an alert. They should have something in place to put a hard cap on monthly spend. That way his free website would go offline when he's spent > $X.
As you say they make it hard deliberately.
Edit: Turn out Azure have this:
https://docs.microsoft.com/en-us/azure/cost-management-billi...
danuker
I see there is a spending limit for the "intro" or "preview" plans designed for students, Visual Studio users, and resellers (the "hook" part in "hook, line, and sinker").
Not for actual cloud usage, like an actual pay-as-you-go plan where this would be useful.
https://azure.microsoft.com/en-us/support/legal/offer-detail...
GordonS
And the alerts are not instant, at least with Azure - they run reports every 24H or so, and execute alerts every 24H or so. So even if you're careful, you can still be on the hook for a couple of days' worth of spend - which could be very expensive.
fendy3002
Eh after skimming it I feel like there's still gotchas with it. Not every account can turn it on and looks like there aren't custom limit.
CMIIW, it'll be my first cloud provider if I can set one.
Ostrogodsky
I dont understand why not all online metered providers are forced by law to do this.
harry8
Dark patterns - this sounds like a colour scheme you don't care for.
"Predatory death-trap pricing" captures the spirit of the thing with rather more clarity. It is wholly intentional after all.
OrderlyTiamat
> Dark patterns - this sounds like a colour scheme you don't care for.
I can see your point- if I'd never seen the term before I might have a similar reaction. But it's quite a common term now I think.
gwd
> Dark patterns - this sounds like a colour scheme you don't care for.
Or craft clothing for goths?
But the "dark" comes from its association with evil: "Defense against the Dark Arts", "The Dark Lord", "Turn to the Dark Side of the Force". It's a clear implication that the people are "selling their souls to the devil": knowingly doing something "a little bit evil" to achieve their aims.
timje1
We had a similar situation to Troy's where several thousand pounds was charged in a matter of days as a result of our misconfiguration of caching in our azure app services (before that month we typically had around £800 a month costs). We emailed Azure / Microsoft and they were happy to refund us. I don't think this is their intended business model.
iso1631
Get a VPS from linode for $5 a month and it costs $5 a month.
gkhartman
They have transfer limits and an associated overage fee iirc. I can still see this sort of thing happening if that is the case.
hkt
Personally I like Mythic Beasts and use their raspberry pi servers and VPSes. Much less terrifying pricing and the support is good too.
ozim
Not really as there is network traffic quota:
If you use up your monthly network transfer pool, you can continue to use your Linodes normally. That being said, you will be charged $0.01 for each additional GB at the end of your billing cycle.
tpetry
Sure that's what i am doing. A beefy dedicated machine with no bandwidth pricing. But that also means i need to do everything by my own. I don't get any of the worry-free services of AWS.
nix23
Or hetzner, or Vultr.
sgustard
If I leave my water running then go on vacation I'll have a huge water bill too. I don't conclude my water company is intentionally trying to overcharge me. The more reasonable conclusion is: building an alert system that addresses every customer need is hard. Most enterprises (where all the customer focus is) want minimal downtime above other considerations, including cost.
addicted
This is like you leaving your tap running slightly as you go on winter vacation so the pipes don’t freeze over.
But the water company does not actually allow you to install proper taps to regulate the water so you use duct tape to do so, and due to an earthquake something falls on the tap causing your duct tape solution to fail leading to a massive surge of water, leading to your massive water bill.
Did the water company cause this? No. Your duct tape solution wasn’t resilient enough because it didn’t factor in an earthquake. But I would be justifiably mad that my water company does not allow me to install actual taps, and allows unforeseen and unpredictable situations to make me run up huge bills that could otherwise have been avoided with a proper tap.
Nextgrid
The problem with the cloud is that the entire Internet is in control of your tap and can choose to open it and waste infinite amounts of water with no way to implement a cap.
nix23
>Every cloud makes this mistake easy!
Funny enough...Oracle (OCI) makes it better, you can buy oracle"coins" 1to1 with $ and load your account just with what you think you need.
christophilus
If Oracle cloud is still shenanigan free in 2 decades, I’ll consider it. Until then, Oracle gets $0 of any budget I’m in charge of.
dx034
Hetzner's cloud offer is limited but they limit your possible spending by default and it's very easy to set up billing alerts. I guess they mostly do it to ensure they get the money at the end of the month, but it's equally useful for their users.
tpetry
Additionally Hetzner's egress pricing is a lot more cheaper. On hetzenr you pay 1,19€/TB (1.13$/TB) vs. 90$/TB on AWS. That's about ~80 time more on AWS!
TedDoesntTalk
> I am still waiting for a cloud without these dark patterns.
This is how mobile and landline phone companies made enormous fortunes before flat rate billing. It’s called post-paid vs pre-paid billing.
Grollicus
Do you have any substance to your allegation of Microsoft hiding behind their pricing model?
This is very straight forward from their view, before: almost no traffic = almost no costs, now: huge traffic = $$$.
On the other hand, it doesn't seem that Troy did try to talk to them about this and seems to want to eat the costs himself. As it was his mistake. I think that's commendable. I also think with the amount of free advertisement Troy has done for them they'd be open to this and I can imagine we might see a followup post like "MS was so nice they waived my costs".
Nextgrid
Putting anything internet-facing on the cloud is as irresponsible as posting your credit card number publicly. Anyone can essentially charge you an infinite bill and you can't do anything about it until it's too late.
Maybe it's not a problem when you're dealing with millions of VC money, but there's no way in hell I would host anything in a bandwidth-metered cloud service when my or my own company's money is involved.
capableweb
Correct me if I'm wrong, but Troy Hunt is a person focusing on security, not infrastructure, deployments or development even. If anyone is near making that mistake, it's people like Troy Hunt. Operators would of course see the problem easily (paying for bandwidth like that would be the first warning sign), while they are sometimes blind to other issues, like security.
Closi
> Correct me if I'm wrong, but Troy Hunt is a person focusing on security, not infrastructure, deployments or development even.
Eh, I don't know - either way he is a Microsoft Regional Director and MVP for Cloud (as well as security), runs courses on cloud deployment on Pluralsight, and has done speeches on Azure and reducing cloud bills, so if a he can get stung it doesn't say a whole lot good about my chances.
capableweb
Thanks for the correction :) I had no idea he did all of those things in addition to the security work. Certainly doesn't paint Azure or him in a good light then.
moritonal
So I guess one method would be to set spending-limits when you setup your account. But that'd lead to constant moments of having to bump your budget (or worse, get approval to do so from Accounting) when you're trying to work.
There are both spending limits and alerting that you could use, but would be impossible to predetermine from Azure's perspective, so they rightly ask you to.
brimble
There's an entire surprisingly-large industry built around providing better UI to the major cloud providers, so you can actually tell WTF is going on with billing, access control, networking, et c. They're so hostile that it has to be intentional.
scapecast
The underlying issue is that the cloud console is owned by a single product team, and THEY decide what gets exposed - not the underlying product teams for the individual services. At least that's the case for AWS.
The result is that you get a lowest common denominator type of dashboard. And hence a whole industry of providing just a prettier dashboard on top of AWS / GCP / Azure metrics.
Datadog started with a prettier dashboard for Cloudwatch data.
Cloudability started with a prettier dashboard for the Cost and Usage Report.
And also works the other way around. The individual product teams buy development environments to circumvent the console restrictions.
For example, a few years ago, the Redshift team purchased "DataRow".
oneepic
It is worth mentioning that the alert itself costs money. So if you're evaluating the alert every 5 minutes on the past 24h of data it can burn a small but surprising amount of money.
From TFA it looks like that would be 10 cents per "time series". Or what I translate it to, is 10 cents every 5 minutes (*I think, but I havent used Azure in some time*). $1.20/hour, $28.80/day, almost $900/month. Not too hard to drop that by making the alert less frequent. (edit: I think I saw AU$ there, so maybe it is AU$900.)
manarth
A time-series represents a "thing you're monitoring" – in this instance, it's aggregate egress, so $0.10 per month, regardless of the evaluation period.
Monitoring CPU? Another $0.10 per month. Memory? Another $0.10.
Thankfully, not $900.
oneepic
I meant to emphasize frequency, not eval period. Apologies. That said I took a look at the pricing docs and didnt see frequency mentioned, so hopefully I am in the wrong about the price.
As an aside, their (Azure's) pricing docs are written in the same fishy way their technical docs are written (my opinion only)...
giaour
The service being used here is Azure Monitor, which charges $0.10/month for each metric you monitor with alert rules: https://azure.microsoft.com/en-us/pricing/details/monitor/
They do charge by the MB for ingesting custom metric data, but storage outbound bandwidth is one of the free "standard metrics" built into the platform. So the alert's full cost is $0.10 per month. Since they charge by the "time series," I think that means the OP could configure different alerting thresholds on storage outbound bandwidth, and they'd still only be on the hook for $0.10/month.
TriNetra
Shameless plug: https://CloudAlarm.in (in beta), sends you real alerts usually faster than azure with multiple reminders. It does this daily unless you tell it to shut up for the month for the given exceed. I call it real alerts because it doesn't wait for consumption threshold to reach the way Azure cost alerts do; as soon as it detects that your current cost * remaining days > the budget amount, it'll send you an alert [1].
The alert emails are way more meaningful (with projected amount in subject for example) unlike generic ones from Azure Alerts, so you see a real alert and prompted to take immediate action.
1: https://cloudalarm.in/Home/Docs/#how-is-budget-alarm-differe...
GordonS
But surely CloudAlarm relies on the same data as Azure's alerts do? Azure support told me that data is only updated daily.
Also, Azure has an option to alert you beforehand if it looks like you'll go over; struggling to see how your service is any better.
TriNetra
AFAIK, Azure only alerts when a threshold you specify is reached. For instance, you set a budget of $10k and you specify 50% threshold. So when you have consumed $5k, Azure will send you an alert. However, suppose your daily run due to some bad selection is $400, in this case, Azure will tell you only on the 14th day that you've consumed 50%. On the other hand, CloudAlarm doesn't need threshold – it just takes your budget and see whether your daily burn rate estimates to exceed your budget. In the above example, CloudAlarm can thus notify you on the 2nd day itself because $400 * 30 is $12k.
Yes, CloudAlarm, as of now, depends on Azure's pricing data API to do this calculation. It's easy for Azure to do what CA is doing but their threshold based design is a problem, which only prompted me to create this service.
CA also has 'new resource' alarms as well, which are almost instant (a few mins after creation of a new resource), which helps you monitor and fix resource created with unexpected, expensive tiers. This can often happen with automated creation of databases, for example.
I just did it because Azure wasn't doing it, despite people complaining, including me, had faced multiple such issues of unexpected expensive resource got created without intention/knowledge.
mnahkies
This is something to be mindful of when using datadog synthetics monitors as well - if you have a short interval, or many locations being tested from they can become expensive quickly
godot
These stories almost always boil down to this fundamental conflict of what you want for a personal project vs a business. (though in this case yes, Troy Hunt's HIBP is larger than a lot of startup businesses)
In a business setting, you want your service to stay up, at the cost of spike in costs if accidents or mistakes happen.
In a personal project, you want there to be hard limit on cost, and your service to go down if spikes call for it. (I'm relatively sure that no one wants their personal projects to incur a bill of thousands of dollars by accident.)
kortilla
> In a business setting, you want your service to stay up, at the cost of spike in costs if accidents or mistakes happen.
No you don’t. This is absolutely not a given. Being a “business” doesn’t mean you suddenly have unlimited budget.
The vast majority of businesses are not “web scale” and are better off taking an availability outage than suddenly handling 1,000,000x the normal volume of traffic.
lytefm
I'd say that it definitely depends on the business.
If you are selling you product via your web site and you're suddenly on TV with millions watching and accessing your site, you definitely don't want the server to go downand autoscaling + a bit higher cost would be great.
Symbiote
Only if you can actually fulfil those orders. If your production can't be increased, and the business is small, the cost could far outweigh the potential profit.
kortilla
Not if you’re selling a real product. The only world where people can scale from 1000 orders a day to a million orders a day overnight is one where you’re not selling anything physical.
trulyme
I don't agree with this - even for businesses there is always a limit over which there is serious trouble for bottom line. I think cloud providers should allow one to set a hard cost limit over which everything shuts down. For personal projects the limit might be $100 and for small businesses $100k, but even rich companies have it (not the same reason, but Knight Capital comes to mind).
ghaff
Certainly the cloud providers probably make money by not having hard limits.
But it's also the case that if they did implement hard limits of some sort, you'd be reading blog posts about how AWS destroyed my project just when it was going big because someone stuck a circuit breaker foot gun in some corner and everything stopped working properly when usage spiked.
I do think there should probably be a hard circuit breaker. It should be simple and therefore inflexible. And it should come with a big warning sign. Still people will get burned because someone will set it, a project grows, and one day it goes off.
metalliqaz
But... they do implement limits.
ghaff
As far as I know, they implement billing alerts but, aside from some student and some other limited account types, they're alerts. You'll get an email that you've hit your limit but your bill will continue to go up until you shut things down.
DasIch
They have limits on things like how many EC2 instances you can have but not on things like bandwidth.
While you can raise those limits by request I'm also not sure whether you can actually reduce them again later.
trulyme
Aws doesn't, there are only alerts. Don't know about others.
benbristow
Azure (and I'm sure other cloud providers do) allow you to set email notifications for when your bill goes over a set amount so you can stop it before it happens.
If you're using a cloud provider I'd highly recommend setting one of those up.
In Azure it's under your Subscription and then Budgets
skeeter2020
AWS can send you alerts when it looks like you will go over your budget for the period, before you're way over your projections, which is a nice feature.
I truly believe they want you to use a lot of their resources on a consistent, long-term basis; they don't get long-term value from people having short, one-off anomalies, so budgets and monitoring are aligned with their customers - just not total cost of ownership calculations :)
temp8964
Is it really that black and white? I think there is a continuum in hosting service. Not just A) very low end VPS, and B) unlimited cloud.
The fact is that there are low end VPS, middle end VPS, high end VPS, and dedicated servers. If you started from a low end VPS, it is very easy to gradually upgrade your VPS.
A $5/month VPS can be used to play for tons of things. I just don't get people who use free tier cloud, unless you just want to learn about the cloud hosting per se.
Nextgrid
The problem is that the cloud alone will not allow you to scale infinitely. Slapping a standard DB-based application on the cloud will not magically make it scalable. RDS will still be a bottleneck and can't be scaled online for example (Aurora might change that, but that's a very recent development).
Making your application scalable is a significant effort that may involve different trade-offs. Your typical Prestashop or Magento e-commerce site will still max out the DB and go down, cloud or not, but with the cloud you'll end up with a huge bill in addition to your downtime.
Engineering your application to be scalable is an option that's often not made for cost/time to market reasons which is fine, but in this case the cloud will give you much less scalability than a lot of people believe.
bushbaba
Isn’t that why services such as AWS lightsail and digital ocean exist?
stevehind
Have you contacted Azure? On one hand you owe the money “fair and square”, but on the other if I were them I’d waive an unexpected $10k bill to a good faith actor that was incurred without any proactive notification by Azure.
jillesvangurp
Yep, we had an incident with Mongo cloud where a bug in their synchronization protocols for Mongo Realm resulted in an insane amount of traffic. This was a development cluster with almost no application load somehow pumping around many TB over the course of a few days. The bill was many thousands of dollars. Their support did the right thing. And we actually ended up with some credits because we were having a rough time with bugs in their software. Ultimately, we gave up on Mongo Realm because it was just not working as advertised for us (high CPU usage on the device, lots of bandwidth, we experienced data loss in the managed cloud storage, etc.). But their support team was great.
Their interests is keeping you as a long term customer. So, they will help you if they can. Unexpectedly high bills like that can end the relation in no time. And 10K is not a lot on a yearly basis. That's a few months of normal usage for lots of companies. So, protecting that revenue is worth something to them. That's also worth realizing when you deal with cloud providers: you are spending non trivial amounts of money on their services and support is part of that deal.
gurraman
A developer on a team I worked with many years ago accidentally committed our AWS keys in a repo. Got a $30k bill due to a an enormous amount of EC2-instances being spawned. We contacted AWS and they were very understanding and reduced the bill to $50.
Fomite
I had this happen to me once on Digital Ocean, and I contacted them - they were rather understanding that the bill I had was clearly "atypical for my account and not intended" and refunded it.
SoapSeller
I'll second that.
I've seen several cases on both Azure and AWS that bills got weaved after someone opened support ticket starting with "oops, I just did..."
ramraj07
I got an $800 aws expense (one line item) waived after I contacted them and they asked me to explain why it happened and how I’ll prevent it from happening in the future. I think it’s a once per account thing they’d probably do and Troy should definitely do it.
0x008
> Secondly, there's cost alerts. I really should have had this in place much earlier as it helps guard against any resource in Azure suddenly driving up the cost.
He did not enable alerts.
sylens
Every online course that requires you to use a public cloud to deploy something should first have you set up a billing alert that notifies you when costs start to creep past something reasonable, like $20 or $50 (depending on the course and work involved).
asadlionpk
OP do this! It works, they are usually very generous (same for gcloud!)
goodguyamericun
Op is Troy hunt, an ms MVP. You can bet there are people from MS doing it for him as soon as they got wind
undefined
suction
I wonder if before cloud computing, has there ever been a successful product / service where it was accepted with just a shrug that the volatility of monthly costs means it could bankrupt you with next month's bill, because of complexities and opaqueness of the cost structure make it virtually impossible to predict and protect against extreme peaks in all parts of the setup.
Even if you run a relatively opaque cost structure business like a restaurant, you can still calculate the maximum cost of ingredients for one month, the salaries, energy, etc. if you simply use the "best case scenario" of having every seat at every table booked for all opening hours, with people ordering your most sold dishes. Cloud computing is still leagues above that in terms of cost predictability.
I once worked for small, non-startup software company who pondered moving servers to Azure. The Azure partner shop analysed the needs and came up with a monthly cost "between 30k and 120k per month". They were really surprised the company stuck with their non-cloud setup because "everybody is moving into the cloud!!"
tolien
> Even if you run a relatively opaque cost structure business like a restaurant, you can still calculate the maximum cost of ingredients for one month, the salaries, energy, etc.
If the restaurant suddenly ordered ten thousand times more ingredients than usual, their supplier would probably call back and say "is that really what you want?" rather than just shrugging and shipping them tonnes of tomatoes with a bill for one billion dollars.
suction
Very true. And in terms of cloud computing, it would mean that alerts and notifications and limits are worth absolutely nothing if it's on the customer to set them up in the correct way for every scenario imaginable. Which is nearly impossible. The tomato supplier's human alerting system is a catch-all-system which would be easily implementable as well.
tolien
Yeah - if you look at Troy's graphs they're already calculating an average bandwidth and the alert he's configured has a threshold ~1/50th his current level.
Trying to set a hard number limit ahead of time is hard (estimating how much you'll use, don't want to set a number too low and get cut off plus cloud cost structures can be really hard to get your head around) but that basic level of anomaly detection should be there by default.
stickfigure
I'll bet Sysco would deliver $10k worth of canned tomatoes to your restaurant without checking.
Symbiote
Since the tomatoes would be worth $8k (or whatever), they might do a bit more diligence on ensuring the customer can pay.
MS's bandwidth cost a fraction of what they're charging, so it's easy to risk people not paying up.
lkbm
At my previous housing co-op, the new kitchen manager accidentally ordered nine cases of limes (~$1000) instead of 9 limes.
They assumed it was a mistake and only delivered a single case, which was still 180 limes, but at least it didn't use up our entire food budget.
(Normally I'd expect a phone call or email to confirm, but this was a smaller, local supplier, so they probably didn't have real systems to deal with outliers.)
corobo
In this scenario though you’ve used tonnes of tomatoes and they’re now asking you to pay
tolien
Tomatoes that were ordered on terms where they’re paid for well after they’re delivered, with a long-running relationship with the vendor. If you went from ordering a few tomatoes to ordering entire lorries full of them, you bet the vendor’s going to check you’re good to pay for them.
Troy Hunt didn’t sneak into an Azure DC and install some hardware any more than this hypothetical restaurateur filled a truck at the local fruit market.
bstpierre
A gas or electric bill works a bit like this... if you have some appliance that fails in a way that suddenly starts consuming much more than usual you can end up with a fairly large bill at the end of the month. Same for old school landlines or cell phones, before flat rate billing became ubiquitous.
Though in those cases the billing isn't really complex or opaque, and you _can_ monitor it if you care to check your meter regularly throughout the month. But, for the electrical case anyway, you can't drill into what exactly is consuming watts without either fancy monitoring equipment or potentially tedious investigation.
Nextgrid
The big difference in case of gas or electric (or even telephone) is that you and only you are in control. Someone would have to physically break into your house or steal your phone to rack up a huge bill.
In contrast, with the cloud the bill is directly proportional to the amount of inbound requests from the Internet, with no out of the box way to implement a limit (I guess you could install Apache/Nginx and enforce a limit there, but doesn't that kinda defeat the whole point of the cloud?).
bstpierre
Some cell plans have a charge for incoming texts, roughly like the cloud bill you’re talking about.
And (as sibling comment points out) if you’ve got critical household infrastructure that needs gas/electric, you’re not always in control. An unexpected hard freeze could provide you with a big bill next month and you can’t fully control it.
I agree with you that there’s definitely a difference of scale and asymmetry but the concept of unpredictable postpaid bills is not a new thing.
macintux
> A gas or electric bill works a bit like this
Just ask Texans.
avrionov
I worked on both cloud computing and on premise project. Before cloud computing the risks were different: - much harder to scale. It was much more common to over provision and have machines and bandwidth being unused for years.
- when we were hit with very high traffic due to a bug or something else, most of the time it would lead to customer outages. Based on the contract some times it requires to pay back because SLAs were not reached. Also an outage could lead to customers canceling the subscription.
We swapped one type of problems with another.
suction
If you have a bug that renders your product unusable and refunds are in order, the flexibility of handling traffic peaks which a cloud provider offers won't solve that problem for you. It could even aggravate it. If a show-stopping bug is introduced, it would probably be preferable to fail quickly.
philliphaydon
If there was an outage in early 2000, we just went outside to play, or watch tv.
Now if Facebook is down for 15 seconds everyone has heart failure like their life is over.
lytefm
> It was much more common to over provision and have machines and bandwidth being unused for years.
But the overprovisioned server might still be a lot cheaper than the cloud bill. It can be totally reasonable to have a server running at 1-5% load 98% of the time if you really need the capacity for the remaining 2%.
Also, neither "scaling up" as in "re-deploying the same setup on a beefier instance" nor "scaling out" as in "let's expand to the US and have a server there" is too difficult if the setup is automated (Ansible).
undefined
octoberfranklin
Banking.
Credit card chargebacks, especially.
usr1106
That's the typical story. Something goes wrong and it costs you (typically a small company) a lot of money. At that time just nobody is looking at metrics. Even alarms don't help absolutely because they can also be missed.
The only thing that would really help were a hard spending limit that stops all services except storage. If your site is important there will be such an amount of user feedback that it is impossible to miss it for a long time.
dspillett
Alerts can also fail to be timely due to mail/SMS/other delivery issues, or the right people being in the middle of something else. This delay means it is still possible to rack up and unexpected cost.
Or they can fail completely.
And the alerts themselves cost if you want something reliable so you have to weight that against the danger. Pay as you go cloud can be a maze of costing concerns..
> The only thing that would really help were a hard spending limit that stops all services except storage.
Yep. Though that is small comfort if you need to guarantee more than a couple if 9s of uptime, hopefully those with that requirement can soak up the unexpected billing blips.
alfiedotwtf
> The only thing that would really help were a hard spending limit that stops all services except storage.
Sadly, I haven't found a way to do that with AWS
dx034
It's funny that even Hetzner can do that and AWS can't. Shows that there's no interest from AWS to prevent these things from happening.
bencollier49
IMO this is something which ought to be written into law. It'd be easy to implement a kill switch, and would actually encourage innovation, as people would feel more empowered to experiment with the technology.
Nextgrid
To be fair, Hetzner doesn't nickel & dime you on everything.
Some things are legitimately very difficult to meter in a real-time way with adequate performance. Imagine if the S3 server had to do a DB query to lookup your account balance & limit on every HTTP request. That'll completely kill performance and availability (what if the billing DB is down? Should that DB now be replicated across all regions? etc).
The reason a lot of cloud alerts and metrics lag is because billing is done asynchronously by parsing logs way after the actual usage occurred. Of course, the real solution here is to just not bill for things you can't easily measure & limit especially when those things are super expensive, but cloud providers' C-suites have to eat & pay their bills somehow.
danparsonson
*won't
UnFleshedOne
I just looked at my AWS account and there seems to be a way to set budget, attach alerts to it and attach actions to alerts. For example there is an action to stop EC2 instances. Not sure if other AWS services have something similar, but at least you can kill your instances if something weird happens.
Actions weren't there last time I checked (few years ago).
alfiedotwtf
Thank you, I'll check it out
Monotoko
Kill switches in lambda I believe is possible, running when the alert is triggered
alfiedotwtf
Nice, I'll have a look. Thanks!
mrb
Most worrying is that even an expert like Troy Hunt was UNABLE to figure out the cause of the issue by himself. He "reached out to a friend at Cloudflare" who investigated and found the cause.
alkonaut
Cloud providers should always have a max spend and it should be a standard feature. The cap shouldn't even be some optional feature or notification service. It should be a hard cap that you can move - at your own risk.
manquer
SMB or indie developers are not the first/primary customers for Azure/AWS that they design their application for.
Any enterprise will not want any limits because of spends, they would be lot more pissed if service was pulled because spending cap set by someone sometime in the past is now exceeded. Likely is why such feature is optional not mandatory.
Excess/unexpected billing would be negotiated in typical sales cycle discussions. Making a default hard cap however would result in a lot of senior people are going midnight calls for emergency budget approvals, management would get annoyed by that.
foobiekr
I 100% work for a large enterprise and we would absolutely like spending policies in place. After all, we have fixed OPEX budgets planned in advance of the quarter.
manquer
Having a policy is not same as a hard block, you would want spending alerts and escalations and reports, but do you really want a hard block on crossing a limit ?
Admittedly I don't have experience working with very many devops /SRE teams, however I have never seen any enterprise vendor relationships were spend is hard capped for b2b sales with centrally managed procurement.
kuu
One thing I hate about the cloud providers is that there isn't an option to set a maximum cost. I would prefer to plug the cable of my side project than just receive an email saying me that next bill is going to be over my cost. I understand not everyone would like to do that, but I would like to have that option.
defaultname
Oracle has fantastic budget tools. Not just "you've passed your budget", but "you're forecast to pass your budget in 22 days before the month is up". And you can couple it with quotas to create hard budgets.
AWS has decent tools in this regard, but it pales compared to Oracle. Azure is a product I've never used with any scale (just small projects), but the fact that it actually costs money to setup alerts is gross (and morally reprehensible). Even if it's a trivial amount, that alone just sours the product in my eyes. I mean, already Azure is pretty uncompetitive unless you're running on free credits, as Troy apparently is (purportedly some $13K per year, so unsure what the pitch for donations to cover a bill is about).
schemescape
This piqued my interest, but a few quick searches (using a search engine--the Oracle Cloud site search only turned up press releases...), indicate that quotas just prevent you from spinning up new instances. That's helpful, but I was hoping for some sort of way to cap my bill (for hobby projects), even if that requries deleting resources.
Oracle Cloud has an enticing free tier, but I'm too afraid to use it because it requires a credit card and I don't see any way to put a monthly cap on my budget. (I'm sure hobby projects with ~$5 - 10/month budgets isn't their target market, but I can dream :)
Edit to add the page I was reading: https://docs.oracle.com/en/cloud/get-started/subscriptions-c...
cma
They'd rather refund small guys for mistakes than give big guys an easy limit to set.
kuu
I guess big guys don't want they service to suddenly stop, so they probably would not use this... But it's just a guess
electroly
Absolutely that. Storage costs money, so in order to absolutely cap your spending they would have to delete all your stored data, too. Deleting S3 buckets and EBS volumes on a spending blip is absolutely the last thing any company with any budget at all wants to happen, ever. It would be preferable for that not to even be possible in any situation. This is the sort of thing that only extremely small casual users want, and it isn't worth it to AWS to cater to those users. For everyone else, more complexity than a "kill everything at $X" switch is needed, and that's exactly what we do have. We don't get to absolutely cap our spending to the penny but we also don't risk having our data vanish because of a billing issue.
cma
For background batch jobs and analytics etc. they might want caps. Say something like a video transcoding workload. And lots of things could benefit not from a cap, but some kind of gradual degradation in bandwidth/instance allocation + a warning so you can raise the limits, it doesn't have to just shut everything down immediately using a hard cap.
frameset
But there is an option. In Azure you can "set a budget". He even goes over it in the post. Did you read the linked article?
cdmckay
It would be really classy if MS forgave that debt, especially considering the service is a public benefit.
kelsolaar
I would go as far as saying that the hosting for such a service should be entirely sponsored by Microsoft.
lodovic
He's a "Microsoft Regional Director and MVP" so Microsoft pays the bill one way or another. I expect that he has reduced Azure rates as well.
akoeplinger
Regional Directors and MVPs aren't employed by Microsoft: https://rd.microsoft.com/en-us/about/
anothernewdude
Would be even classier if the major cloud providers responded to customers calling out for budget limits for the past decade. Not many people want to risk potentially infinite costs.
Get the top HN stories in your inbox every day.
Don't put Cloudflare in front of a Cloud egress bill. i.e. don't do this: Azure|Amazon > Cloudflare
Always use your own proxy where the egress is well within your free tier, i.e. do this: Azure|Amazon > Hetzner|Linode > Cloudflare
Why?
Because Cloudflare cache is a massively multi-tenant LRU cache and whilst hot files will be cached well (and with Cloudflare Tiered Cache even better - but this itself is a cost) anything else is still going to expose you to some degree of egress cost.
When I exposed AWS to the web I paid $3k per month to AWS. With Cloudflare in front of AWS I paid $300 per month to AWS. With Linode in front of AWS and behind Cloudflare I paid $20 per month to Linode and about $12 per month to AWS.
A Linode, Hetzner instance... or any other dumb cheap web server that comes with a healthy free tier of bandwidth is all you need to set up a simple nginx reverse proxy and have it cache things to disk https://docs.nginx.com/nginx/admin-guide/content-cache/conte...