I call this "blast radius". Earlier this year during the Claw hype I was reading about all kinds of elaborate schemes to prevent the agent from getting API keys.
I realized, what am I actually afraid of. Well, overspend. So I just set then all to disable auto-reloading. Now if it blows up, I'm down $5.
Same story with containers. Just give it root on a VPS, and if it blows up, I'm down $3.
I used to work on a support team of a well known backend type service that had hard budget caps.
It was, unfortunately, a nightmare. There were tons of tickets and even threats of lawsuits from customers whose service got cut off hard at the worst possible time due to organic growth/going viral/big event/nobody knew about the limit/etc. Not only did they lose all the leads and revenue they would have gotten from that bump, but they also pissed off their own existing users who suddenly couldn't use the service either.
Generally speaking, it's much better to use alerts instead of hard limit. Even in the worst case (hackers pwn your credentials and mine Bitcoin or whatever) the rest of your business is unaffected and you can negotiate with the billing department at comparative leisure.
This is all assuming you have humans operating the service. If you're letting AI agents yolo infra in prod, you have a whole series of new problems.
That doesn't mean you couldn't have a service which by default has no hard cap, and people have to opt into it. You could even put a user interface thing where people have to type a whole sentence perfectly matching and hit OK, like "I understand that enabling a hard billing cap will shut off services if it exceeds my monthly quota". Wrap it in as much service agreement contract, TOS language as is necessary.
Heck, have it do the equivalent of send people a DocuSign equivalent PDF to sign acknowledging the risk before enabling it. Would it still stop pissed off people? Probably not. Would it help with the risk of lawsuits, very possibly.
Yes, this is an important point although it has changed with A.I. Software is traditionally very high margin and so a $10k bill can be written off by the provider without any meaningful loss.
As a customer, the big number is scary and causes panic but for the provider⌠customers constantly fail to pay bills, providers are constantly writing off bills because it just isnât worth the cost to chase, if a customer says âhey that usage was a mistakeâ itâs usually worth it to write it off to save the relationship. If you write off a big bill that wouldnât have been paid anyway, the customer will perceive you as wonderful and benevolent and be loyal for life when they are ready to spend their money.
With tokens though the actual cost being incurred is much, much higher. If your service is just a wrapper around tokens, and a customer incurs $10k of usage that you paid OpenAI $5k for, it becomes much more difficult to write off.
Google Cloud is one of the few services that actually pursues unpaid bills even on their high margin services.
Google Cloud is also the scariest because of how much damage it can do and how bad their payment system can be.
I recently loaded up on prepaid api credits for gemini and it somehow triggered some billing shenanigans in my linked accounts where it said I had a negative balance (from the credits), and they were going to discontinue my services. I had to reset some settings to sort it out, mainly using their chat ai and mine (because theirs gave me right status info, but wrong conclusions).
It's pretty messy across like aistudio.google.com, and their typical console, and google workspace business account. I'd be so fucked if they froze my account, I'd rather just pay openrouter to access credits in the future.
> Google Cloud is also the scariest because of how much damage it can do
Google is the only cloud platform I'll not just never use, but personally discourage anyone from using it.
Simply because there have been way too many horror stories on here about people who had gotten their personal gmail accounts frozen for whatever BS reason - and absolutely zero recourse. With any other large service you can always get ahold of a human, with anything tied to Google it's impossible and even raising a major stink on HN or "legacy media" often does not help.
I think it's not generally 'much better' to use alerts. People - end users - are by now quite used to seeing things go down for a while. No biggie. But a infra oopsie can kill a company in ways a short outage won't.
And its of course not just people yolo'ing with AI. People were quite capable of causing such outages themselves just fine. Distributed, serverless systems are hard.
> I think it's not generally 'much better' to use alerts.
Anyone who says its 'much better' to use alerts instead of hard caps needs to Google 'alert fatigue'.
Alerts are soft. You ignore them or miss them, nothing happens except you spending $$$$$$$$$$ more.
Great if you're the cloud provider raking in the cash, but a poor way to run your infrastructure.
Hard caps force you to implement correctly (to control costs in the first place) and have correct monitoring in place (to keep a healthy cap buffer).
So it means you can't just vibecode some slop and blindly devops it via Github CI/CD. You actually need to think and reason about your infrastructure.
> you can negotiate with the billing department at comparative leisure.
I'm curious. How likely is the billing department to waive off a huge bill as bad debt because an inexperienced builder misconfigured their infra or was hacked?
Not the OP but run a high margin service that has customers run up accidental bills often. Customers running up bills intentionally and then not paying is even more common. There is almost no situation where trying to force a customer to pay makes sense, we write off any amount without question. The goodwill is worth it every time. Most SaaS companies donât even have the processes in place for debt collection anyway.
When I first started working with cloud providers it was shocking to discover this feature doesnât exist, basically anywhere.
Itâs such a basic thing, not having it has to be deliberate to make you accidentally spend more than youâd like.
>Itâs such a basic thing, not having it has to be deliberate to make you accidentally spend more than youâd like.
This is unlikely. The big cloud providers routinely cancel bills based on accidental usage. The real explanation is just that it's technically difficult to calculate all usage in real time and shut down systems immediately as soon as a certain cap is reached.
These shouldn't even exist without a negotiated contract.
I can subscribe to your service for a specific fee on a monthly basis ($20/month say), and take the risk of losing that month's fee if your service or I make mistakes, or I can choose to drop another $20 mid-month, or anything for convenience.
Saying that the computer will "control" the billing and can run haywire tells me that I don't want to be anywhere near your pile of bad incentives.
Monthly electricity bills are based on usage, and it works well but thereâs a limit to how surprising a bill can be. The difference is the relative orders of magnitude you can be charged for these services you can go from 20$/month to 200k/month without warning.
There is also a hard physical limit on how much electricity you can use before you blow out the fuse box
Also, people don't casually swing by my house and start using my electricity.
So my powerbills are predictable.
Whereas traffic spikes to websites are not.
This age of abusive AI crawlers and the non-revenue generating traffic has been a very real problem for me!
> but thereâs a limit to how surprising a bill can be
I think they explicitly said that.
Hard budget caps are readily available in Europe.
And everywhere in Europe you have clear price sheets (unlike the deliberately opaque mess of the US price sheets with more small-print than a packet of pills), which means even if you are at an EU provider with no hard caps you can still accurately reason and predict your costs.
Just a few examples....
Cloud providers:
- Upcloud
- Exoscale
- Verda
- PrivateMode
The "viral" bit is easily solved through basic monitoring of metrics that everybody should be doing. I believe the cool-kids give it the fancy name of Site Reliability Engineering (SRE). All you need to do is top-up your balance / adjust your cap if your metrics are trending upwards for an explainable reason. Its not rocket science.
As for the "too difficult" that's just a lie. It just suits the US cloud providers better to have you spend spend spend on their messy soup of random interdependent microservices.
Why don't I have the ability to delete my payment details from Codex or Claude? That way I cannot be billed more until my subscription is over.
Turns out it's impossible to do. You have to delete your entire account.
Network saturation is difficult. Even if you turn off the endpoint you can still saturate the network in between. And it's still bandwidth.
I actually think network ACL triggers based on billing might be the only way to really enforce this.
I witnessed a DDoS attack once that changed how I think about billing. It was locally provisioned hardware and the attackers had saturated the switches. Naively I said "just block the CIDRs" but the problem was the incoming ram is so saturated that it can't even get to the point of "deny" in the firmware.
So from a technical perspective if there's an internal DDoS at AWS what do you do? Do you turn off the endpoint? Do you drop the sources from hitting it at the router? And even that costs money. Anyway that incident gave me a different level of appreciation for this challenge.
Edit: this is mainly targeted at the people complaining why this took so long. At some point in scaling even telling you "no sorry" in a nice way is expensive. I'm sure recruiters can sympathize with this nowdays.
The only way to deal with those is to blackhole the traffic at BGP level.
The fancy alternative is to anycast that network and do distributed filtering so that the flood is manageable.
The first can be done by your ISP but it does lead to the temporary loss of traffic via that IP.
The second one is what DDoS mitigation services do and AWS has a basic one built in ( AWS Shield ) and several additional services you can get.
The challenge with cloud has always been the selling point of âinfinite scalingâ vs the flip side âinfinite billingâ.
Itâs not in their interest to make cost controls work well.
Ideally youâd be able to set something granular like âallow this service to scale up only 10x, measured at an hourly level, and alert me when it happens. Drop all requests that exceed 10xâ.
And then you are mostly in a throttling situation until the burst clears or a human can review & accept increased usage is ok/increase thresholds. Iâd rather have services go slow during excess load (like a real server) than go dark for remainder of month.
This seems a lot better than brute force âturn everything off at $X level of monthly billingâ or âno limits you can charge me infinity dollarsâ.
Get a daily email with the the top stories from Hacker News. No spam, unsubscribe at any time.