Hetzner GEX44 with Nvidia GPU
hetzner.comFYI do not run blockchain related software on Hetzner nodes, even their dedicated bare metal. They are likely to shut the host down.
I was running a Solana Testnet Proof of Stake validator for work and they decided to shut the host down at the network layer and refused to give us access to the host. After my employer's legal threatened them, they offered to give us access back, but to be a pain in the butt, they only gave us console login access. Luckily, it was just our Testnet validator, and we keep the keys in cold storage which we were able to recover from.
Their nodes are cheap, but beware of Hetzner. Something that they didn't mind hosting for almost 4 years they suddenly changed their minds on and shut the host down.
See: https://www.theblock.co/post/182283/1000-solana-validators-g...
Your fault for not reading their ToS:
https://www.hetzner.com/de/legal/terms-and-conditions/ number 8.1 in the German version, number 4 in the English one: "Furthermore, the operation of applications for mining cryptocurrencies is prohibited. This includes, but is not limited to, mining, farming and plotting of cryptocurrencies. In the case of non-compliance, we are entitled to lock your access to the Service and/or to your account."
Lots of hosters and cloud providers have this and similar exclusions, to keep out the typically shady cryptocurrency crowd, their excessive resource usage, high payment risk and reputation damage.
E.g. Google Cloud also doesn't allow cryptocurrency stuff without prior approval, 3.3 iv: https://cloud.google.com/terms , same for AWS, 1.25: https://aws.amazon.com/service-terms/ So I hope you got your written approval first, otherwise AWS might be equally nasty to you.
Their ToS did not reference blockchain at the time. This is why our lawyers got involved. This was updated after they caused an outage unannounced for a service running almost four years.
Google actually offers blockchain rpc. Fwiw, Solana nodes run poorly in the cloud and generally only make sense on bare metal. As far as slow distributed databases go (aka blockchains), it’s 1000x+ faster than the ethereum style blockchains.
Ah, if that is new, then you are right, that's probably a breach of contract on their side.
Cloud doesn't necessarily mean virtual machines with scheduling and sharing problems. You can get non-shared machines with almost all popular cloud platforms, e.g. https://cloud.google.com/compute/docs/compute-optimized-mach... Of course those aren't cheap.
Thanks for writing about this. Makes me like Hetzner even more.
I was just about to write that. No crypto scam nonsense in the same datacenter sounds like a plus to me.
(On a more serious note, I imagine the reason Hetzner does not want crypto-related stuff is for the same reasons most hosting companies are not very fond of porn-related sites: High rates of fraud, charge-backs and other potential legal liability).
They did officially stated on reddit [1] that they are not supporting crypto related hosting. Just wonder why? Is the GPU/CPU/Network related over use? Or legality related?
[1]: https://www.reddit.com/r/hetzner/comments/wucxs4/comment/ilf...
They stated that AFTER they just cut the network access to the hosts. It isn’t as though we were doing anything ethically or legally wrong. We were running rpc nodes for legit applications and a validator to secure the network.
For me, who’s worked in 50,000+ bare metal Linux environments before, it was just so surprising how sudden and violent it was. Their customer support was amazing until it wasn’t.
Just let it be a lesson, they will arbitrarily change their mind and put your nodes running your business at risk. Even if you’ve broken no laws and are operating in good faith. We were not running and proof of work nonsense that breaks the hardware or uses excess electricity.
> It isn’t as though we were doing anything ethically or legally wrong.
Legally? Well, they probably have a "if we think you do something we don't like we can kick you off" clause. Ethically? Eh .. Crypto. Ethical. Opinions differ.
I am not a big fan of crypto either, but don’t understand why crypto (the technology itself) would be unethical.
Could you explain?
I don't know any working thing using crypto in a way that it acutally make sense and couldn't be already be done easier and similiar safe.
Even in big companies you always see examples which break the technology like creating something on the blockchain which has a connection to the real world.
Try to send some big amount of money internationally and you will see why crypto is the future. With regular banks it takes days, weeks and many times months due to intermediate banks, AML questions, reviews, etc and have very high costs associated for simple transfers of money hardly earned.
With cryptocurrencies it takes seconds or minutes and no questions asked with very low fees
Energy use that doesn't deliver concrete value to the world. Mostly just speculators and gambling, but uses a a couple hundred TWh of electricity worldwide.
Yes yes I know, some applications are lower energy usage but parent comment is asking about the ethics of crypto in general.
> Energy use that doesn't deliver concrete value to the world
How is that any different from fast trading computers producing 'wealth' by juggling inflated derivatives and 'generating' water vapor value from hot air...
I don't think these are ethical either.
> Mostly just speculators and gambling, but uses a a couple hundred TWh of electricity worldwide.
Parent asked about ethics of crypto, the technology, in general. This comment isn't about the technology, but the applications, and doesn't answer their question.
I don't necessarily think we should separate applications from theory, but to be more crisp, I would probably say the facilitation of money laundering and funding of terrorist groups is something unethical.
Would you be shocked to know that US dollars are used more for both money laundering and funding terrorism than cryptocurrencies? Should we outlaw US dollars due to facilitating unethical uses?
I dont think so because of the centralized oversight. Crypto is designed to avoid that oversight (but even fails at that much of the time).
> I don't necessarily think we should separate applications from theory
Sure, not in general, but shaism was specifically asking for theory, and you replied to them without actually answering their question. And you still haven't.
> the facilitation of money laundering and funding of terrorist groups
...and this isn't unique to crypto, either, so it's unclear that it's relevant.
Wait until you find out what we do with all the water.
You're angry in the wrong direction. It is not their fault that you, or someone in your company, didn't read their TOS. Hetzners service is cheap, this attracts bad actors. In their eyes you are a bad actor and according to their TOS they're right. I, as a customer of Hetzner, are happy about how they deal with people abusing their services.
We read their ToS. Our lawyers read their ToS. It has a vague “we can kick you off if we choose to” part, but removing service with no recourse was not ok. So we told them we would sue them if they didn’t give us access to shred sensitive data and they relented.
This is the sentence, in Hetzners ToS, that you're referring to: "In the case of non-compliance, we are entitled to lock your access to the Service and/or to your account." Source: https://www.hetzner.com/de/legal/terms-and-conditions/#16
If your lawyers thought that this line means something different than "don't use crypto, mining, etc. we will lock you out", you should change/sue your lawyers. This is a clear as it gets, I see zero fault on Hetzners side.
Honestly hetzner was never the 'high end / high quality' cloud provider for me.
They have decend hardware and infra but no encryption on rest and everything else a real business requires (certifications etc.).
But i'm so dismissive about all shitty crypto because no one ever showed me a good use case, as others said, i think its great that hetzner has this tos.
-So what do you actually do?- forget about it, finance got it...
Many hosters banned all kind of crypto after advent of file coins, which wear out ssds too quicky.
Indeed. Some bad actors (which is most of the industry) ruin it for those of us who just want to build cool tech. Very annoying.
You know what they say, a bunch of bad apples spoils the few.
AFAIK it's the high disk IO causing wear on the SSDs, since they specifically mention Solana nodes
Likely. We put the accounts db in memory (on a tmpfs) to mitigate this and were paying them for a beefcake node with 1T of memory.
Wild guess: They use significantly more electricity than the average customer that does just about anything else with their hardware. The GPU can easily run at hundreds of watts, and when you add in the required cooling you're probably at 1.5-2x that in total power usage.
This is not a proof of work (bitcoin or pre merge ethereum) node and it does not use more electricity. They singled out solana seemingly and I’m unsure why. We moved about 300 services off of dedicated higher end Hetzner nodes within a week.
What implication does this have for AI workloads - the computation intensity would be similar? (Assuming a popular service saturating the machines)
I see some other comments mentioning ssd wear which I don't think would apply to AI stuff, but the utilisation/power consumption certainly would.
My guess is it's a huge liability if keys are stolen, and it puts a huge target on their back from people around the world who want to steal crypto keys or DDoS crypto systems.
Most of the Hetzner DdoS come from gaming fyi. They’re pretty good at handling them but we’d get paged every few months due to this.
Our uptime is better since we left hetzner. We went with a host that’s more commonly used for electronic trading firms, Teraswitch. They’re great.
Thanks for recommendation, we are looking to move our servers off Hetzner for precisely the same reasons.
They shut down my node, because of a "billing problem". (Card was unlocked for online payments, etc.) In response, I tried in vain to pay in advance, and in response my company is now blacklisted by them. It was for a new deployment, so no harm on my side, but it's kind of comical. I didn't even get around to run any services except for a web server with almost no traffic. (It was for a simple backend service, not for public consumption.)
Kind of comical!
Despite that I wouldn't have minded using them again, but never for something which I don't have redundancy with some other provider I can fall back to. AWS can go wrong too, but I think the risk of getting booted before even starting is very low with them.
They use to provide servers with Nvidia 1080 GPUs. Even though several generations of new GPUs appear, Hetzner didn't vend those new GPUs. Let's see what happens this time.
(Disclaimer: I love Hetzner and their business model/customer service.)
It is somewhat limited vs a RTX 4090 [1]
We rent a 4090 for about EUR300/month on vast.ai, for reference.
1. https://technical.city/en/video/GeForce-RTX-4090-vs-RTX-4000...
Is vast.ai reliable for production inference where downtime is important?
Dupe from two weeks ago: https://news.ycombinator.com/item?id=39440503
2208 EUR per year.
RTX™ 4000 SFF Ada Generation (20GB GDDR6 ECC)
64 GB DDR4
1.92 TB (Gen4)
Intel® Core™ i5-13500
30.9 TBit total bandwidth
Time used to setup and maintenance makes this valuable. Otherwise you can just buy similar system and pay for electricity.
If you have a use case where you utilize Hetzner's RTX4000 GPU 24/7/365 around the clock, then it's a great deal. However, I think many people will just load some quantized LLMs into it and call it a few dozen times per day. For the latter use case, €2,200 is a bit too much, I think.
While investing in bitcoins, be cautious. If your bitcoin gets stolen, then it can be a very frustrating time but instead of getting frustrated, it is important to act fast so that you can get your money back. Unluckily, there are many victims that do not get their money back because it is very hard to track down the people behind it. The thief is either in another country or using fake identities. But in some cases, it is easy to track the person who has stolen your bitcoin. Tracing bitcoin is not hard for the bitcoin forensics or specialists. For more information, kindly reach out to (davidmoore9951 (@) GMAIL COM
Can you game with these GPU (moonlight or parsec)?
And by "can" I mean:
- Does the ToS allows it?
- Does the GPU allows it?
Hetzner are great. They are 'special' and I love it
I can’t stop giving you positive reviews For recovering my stolen 4.3 Bitcoin in just few minutes. i lost my BTC in a bitcoin investment scam and Chris Harvey unbelievably recovered my coins back to my wallet in just few minutes. i believe we all need such a wonderful helper. contact him via email: chrisharvey553@gmail.com
Seems like the main memory for the system isn't ECC, and can't be configured to be ECC either? Ugh. :(
Some lady contacted me on IG and got me to trade with this company. Constantly convincing me for one reason to the other to keep increasing my deposits. Long story short, I lost about $88,000 I've never been so disappointed in my life, trusted them only to be deceived like this. Just another one of those shady companies that have nothing to offer. Wish I had read some comments in forums earlier, wouldn't have dealt with them. Glad I found a reliable expert at Henry Walter in some forums who helped me get back everything I lost. If you're also a victim of these guys or lost your money to similar companies, write me and I'll refer you to the Recovery Experts. I'll make sure you get back everything you lost Email: davidmoore9951 (@) GM AIL , C0M