ffhein
- 1 Post
- 2 Comments
Joined 1Y ago
Cake day: Jun 14, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
- ffhein to
English - •
- 1Y
- •
Maybe I'm using the wrong terms, but what I'm wondering is if people are running services at home that they've made accessible from the internet. I.e. not open to the public, only so that they can use their own services from anywhere.
I'm paranoid a f when it comes to our home server, and even as a fairly experienced Linux user and programmer I don't trust myself when it comes to computer security. However, it would be very convenient if my wife and I could access our self-hosted services when away from home. Or perhaps even make an album public and share a link with a few friends (e.g. Nextcloud, but I haven't set that up yet).
Currently all our services run in docker containers, with separate user accounts, but I wouldn't trust that to be 100% safe. Is there some kind of idiot proof way to expose one of the services to the internet without risking the integrity of the whole server in case it somehow gets compromised?
How are the rest of you reasoning about security? Renting a VPS for anything exposed? Using some kind of VPN to connect your phones to home network? Would you trust something like Nextcloud over HTTPS to never get hacked?
For LLMs it entirely depends on what size models you want to use and how fast you want it to run. Since there’s diminishing returns to increasing model sizes, i.e. a 14B model isn’t twice as good as a 7B model, the best bang for the buck will be achieved with the smallest model you think has acceptable quality. And if you think generation speeds of around 1 token/second are acceptable, you’ll probably get more value for money using partial offloading.
If your answer is “I don’t know what models I want to run” then a second-hand RTX3090 is probably your best bet. If you want to run larger models, building a rig with multiple (used) RTX3090 is probably still the cheapest way to do it.