• 0 Posts
  • 7 Comments
Joined 1Y ago
cake
Cake day: Oct 11, 2023

help-circle
rss

A VPS with fail2ban is all you need really. Oh and don’t make ssh accounts where the username is the password. That’s what I did once, but the hackers were nice, they closed the hole and then just used it to run a irc client because the network and host was so stable.

Found out by accident, too bad they left their irc username and pw in cleartext. Was a fun week or so messing around with their channels


Currently seeing this up at work. Ollama with danswer as a rag / frontend. On a bunch of Nvidia L4 /L40 on kubernets

Is pretty plug and play honestly.


There’s a few.

Very easy if you set it up with Docker.

Best is probably just ollama and use danswer as a frontend. Danswer will do all the RAG stuff for you. Like managing / uploading documents and so on

Ollama is becoming the standard selfnhosted LLM. And you can add any models you want / can fit.

https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image

https://docs.danswer.dev/quickstart


Depends on what you want to self host? Could be worth it to see if what you self host can be deployed as containers instead


Minikube and try to get everything on Kubernetes?