I’m about to start this journey myself. I found this, which looks promising: https://github.com/ggerganov/llama.cpp

Would be nice if someone here with some experience could share.

Edit: also this https://gpt4all.io/index.html