vLLM can only run on linux but it’s my personal favorite because of the speed gain when doing batch inference.