I’ve been a Linux enthusiast since ~2006, but I still have gaps in my knowledge and I would not consider myself a “fully-competent” Linux server admin at this point in time. I have to read a lot and ask a lot of questions to figure out things more knowledgeable users may do in their sleep. I’m gonna call myself “begintermediate”.
I’m working on simplifying my storage, backups, and general digital hygiene. I have multiple devices split across two locations and I end up having to use hard drives to periodically move files back over to my main desktop for sorting and archiving. If I want to access older files, I have to copy them from my main storage on the desktop to a hard drive, my NextCloud, or whatever device I want to access them on. I would like to avoid this drudgery by moving my file storage to a NAS (don’t really even need access outside the network, though it could be useful if I understood it enough to keep it secure). I also hope to simplify by backups in some way because currently all my devices just back up to a different pair of portable drives one of which I hand-carry offsite.
Requirements:
4TB+ storage to start
Expandability, I don’t know how storage needs will change over time, but 32TB seems like a fair upper end before wanting to update the whole system.
Would like to be able to run a few docker images for things like media server, open project, restyaboard, etc. I’m not sure if it makes sense to do this on the NAS or just get a simple NAS and do this stuff in a VM on my laptop or with a Rasberry Pie.
I don’t particularly want to spend more than $600 to get started, but wouldn’t mind having empty bays for later as I currently don’t have too much data.
Usage:
1-4 TB (someday up to 32TB) of files (docs, books, photos, videos, device backups, configs & code snippets, etc.)
Video, Photo, Music Access via Android Devices
Video and Photo access via a media portal (like plex or open media vault)
Would consider moving nextcloud here (currently on the public cloud) if uplink is fast enough.
Some sort of access via iDevice would be nice in case I want to give another some storage space.
Questions:
Does it make sense to mix my uses, i.e. media server, open project, etc. co-existing with file server for my docs and general files. Can I segregate portions for only local access?
I don’t have tons of time to maintain this. Nextcloud hasn’t been a pain, I log in here and there and make sure everything is updated (nextcloud and the server) and I run the NextCloud security scan to make sure I get an A+. Does it make sense to go for something like the better Synology NASs that can run docker images or would it provide better affordability/functionality to use a mini-pc or a FBmarketplace/craigslist slim pc hooked up to a drive enclosure or something else frankenstein-y. I don’t mind doing basic maintenance, but I can’t afford to spend every other weekend rebuilding things.
I have a dead WD MyBook Live and MyBook Cloud on my shelf. WD never updated them to fix the critical security issues, I missed the 40% off upgrade window, and they’re not safe to run with network access. They also sucked even when they were new. I want to avoid products doomed to become dead-end abandonware before I’m ready to upgrade. Are there NAS brands that are known to be better/worse with this? How does homemade NAS fare as far as hardware support and having to upgrade/rebuild when OS versions change.
Can I purchase/build a simple NAS that I use for storage and serve the files for my media server through a different device like my laptop? Is this better/worse than just streaming from the NAS itself or will I not notice in most cases?
It sounds like some of the pre-built machines can use drives of different sizes which would allow me to re-use the barely used drives inside of the WD devices. Do any of the self-build solutions allow for this.
I would LOVE some book/media/community recommendations for digital hygiene and how to handle store, backup, maintain the deluge of information in our modern lives.
All in all, I would appreciate any insight on a solution that gives a good balance between features & configuration, affordability. and maintenance time-investment. I figure a community of enthusiasts is a better place to learn than marketing copy.
Does it make sense to mix my uses, i.e. media server, open project, etc. co-existing with file server for my docs and general files. Can I segregate portions for only local access?
Yes and yes.
I don’t have tons of time to maintain this. Nextcloud hasn’t been a pain, I log in here and there and make sure everything is updated (nextcloud and the server) and I run the NextCloud security scan to make sure I get an A+. Does it make sense to go for something like the better Synology NASs that can run docker images or would it provide better affordability/functionality to use a mini-pc or a FBmarketplace/craigslist slim pc hooked up to a drive enclosure or something else frankenstein-y. I don’t mind doing basic maintenance, but I can’t afford to spend every other weekend rebuilding things.
Most of the time, setting up for the first time and tweaking for maybe couple weeks top is a pita. I have not spent more than an hour a month after the initial setup phase. Realistically, you wouldn’t need more than what I spent.
I had one time that my OMV fucked up my docker containers but spent about 30 min and it was back to normal. All the other problems were network problems which will happen even if I used Synology.
I have a dead WD MyBook Live and MyBook Cloud on my shelf. WD never updated them to fix the critical security issues, I missed the 40% off upgrade window, and they’re not safe to run with network access. They also sucked even when they were new. I want to avoid products doomed to become dead-end abandonware before I’m ready to upgrade. Are there NAS brands that are known to be better/worse with this? How does homemade NAS fare as far as hardware support and having to upgrade/rebuild when OS versions change.
In the end, if you go with the premade NAS, you will want Synology. There are no real competitors in the market atm.
Homemade NAS has been constantly getting better. Trunas is amazing. OMV is nice although I’m using it just because I’m too lazy to upgrade to Truenas.
I can’t say anything about other NAS brands but homemade NAS is going to get basically the best and the fastest updates there is, depending on the OS of uour choice. They will be painless for most of the time.
Can I purchase/build a simple NAS that I use for storage and serve the files for my media server through a different device like my laptop? Is this better/worse than just streaming from the NAS itself or will I not notice in most cases?
If you’re using homemade NAS, that will be more than enough. If you’re using something like Synology, you’ll be fine but also want something better depending on your usecase.
It sounds like some of the pre-built machines can use drives of different sizes which would allow me to re-use the barely used drives inside of the WD devices. Do any of the self-build solutions allow for this
Check out Unraid. I personally would not do that though.
I would LOVE some book/media/community recommendations for digital hygiene and how to handle store, backup, maintain the deluge of information in our modern lives.
I had been self-hosting stuff on my QNAP NAS for years before it died due to the infamous Intel clock drift issue and now I am in the process of making a DIY NAS (last few parts are coming in this weekend). I don’t have answers to all your questions but I’ll try my best with the experience that I have.
It is absolutely possible to mix your usecases on one machine, with the caveat that if you’re running on less-powerful hardware (like a off-the-shelf NAS), some of your services might be competing with each other for resources. CPU usage and disk access times (especially with a RAID 5 HDD array) can all impact performance. My QNAP NAS did start to bog down a few times with both Jellyfin and Nextcloud running at full tilt, but it was generally pretty usable.
Most NAS products support docker images so I wouldn’t worry too much about NAS vs PC in this case. Also, docker-compose is your friend. Write your yaml file once and it will make for easy setup and upgrading.
Dude, I am with you on dead-end products. The death of my QNAP NAS has caused me lots of headache and I basically swore off products that I can’t upgrade and fix myself. The problem is price. The cheapest x86 PC that I personally think will handle multiple usecases (media server, Nextcloud, SAMBA, maybe a Valheim server or a VM when I need it) costs roughly around $650-$750 depending on your build. You can probably find a Synology or QNAP NAS for about $500-$550. Granted, they most likely aren’t going to be anywhere near as powerful as a DIY x86 PC, so I think its worth going the DIY route. Those prices do NOT include the drives either, so be sure to factor that into your calculation. If you’re curious, here’s one of the cheaper builds I was considering building: https://pcpartpicker.com/list/rtqDbK. Ultimately I decided to go for a crazier build because I did not want slow HDDs anymore: https://pcpartpicker.com/list/Lm92Kp
You mean running a media server on your laptop, but pointing the media libraries to a Samba share on a NAS? I did that for years with my QNAP NAS and a little Intel NUC running Plex. The only issue is that you won’t get incremental media library updates whenever you add new files into the Samba folder. Usually, Plex (and Jellyfin) can detect file changes if the media library is local and automatically process only those files instead of rescanning the entire media library. Over Samba, there’s no such automatic detection so whenever you add a file, you have to manually trigger a full rescan in order for it to pop up in your media library.
I believe Unraid does this. I have not tried it myself and I plan on going with ZFS for my DIY NAS.
I don’t have any resource recommendations, but personally I’ve taken the docker-compose approach which helps quite a bit for isolation. For media servers, you only need to give read-only access to the volumes hosting your media storage. It is also recommended to put media servers like Jellyfin behind a reverse Nginx proxy because Nginx has been battle-tested in terms of security and Jellyfin’s web server has not. You can use docker-compose to easily spin up a Nginx proxy alongside your media server and have them contained in their own isolated network.
Do not open any more ports than is necessary to host your services. This means even remote administration should not be available via your public IP. Learn how to setup Wireguard so that if you’re away from home, you can quickly VPN into your network and do remote administration. If you’re using SSH, make sure you disable password authentication and only rely on SSH keys. I am sure other people can add more, this is just the basics.
I have multiple devices split across two locations and I end up having to use hard drives to periodically move files back over to my main desktop for sorting and archiving. If I want to access older files, I have to copy them from my main storage on the desktop to a hard drive, my NextCloud, or whatever device I want to access them on. I would like to avoid this drudgery by moving my file storage to a NAS
A NAS is a good idea but do note that this sort of setup can work aswell with the correct tooling.
don’t really even need access outside the network, though it could be useful if I understood it enough to keep it secure
I can highly recommend Tailscale for this purpose.
run a few docker images for things like media server, open project, restyaboard, etc. I’m not sure if it makes sense to do this on the NAS or just get a simple NAS and do this stuff in a VM on my laptop or with a Rasberry Pie.
Depends. Many people host such things on their NAS since the NAS is always on anyways and barely does anything most of the time, so it’s perfectly valid to do that.
Can I purchase/build a simple NAS that I use for storage and serve the files for my media server through a different device like my laptop?
Yes but in a home setting, it usually makes sense to keep the services running on the same device which stores the service’s data.
It sounds like some of the pre-built machines can use drives of different sizes which would allow me to re-use the barely used drives inside of the WD devices. Do any of the self-build solutions allow for this.
Sure. Unless you need assistance setting up a Linux system (I doubt you do) or building a computer, self-built is almost always better. I use a low-power Intel single-board-computer (Celeron J4105) in a small PC case for this purpose.
For pooling different sized drives, I use btrfs but the same could be achieved using ZFS or even LVM.
Do note that, unless you have specific uptime requirements, RAID is pretty wasteful in a home setting in both hardware and time thinking about it.
When it comes to digital hygiene, figure out backups first. 3-2-1!
I would LOVE some book/media/community recommendations for digital hygiene and how to handle store, backup, maintain the deluge of information in our modern lives.
I’ve found git-annex for myself. It’s quite a rabbit hole and takes a lot of effort to understand and really use well but it’s an incredible tool that has greatly aided simplifying my storage setup.
One of the best things about it is that it separates data from metadata. You always have the metadata but you don’t need to hold the data all in one place.
This means you can re-organise files on your laptop without those files actually being present on the laptop. They could be anywhere; on a hard disk sitting unplugged on a shelf, in the cloud, on some other machine that’s turned off, on the NAS etc. but you can move them around in the filesystem as if they were there. If you needed some file’s content content, you can ask git-annex where the file’s content is stored, i.e. plug in the hard drive and then ask it to copy the data over.
That’s the gist of it (git-annex can do a bunch of other cool stuff) but this really helped my get my shit together w.r.t. storage hygiene.
I found that running just a NAS is no good because it lacks performance. And running just a server is no good because of data safety. So I run both.
I run a Synology NAS for storage, and an Ubuntu Box for services (Paperless, Mattermost chat, Photoprism gallery, Gitea for Code, picoshare, and lots other small stuff). All services run on Docker (plus Portainer for handling them, plus NPM for certificate handling and subdomains).
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
Yes and yes.
Most of the time, setting up for the first time and tweaking for maybe couple weeks top is a pita. I have not spent more than an hour a month after the initial setup phase. Realistically, you wouldn’t need more than what I spent.
I had one time that my OMV fucked up my docker containers but spent about 30 min and it was back to normal. All the other problems were network problems which will happen even if I used Synology.
In the end, if you go with the premade NAS, you will want Synology. There are no real competitors in the market atm.
Homemade NAS has been constantly getting better. Trunas is amazing. OMV is nice although I’m using it just because I’m too lazy to upgrade to Truenas.
I can’t say anything about other NAS brands but homemade NAS is going to get basically the best and the fastest updates there is, depending on the OS of uour choice. They will be painless for most of the time.
If you’re using homemade NAS, that will be more than enough. If you’re using something like Synology, you’ll be fine but also want something better depending on your usecase.
Check out Unraid. I personally would not do that though.
Ask here!
Feel free to ask more/correct me.
I had been self-hosting stuff on my QNAP NAS for years before it died due to the infamous Intel clock drift issue and now I am in the process of making a DIY NAS (last few parts are coming in this weekend). I don’t have answers to all your questions but I’ll try my best with the experience that I have.
It is absolutely possible to mix your usecases on one machine, with the caveat that if you’re running on less-powerful hardware (like a off-the-shelf NAS), some of your services might be competing with each other for resources. CPU usage and disk access times (especially with a RAID 5 HDD array) can all impact performance. My QNAP NAS did start to bog down a few times with both Jellyfin and Nextcloud running at full tilt, but it was generally pretty usable.
Most NAS products support docker images so I wouldn’t worry too much about NAS vs PC in this case. Also,
docker-composeis your friend. Write your yaml file once and it will make for easy setup and upgrading.Dude, I am with you on dead-end products. The death of my QNAP NAS has caused me lots of headache and I basically swore off products that I can’t upgrade and fix myself. The problem is price. The cheapest x86 PC that I personally think will handle multiple usecases (media server, Nextcloud, SAMBA, maybe a Valheim server or a VM when I need it) costs roughly around $650-$750 depending on your build. You can probably find a Synology or QNAP NAS for about $500-$550. Granted, they most likely aren’t going to be anywhere near as powerful as a DIY x86 PC, so I think its worth going the DIY route. Those prices do NOT include the drives either, so be sure to factor that into your calculation. If you’re curious, here’s one of the cheaper builds I was considering building: https://pcpartpicker.com/list/rtqDbK. Ultimately I decided to go for a crazier build because I did not want slow HDDs anymore: https://pcpartpicker.com/list/Lm92Kp
You mean running a media server on your laptop, but pointing the media libraries to a Samba share on a NAS? I did that for years with my QNAP NAS and a little Intel NUC running Plex. The only issue is that you won’t get incremental media library updates whenever you add new files into the Samba folder. Usually, Plex (and Jellyfin) can detect file changes if the media library is local and automatically process only those files instead of rescanning the entire media library. Over Samba, there’s no such automatic detection so whenever you add a file, you have to manually trigger a full rescan in order for it to pop up in your media library.
I believe Unraid does this. I have not tried it myself and I plan on going with ZFS for my DIY NAS.
I don’t have any resource recommendations, but personally I’ve taken the
docker-composeapproach which helps quite a bit for isolation. For media servers, you only need to give read-only access to the volumes hosting your media storage. It is also recommended to put media servers like Jellyfin behind a reverse Nginx proxy because Nginx has been battle-tested in terms of security and Jellyfin’s web server has not. You can usedocker-composeto easily spin up a Nginx proxy alongside your media server and have them contained in their own isolated network.Do not open any more ports than is necessary to host your services. This means even remote administration should not be available via your public IP. Learn how to setup Wireguard so that if you’re away from home, you can quickly VPN into your network and do remote administration. If you’re using SSH, make sure you disable password authentication and only rely on SSH keys. I am sure other people can add more, this is just the basics.
Hope this helps!
A NAS is a good idea but do note that this sort of setup can work aswell with the correct tooling.
I can highly recommend Tailscale for this purpose.
Depends. Many people host such things on their NAS since the NAS is always on anyways and barely does anything most of the time, so it’s perfectly valid to do that.
Yes but in a home setting, it usually makes sense to keep the services running on the same device which stores the service’s data.
Sure. Unless you need assistance setting up a Linux system (I doubt you do) or building a computer, self-built is almost always better. I use a low-power Intel single-board-computer (Celeron J4105) in a small PC case for this purpose.
For pooling different sized drives, I use btrfs but the same could be achieved using ZFS or even LVM.
Do note that, unless you have specific uptime requirements, RAID is pretty wasteful in a home setting in both hardware and time thinking about it.
When it comes to digital hygiene, figure out backups first. 3-2-1!
I’ve found git-annex for myself. It’s quite a rabbit hole and takes a lot of effort to understand and really use well but it’s an incredible tool that has greatly aided simplifying my storage setup.
One of the best things about it is that it separates data from metadata. You always have the metadata but you don’t need to hold the data all in one place.
This means you can re-organise files on your laptop without those files actually being present on the laptop. They could be anywhere; on a hard disk sitting unplugged on a shelf, in the cloud, on some other machine that’s turned off, on the NAS etc. but you can move them around in the filesystem as if they were there. If you needed some file’s content content, you can ask git-annex where the file’s content is stored, i.e. plug in the hard drive and then ask it to copy the data over.
That’s the gist of it (git-annex can do a bunch of other cool stuff) but this really helped my get my shit together w.r.t. storage hygiene.
I found that running just a NAS is no good because it lacks performance. And running just a server is no good because of data safety. So I run both.
I run a Synology NAS for storage, and an Ubuntu Box for services (Paperless, Mattermost chat, Photoprism gallery, Gitea for Code, picoshare, and lots other small stuff). All services run on Docker (plus Portainer for handling them, plus NPM for certificate handling and subdomains).