For now my server doesn’t have very important data most of it are your “Linux isos” I can just download again and I’m thinking of starting to move my file and photos to the server but in afraid. What if I get a ransomwarei don’t realize and all my backups get encrypted too? Or if the backups are corrupted and my disks breaks? But also I’m afraid about cloud because I’ve seen some posts about people getting their google accounts closed without notice for breaking TOS (maybe they did something wrong maybe not).
It is impossible to fully eliminate the risk but with a decent backup system in place it is somewhat unlikely to lose all of your data.
The 321rule should be used as a baseline. Your local backup should be snapshotted and somewhat hardened against ransomware (pull backups instead of pushing them, do not mount the backup volume to other machines). Cold backups also help.
Can I construct scenarios in which I lose all my stuff? Sure. But in those, we are either in deep shit anyway (CME, some big astroid) or it is pretty unlikely (targeted hacking)
I’m too poor for a 321 lol
My backups are tiered. Some stuff gets no backup at all, some gets even more than 3.And I tend to reuse HDDs that got replaced in my main machine due to size for my backups. Power consumption hardly matters when it only runs for a few minutes a day.
me too
How much value does the data have for you?
If it’s of very low value, that it doesn’t even justify the costs of doing proper backups, then it’s not so important to worry about it either.
This. With a proper backup strategy, you are reducing the probability of a catastrophic sequence of events. It becomes P(some unlikely event) x P(some other unlikely event) x … Etc. for as many events you can think of and/or can afford to mitigate.
As you say, the risk will never be zero. And even the best-laid plans can fail - the Gitlab incident a few years back saw five layers of backups and disaster preparedness fail.
Really, all you can do is backup your data using standard methods, and TEST THE RESTORE before you need to rely on it!
I’m more scared of online services being discontinued and/or being getting vendor locked and forced to pay ransom on a regular basis. Therefore, I host and back up everything on my own.
A) Make backups B) take them offline.
Aren’t you scared about loosing your data?
No. I still have files from 1991. I’ve got files that have migrated from floppy disk to hard drive to QIC-80 tape to PD (Phase Change) optical disk to CD-RW to DVD+RW and now back to hard drives.
What if I get a ransomwarei don’t realize and all my backups get encrypted too?
Then you need to detect the ransomware before you backup. I use rsync --dry-run and look at what WOULD change before I run it for real. If I see thousands of files change that I did not expect then I would not run the backup and investigate what changed before running the rsync command for real.
Or if the backups are corrupted
I have 3 copies of my data. Local file server, local backup, remote file server.
I also run rsnapshot on /home every hour to another drive in the machine. I also run snapraid sync to dual parity drives in the system once a day.
I generate and compare stored file checksums twice a year across all 3 copies to detect any corruption. Over 300TB I have about 1 failed checksum every 2 years.
and my disks breaks?
If one of my disks breaks I buy a new one and restore from backups.
But also I’m afraid about cloud
I don’t use any cloud services because I don’t trust them.
About rsync --dry-run, let’s say I got a ransonware but its till encrypting the data will it detect the changes?
Follow the 3-2-1 principle and there is less reasons being scared of loosing data.
losing
Some data are backed up to a local NAS, some of that data is backed up to cloud (not Google or the big ones).
Most of my data aren’t important. Photo library is both local, in the cloud, and most on offsite DVDs.
~45K lossless music files is local and cloud. Those would suck losing, but I could rip them again.
I’ve been considering tape backup again, it’s like 20 years since I used it at home.
I loose my data all over the place.
I loose it here, I loose it there, I loose it all over your mother’s hair.
As far as losing data, I trust myself and my adherence to 3-2-1 more than I trust a third party solution.
Paranoia is the reason I self host. Clouds can kick you out or lose your data at any time.
ZFS (mirrored) two HDDs. If one HDD fails, then replace it and let it rebuild. Use 3 HDDs mirrored if you really think you could get a failure while the array is rebuilding.
Also have two external backups, one you do regularly at home, and another you keep off-site. When you visit that location (be it your parents, siblings, relatives, friends house) swap out your external backup with their off-site to ensure its kept up to date.
Make sure all disks are fully encrypted of course.
I backup to Backblaze b2. I encrypt myself using rclone. Costing me $1-2/mo for about 100Gb that I’m currently using.
API key I use for automated backups is pretty much limited to write only and files are set to hidden when deleted, so not much risk, just an annoyance, if the key were stolen and they defaced my backups.
Once a year I might go delete some history to reduce my usage.
I lean towards scripts to automate setting up a system, so I don’t do full system backups. Downloaded video I also mostly skip using mirrored storage. In the event of a real disaster, its an acceptable loss.
Not as many years ago as I would prefer given my professional experience, I was running a lot from home. Most of it for myself to learn more (so nothing my home itself was dependent on) but a percentage of it was for storage of pictures, home videos, digitalized documents, emails, etc.
I ran my own exchange server for years (utilizing my own TLD that I bought in 2008).
I was in the process of migrating data from a couple of older hosts to the newer ones I had setup in the garage; basically from two cobbled together Dell T series poweredge servers in my hall closet to a small stack of R series poweredge in a 42u cabinet rack in the garage.
My whole stack was setup across the two hosts including backups from veeam from one to the other and copies.ofnthe backups stored on an external. Due to the size of the backups and where I was on my life financially, anything hosted up in cloud space was just a little out of my budget. Anything I could afford was suspicious at the time.
This too long story ends basically by me not paying attention to what I was doing and ended up destroying the raid on both of the original hosts without having finished moving all my data.
I lost years of emails from my exchange server, all the pictures and home made videos of my daughters life from birth to that point in time, and my backup data.
All from my own mistake(s).
I did everything I was “supposed” to do to keep stuff protected until I messed it up.
My daughter passed away this year in February at 16 years old. I’d give anything to have those pictures and videos back.
My point is, you can plan and execute and throw money at it if you’re able. And you’ll likely be fortunate enough to never really lose anything that’s valuable to you. But even planning and implementing, you can still lose stuff just by oversight and human error.
That’s the game, man. 🤷
I’m really sorry to hear about your daughter.
I’m sorry for your loss. Thats the type of scenario I fear about, I can have the 321 backup but errors happen like not configuring you backup correctly, or destroying the raid… also I dont have a lot of money to buy drives and most of them are refurbished and I dont know if I can trust them.
The thing is, these sorts of losses aren’t limited to selfhosting. Selfhosting introduces some new risks and reduces some other risks.
Digital data is inherently fragile. It takes active work to preserve it.
That’s one of the reasons my wife and I make an actual physical photo book each year of our favourite photos.
I also have been there kinda when i was about 16 years old i guess such a big lesson for me now if i do some extra backups and al kinda backups and shit my friends think i am paranoid.
Well yes i am because i have been there
As the saying goes: That you’re paranoid does not mean they’re not out to get you … 😃
I have those battle scars too. :(
Only thing dear to me is my family photos and videos over the years. They’re backed up to two different cloud providers. Everything else is ultimately downloadable.
You SHOULD be scare of losing your data. In fact it’s a very likely outcome if you’re storing data and you don’t know what you’re doing. This is true of every electronic storage format. If you’re not ready to lose everything, you have periodically practice recovery from backup.
Over the years, i’ve met tons of novice computer users who tell me “I’m worried my files will get hacked if I store them on dropbox / cloud”. I always set them straight: the number one risk for you is losing your own data, not data theft, unless somehow your files contain industrial secrets worth hundreds of millions of USD.
I consider myself an experienced computer user and developer, having had various roles that border on sysadmin. I don’t trust myself to run my DIY NAS. For the stuff that matters, you should fear complexity as it 's a source of errors. You should doubt yourself at every step. Practice recovery. This is true for everything. I messed up my pfSense router config this weekend, it wouldn’t boot. I took the opportunity to practice recovering from backed up config (I should have done that much earlier).
lol
Have multiple copies of the data. Use snapshots, they don’t get encrypted by a ransomware because they are read only and can’t be accessed via samba or nfs. It’s only a problem when the attacker gets root access to your NAS. Use a cloud provider like backblaze and backup your data encrypted. If you are really scared that ransomeware data will overwrite your backups use 3-2-1 and Grandfather-father-son backup strategy. But all this comes at a cost.
The key is to do regular backups to a different location, and to keep previous versions as read-only backups for a certain timespan. If something happens to the local data you can just restore from the remote backup, and also pick an unmodified previous version in case of a ransomware attack.
E.g. I do a daily encrypted cloud backup of everything that can’t just be downloaded again, and the backup provider keeps previous versions for 30 days.