I got a home NAS (Network Attached Storage) with a formatted volume size of about 11TB (4x4TB drives in a custom RAID). I've had a few local USB/Firewire-based home storage solutions in the past such as a Drobo, but this time I wanted to go to the network because I thought it would be more flexible. The drivers were:
- Its good to upgrade storage periodically over your lifetime (see below)
- The NAS is more flexible that a single-computer USB-based connection
- It has more storage
- The NAS device has a lot more features than a simple storage array
More on all of this below.
The model I chose is a Synology DS920+.
The DS920+ has four drive bays and two m.2 slots for a read/write SSD cache. This is a nice feature because you can get inexpensive high-capacity spinning disk drives and use the SSD as read/write cache to mitigate the performance issues you'd normally see on low-rpm spindle-based storage such as horrible parallel reads/writes and slow random access/seek speeds.
I got four Seagate IronWolf 4TB NAS Internal Hard Drive HDD. These have a modest 64MB cache and only spin at 5900rpm. I also filled both m.2 slots with a pair of Samsung 500GB 970 EVO SSD's.
The DS920+ also offered a 4GB RAM expansion, bringing total RAM to 8GB. This is useful since the NAS is really a Linux computer and my gut says 8GB will give the OS more breathing room for disk cache and running various apps and protocol servers it supports.
On that note, the Synology DS920+ has a ton of apps and services and I've only scratched the surface on what I use. Some of the highlights:
- SMB server (for WIndows sharing)
- AFP/Bonjour server (for Mac)
- rsync
- NFS
SMB and AFP are pretty slow protocols and I always hate using them for large file transfers, like the multi-terabyte transfers I need to make to get all of my home videos and photos off my old Drobo. I found a great writeup on the performance of these protocols by Nasim Mansurov, here at the Photography Life blog. These protocols are great for general use, but not for the initial data-loading phase.
Part of my apprehension is not knowing the state of a large transfer, particularly if its interrupted. If I moved files and the transfer was interrupted, were they really moved? Sometimes I'm dealing with 50k files and it's not easy to get a warm and fuzzy feeling about whether a large file transfer worked or not even if it appeared to finish. Sure, when the copy is done I could compare file counts and byte sizes between both source and destination. This would give me some confidence that I can now delete the source directory, but that's not the real problem.
The real problem is managing a large transfer and being able to optimally start/stop/resume it at will. This is trash using a GUI and AFP/SMB. For instance, if I want to resume a copy, do I need to copy the whole thing again? Do I manage the copy by doing one folder at a time and sit and wait for it to finish before starting the next folder? LOL, I've been there! Also, what happens when I find destination files that already exist? Walking into my office at 7am to check on the progress of a a twelve hour file transfer only to find that twenty minutes after I went to bed the system politely prompted me about how I want to handle a destination file conflict. I never want to be that angry holding a cup of hot coffee ever again. Screw all that.
The answer of course is rsync, a tried and true utility that's a foundation of Linux. Since the NAS is a Linux machine, it's got an rsync facility. rsync is single-purposed yet sophisticated piece of software that runs on both ends of a file transfer. The client and server both have access to their respective file systems and can negotiate a large transfer efficiently. If it's good enough for Fortune 500 companies to use in production, it's good enough for home videos of my kids fighting in their diapers.
rsync negotiates file listings and checksums on both sides of the transfer and will always do the right thing in terms of resuming a partial transfer. It's like magic.
To get this to work smoothly, I had to root into the NAS drive and change a few SSH options to allow for a more or less automated transfer. Right now I'm in the middle of what will be a 2+ day transfer of 1.6TB of home videos.
It's Good to Upgrade Storage
On a final note, I wanted to say one simple thing about why I like to upgrade my storage every decade or so. It's based on a few simple points:
- Hard drives have a life expectancy and they don't last forever
- New connectors and protocols are constantly getting faster and more robust
- Old connectors and protocols are always becoming obsolete. If you plan the frequency of your copies properly, you'll always be in an era where both the old and new technology are still around and you can get adapters and whatnot to ensure the transfer is possible
- I like to play with new things
Ok, the last point is probably the real reason. You figured me out.