SoulsCollective
04-24-2014, 02:20 AM
Hi folks, long time no post but good to see XS still kicking along. Not sure if this belongs more in Server or Storage, but I'm guessing Server, so here goes:
Looking to redo our home network and centralise things a bit without hopefully sacrificing too much in the way of performance and usability. Goals for this build are
serve as a central and convenient point of storage for everything from movies to wedding photos to files for my wife's Masters thesis, so need both space and a fairly high level of redundancy and data protection. Truly important commercial (tax) stuff is backed up offsite, so what is left, while valuable, isn't justified off-siting (plus there's a lot of stuff, currently sitting at about 4TB)
be relatively futureproof, providing at least 12TB of total space, whilst running relatively quietly
play nicely with both Windows and OSX
as I'm a bit concerned about the recent surveillance/data mining revelations brought to light (yeah yeah tinfoil etc), I'd like to retain control and ownership of all the stuff we currently have sitting in various cloud services (Dropbox etc), so needs to run some sort of personal cloud service (BitTorrent Sync, OwnCloud, etc - open to suggestions here)
be fast, or at least as fast as reasonably possible, because if it doesn't relatively seamlessly replace storing stuff on local HDDs I know my wife won't use it which defeats the purpose
hence perform well over our LAN (support teaming, jumbo frames, etc)
not die (server etc) and not require fiddling to keep up. Particularly as we want to use it as a cloud replacement; if my wife's at Uni and can't access her Masters stuff like she currently can with her Dropbox account, she won't use it again which again defeats the purpose. I'll also be somewhat unimpressed if I can't access notes and files from work. Hence, uptime, reliability, etc (another reason why we need data protection and redundancy).
I've done my own digging and contemplating, and have worked out what I think will do a reasonably good job hardware-wise. Keep in mind that I'm in Australia, so availability of parts locally is somewhat restricted compared to you folks in the USA.
CPU: some manner of low-TDP i5 part, currently looking at a 4570S
CPU cooling: Noctua NF-12P. Overkill, but I have one spare from a prior build. May even be able to run fanless (that'd be nice).
Mobo: Probably an MSI Z87I mITX board, but really anything in the form-factor with dual NICs that support teaming will do
RAM: 2x4GB DDR3-1600 of some manner (OR - contemplating 2x8GB if I do end up running ZFS, see below)
OS HDD: 240GB Intel 730 SSD. Way too much capacity for intended usage, but this is the smallest size I can find one in (AND - wondering whether I can partition it and use some for ZIL/L2ARC if I end up running ZFS, see below)
RAID: LSI MegaRaid 9361-8i
RAID HDDs: 6x WD 3TB 'Red' drives in RAID-6
Case: BitFenix Phenom mITX. Will replace fans with Noctua variants, as much happier with their noise profile.
The RAID controller is already purchased (and was the subject of some debate, see below). The rest of the hardware is open to any alternate suggestions.
All this is missing is an OS (and any suggestions/feedback). I am currently leaning towards running some flavour of Debian. I am, however, by no means an expert in *nix, with experience being limited to running Debian rather than MacOS on some inherited hardware back in the PowerPC days. I like to think that I'm reasonably capable of working around difficulties as they arise, however, so despite being much more comfortable in Windows environments I'm reasonably sure some manner of *nix distro will serve the intended purpose better than running Server 2012 or something (I'd also really like to not have to shell out more cash just for a software license).
I am undecided on some specifics, though. Firstly, I'm reading a lot about ZFS. Filesystems are not my forte, but there is a good deal of positive buzz about it, both from a performance (good) and data security (doubleplus good) perspective. On the other hand, I am also hearing from parties reporting slower performance than running EXT3 or 4, that it can be hugely complex to maintain, and that it has enormous hardware requirements to produce good performance (a friend tells me not to even consider it unless I have >24GB of RAM, which seems a rather high requirement). I have done as much digging as I can on this issue, but haven't really found a clear (and recent - a lot of stuff written on the topic is years old) cost/benefit summary vs just running EXT4.
On a related note, I am also unsure whether I should be looking at Debian-Linux, or Debian-BSD. I know nothing about BSD, but again, my Google-Fu tells me that it is supposedly not quite as well-supported as the Linux kernel and can be slow to receive updates - however supposedly its native version of ZFS is more reliable and generally better than ZFS-on-Linux (which is, again according to sources some years out of date, supposedly much slower). I'd like to maintain at least some degree of familiarity with the OS running on this build, but if running Debian-Linux will measurably impact performance, uptime or access to updates then I'd consider it worth the time to learn my way around BSD.
Finally, I'm wondering if anyone has any input on use of a HW RAID controller vs software RAID (either using mdam or ZFS). Back when I last had any exposure to this stuff HW raid was miles faster (particularly in relation to parity RAID, which I am proposing to use), however when I posed these questions to my closer circle of friends opinion was split on whether to use software RAID instead as it removed the RAID controller as a potential single-point-of-failure and supposedly was just as fast. My Googling turns up a few threads on the issue, and from what I'm seeing performance on a HW RAID controller is still faster, up to about 35% when running mixed read/write operations. As above, I already own the controller I'm proposing to use (had to order the thing from the US and use a shipping forwarding service, there's basically nothing available locally that isn't already crammed into a server enclosure), so this is more from academic interest than whittling down parts list. To my mind, though, the risk of the RAID controller dying seems vanishingly small (all the ones I've had experience with have delivered years of continuous uptime with nary a hitch), and probably significantly smaller than that of the hosting MB dying. Still, if SW raid is as fast/faster (ZFS RAID management in particular is supposedly very good indeed) then I'd be silly to be using the card just for the sake of it. Interested in opinions :)
Thanks for any input! Expect a build log when all is settled ;)
Looking to redo our home network and centralise things a bit without hopefully sacrificing too much in the way of performance and usability. Goals for this build are
serve as a central and convenient point of storage for everything from movies to wedding photos to files for my wife's Masters thesis, so need both space and a fairly high level of redundancy and data protection. Truly important commercial (tax) stuff is backed up offsite, so what is left, while valuable, isn't justified off-siting (plus there's a lot of stuff, currently sitting at about 4TB)
be relatively futureproof, providing at least 12TB of total space, whilst running relatively quietly
play nicely with both Windows and OSX
as I'm a bit concerned about the recent surveillance/data mining revelations brought to light (yeah yeah tinfoil etc), I'd like to retain control and ownership of all the stuff we currently have sitting in various cloud services (Dropbox etc), so needs to run some sort of personal cloud service (BitTorrent Sync, OwnCloud, etc - open to suggestions here)
be fast, or at least as fast as reasonably possible, because if it doesn't relatively seamlessly replace storing stuff on local HDDs I know my wife won't use it which defeats the purpose
hence perform well over our LAN (support teaming, jumbo frames, etc)
not die (server etc) and not require fiddling to keep up. Particularly as we want to use it as a cloud replacement; if my wife's at Uni and can't access her Masters stuff like she currently can with her Dropbox account, she won't use it again which again defeats the purpose. I'll also be somewhat unimpressed if I can't access notes and files from work. Hence, uptime, reliability, etc (another reason why we need data protection and redundancy).
I've done my own digging and contemplating, and have worked out what I think will do a reasonably good job hardware-wise. Keep in mind that I'm in Australia, so availability of parts locally is somewhat restricted compared to you folks in the USA.
CPU: some manner of low-TDP i5 part, currently looking at a 4570S
CPU cooling: Noctua NF-12P. Overkill, but I have one spare from a prior build. May even be able to run fanless (that'd be nice).
Mobo: Probably an MSI Z87I mITX board, but really anything in the form-factor with dual NICs that support teaming will do
RAM: 2x4GB DDR3-1600 of some manner (OR - contemplating 2x8GB if I do end up running ZFS, see below)
OS HDD: 240GB Intel 730 SSD. Way too much capacity for intended usage, but this is the smallest size I can find one in (AND - wondering whether I can partition it and use some for ZIL/L2ARC if I end up running ZFS, see below)
RAID: LSI MegaRaid 9361-8i
RAID HDDs: 6x WD 3TB 'Red' drives in RAID-6
Case: BitFenix Phenom mITX. Will replace fans with Noctua variants, as much happier with their noise profile.
The RAID controller is already purchased (and was the subject of some debate, see below). The rest of the hardware is open to any alternate suggestions.
All this is missing is an OS (and any suggestions/feedback). I am currently leaning towards running some flavour of Debian. I am, however, by no means an expert in *nix, with experience being limited to running Debian rather than MacOS on some inherited hardware back in the PowerPC days. I like to think that I'm reasonably capable of working around difficulties as they arise, however, so despite being much more comfortable in Windows environments I'm reasonably sure some manner of *nix distro will serve the intended purpose better than running Server 2012 or something (I'd also really like to not have to shell out more cash just for a software license).
I am undecided on some specifics, though. Firstly, I'm reading a lot about ZFS. Filesystems are not my forte, but there is a good deal of positive buzz about it, both from a performance (good) and data security (doubleplus good) perspective. On the other hand, I am also hearing from parties reporting slower performance than running EXT3 or 4, that it can be hugely complex to maintain, and that it has enormous hardware requirements to produce good performance (a friend tells me not to even consider it unless I have >24GB of RAM, which seems a rather high requirement). I have done as much digging as I can on this issue, but haven't really found a clear (and recent - a lot of stuff written on the topic is years old) cost/benefit summary vs just running EXT4.
On a related note, I am also unsure whether I should be looking at Debian-Linux, or Debian-BSD. I know nothing about BSD, but again, my Google-Fu tells me that it is supposedly not quite as well-supported as the Linux kernel and can be slow to receive updates - however supposedly its native version of ZFS is more reliable and generally better than ZFS-on-Linux (which is, again according to sources some years out of date, supposedly much slower). I'd like to maintain at least some degree of familiarity with the OS running on this build, but if running Debian-Linux will measurably impact performance, uptime or access to updates then I'd consider it worth the time to learn my way around BSD.
Finally, I'm wondering if anyone has any input on use of a HW RAID controller vs software RAID (either using mdam or ZFS). Back when I last had any exposure to this stuff HW raid was miles faster (particularly in relation to parity RAID, which I am proposing to use), however when I posed these questions to my closer circle of friends opinion was split on whether to use software RAID instead as it removed the RAID controller as a potential single-point-of-failure and supposedly was just as fast. My Googling turns up a few threads on the issue, and from what I'm seeing performance on a HW RAID controller is still faster, up to about 35% when running mixed read/write operations. As above, I already own the controller I'm proposing to use (had to order the thing from the US and use a shipping forwarding service, there's basically nothing available locally that isn't already crammed into a server enclosure), so this is more from academic interest than whittling down parts list. To my mind, though, the risk of the RAID controller dying seems vanishingly small (all the ones I've had experience with have delivered years of continuous uptime with nary a hitch), and probably significantly smaller than that of the hosting MB dying. Still, if SW raid is as fast/faster (ZFS RAID management in particular is supposedly very good indeed) then I'd be silly to be using the card just for the sake of it. Interested in opinions :)
Thanks for any input! Expect a build log when all is settled ;)