RAID 5.. or Raid 10? [Archive] - XtremeSystems Forums

OkydOky

01-11-2007, 11:52 AM

Hello,

I am considering building myself a NAS.

This would Mostly Store my Anime Collection and MP3s.
I was thinking of maybe Going beyond the Definition of a NAS and have it also take care of playing the MP3s, and streaming it over the LAN, and be Downloading torrents (that would be managed remotely; through a web interface for example) The data, I would really NOT want to lose, since it is already a Rather large Collection.. and would be a real pain to find again.

I am Looking at the Kontron 986LCD-M/mITX. To make a small Sized and somewhat carry able rig.

This would Probably be using the ICH7R's controller. But I would be willing to get a separate PCI or PCIe controller if needed.

The Fact that this board has 3x 1000Mbps Ethernet ports, interested me aswell, since I could also try to get a Dual Gigabit Connection working.

My main Consern is Raid 5? or Raid 10?
There will Be a lot of Reading done, but there also will be some writing.
The Writing probably won't need to be too fast, as it will be the speed I am downloading at.. so a HIGH theoretical of 10Megs. (My Cable connection speed :p), and probably more realistically closer to 4-5 at peak.

There will not be too many Computers trying to access this at the same time.. usually One or Two. With a Max of 4 (That would be with 1 person, watching on 2 comps at once... or leaving one on... )

I have been leaning towards Raid 10. Since it has the better fault tolerance, and the Performance of Raid! -- There also is not as much of a need for an Expensive Controller. But This is at the Cost Storage Space..
This will bed done with 4 Hard Drives. Probably 4x 400Gb 7200.10s

Am I correct in leaning towards Raid10?

alfaunits

01-11-2007, 02:28 PM

For the price of a medium good RAID5 controller you can get 2 extra drives, and still use RAID10. There's the benefit of no additional heat over an addon controller.

Note though, ICH7R has acceptable RAID5 write performance. It will not be comparable to read performance by far, but as you say, for internet downloads it will be sufficient.

nn_step

01-11-2007, 02:30 PM

how many drives do you have to work with and how important is capacity?

OkydOky

01-11-2007, 05:45 PM

Well I don't have the drives.. hehe.. or anything.

But I was thinking of Four.
Maybe 6. But Prolly Four.
And Probably 400gb 7200.10
If I get 6, or go Raid5 I would need a separate controller as well.
What one would you Suggest? For raid10? and raid5?

Capacity is Important. But Redundancy I would say more so.
Scalability would be awesome.. I m not sure if Scalability is the proper word: What I mean is Being able to add drives to the array for more capacity without loosing anything, and just gaining capacity. But that is far from what I am looking at.

Cost is probably one of my Biggest issues, as I am still a student, and money is precious...

nn_step

01-11-2007, 11:28 PM

Raid 5 would probably be the better choice of the two, though Raid6 would be beneficial. Also, if you get a good raid card it'll be extremely beneficial to you when you plan on upgrading.

alfaunits

01-12-2007, 02:28 AM

nn_step

01-12-2007, 05:29 AM

RAID6 on 4 drives is s waste of processing power. For 4 drives, the failure chances are the same as that for RAID10 (2 drives can fail).
I don't know of any controllers that can expand arrays - migarate yes, but that will not change the capacity.
RAID5 of 4 drives with onboard is good enough for you.
ummm wrong, in Raid10 if two of the right drives die, good bye raid and everything on it. Raid6 You can lose 2 drives and still be able to recover.
it is also known as point sensitivity. Yes Ideally Raid10 can survive losing 2 drives but if they are a single Mirror group you lose everything.

OkydOky

01-12-2007, 09:37 AM

I had not Considered Raid6.. But it could be interesting. Especialy with 6 Drives..
My Concern is Finding a SATA Controller for it.. that is not more expensive then all the Hard Drives put together. I did a little bit of quick looking around and none of my online retailers seem to have any...
Any Thoughts on Raid 50?

Heh.. and this is all for a simple "Home server"... :p

uOpt

01-12-2007, 10:15 AM

I'm not sure I would trust the onboard SATA junk with either raid-5 or raid-10. The processing power for raid-5 and -6 is really not the problem. The recovery in difficult situations such as the computer crashing and rebooting during an error condition and then the BIOS and the OS driver disagreeing on a course of action is the problem.

I'd take a 3-disk RAID-5 over a 4-disk RAID-10. Reliability-wise it is less likely that two out of three disks fail than 2 out of four. (although raid-10 can survive in a dual-fail situation if opposite disks were affected).

The major problem with RAID-10 is expandability. A raid-5 or raid-6 can be expanded by buying single new disks, and the loss from the raid mechanism even shrinks as you add more disks. Disks in a raid-10 can only be added in pairs or worse (if you have more than two-way raid-1 in it) and the loss always stays at 1/number_of_slices_in_raid1.

halo112358

01-12-2007, 02:38 PM

I just finished building a 6 drive raid5 array for work - I very highly recommend running software raid on linux. The nicest thing about this approach is that the machine can die and you can just plug the drives into another machine and recover the array with a knoppix live cd. I like that I can rescue data from any machine without having to potentially replace an expensive hardware raid card.

For this install I used debian linux and built the array as follows:

disk1: boot (raid1, 1gb) / data (raid5, 230gb)
disk2: boot (raid1, 1gb) / data (raid5, 230gb)
disk3: data (raid5, 230gb) / swap (raid5, 1gb)
disk4: data (raid5, 230gb) / swap (raid5, 1gb)
disk5: data (raid5, 230gb) / swap (raid5, 1gb)
disk6: hot spare

The breakdown works out like this:

(md0) 1gb raid 1 on /boot
(md1) ~1tb raid 5 on /
(md2) 2gb raid 5 for swap

Disks 1 and 2 hold redundant boot sectors and a mirrored copy of the boot partition, disks 3,4+5 hold a 2gb raid5 swap space. If one of the bootable drives crashes I can still boot from the other, If any one of the swap drives crashes the swap raid array stays up (if swap crashes the kernel can crash as well :( ). Additionally you can lose any of the data drives and not have a prob lem. The system should automatically bring in the hot spare if any drive dies :D

The other really nice thing about this setup is that I can bring in more drives as needed and expand any of the raid 5 arrays, plus the wasted storage % drops as you add more drives.

alfaunits

01-12-2007, 11:15 PM

halo112358

01-12-2007, 11:39 PM

With linux software raid supposedly you can grow an array just by adding disks to it, the management software should start an array rebuild automatically once you instruct it to add a new drive to the array. Theoretically this is all done live, the array goes into degraded mode but still provides access to all of your data during the process.

Growing the array was my one major concern after I setup that fileserver, I was pretty happy to see reports of an easy upgrade path.

OkydOky

01-13-2007, 01:28 AM

Halo,

Thank you for the wonderful advice.
I think Linux Software raid is what I'll do.
This adding of drives to the arrays would be great.
Have you had any experience with it?

Also which file system did you use and would recommend?
And How is the Performance if this software Raid 5?
I am Mostly interested in reads. Writes are not too important (Unless it's completely crippled)

Also, each drive does not need to be on the same controller for this to work, correct? and out of curiosity: What were your system's specs?

nn_step

01-13-2007, 01:52 AM

Yeah, the onboard RAID5 recovery is neither fast nor easy.
You can hack Windows XP to do RAID5 software. It is not as easy to recover as Linux RAID5, due to dynamic disks.

But seriously, consider how important your data is. If it is multimedia, buying an addon controller is seriously not worth it.

uOpt, how would you expand a RAID0/5/10? I know we one can migrate RAID0 > RAID5, but the capacity remains. I didn't really try, but I never saw an option to add a disk to an existing array (for any array that would mean a rebuild).
I'd take a $50 hardware raid card over ANY SOFTWARE raid anyday, for the following reasons:
1)Faster rebuilds
2)Superior performance
3)The ability to swap mobos and not have to worry about drivers (windows)
4)$50 raid card is cheaper than a $90 Mobo
5)It supports everything that software raid supports and much more.
6)Higher end cards include on-card ECC DRAM/SRAM, that reduce access latency and increase bandwidth
7)I have never had a single Raid card die on me, ever

Movieman

01-13-2007, 02:09 AM

some thoughts:
Listen to uOpt..He knows, doesn't think, but knows what he's talking about.
Then what nn_step told you is correct also..Buy a raid card..Very good investment..on board raid is garbage IMO..
Then one simple thing: BURN what you have if it's important.
NEVER trust a hard drive with anything you value.
With disc's so cheap there is no reason not to and many reasons to do so.
Yes, I know, you'll say "BUT all that time to do it"...And how much time did it take to collect it? See my point? Do a few disc's a night and it's not a killer and when your done, make a copy of each..dvdr's can go bad too!:D
I'm an old fart and like 60's music..Have 22 cdr's that I made 3 years ago of mp3's..The drive that they were on is dead and gone but the music is still here and yes, I did make a copy of each.;)

alfaunits

01-13-2007, 12:45 PM

$50 hardware raid card
There's an oxymoron if I ever saw one. The few SATA RAOD cards < 100$ all performed way worse than the onboard ones (I have currently Intel ESB2, had ICH5R on PC-DL, nvRaid on A8N32).
More important none of them had > 4 ports, none had RAID6.
#6 contradicts $50 point.
Not that I tried, but I don't recall an option to expand an array with any controller (including high-end ones).

Don't get me wrong, I consider EX8350 with 8 Raptors to be a slow solution and am looking for a SCSI setup to replace it, but that's for a work environment. For a multimedia setup, that is run over a network, I do not consider an addon card worthy. If you/someone values your multimedia, go for it, but you will have it on DVDs or other off-computer media anyway.

OkydOky

01-13-2007, 02:32 PM

Hehe.. everyone is of different opinions..
Regarding how important the data it is:

If it were to be destroyed: I would be mad,, but it would not be the end of the world. My ISP simply would not be to happy about my bandwidth usage during the next few months...

Price is what is making me look away from separate hardware controllers..

Regarding performance all that matters is good read performance. All the writing will be the downloading, which will be far from maxing the drives write.. (and the Initial Transfer.. but that is only a one time thing..)

Price is my main Issue.
And I have yet to find a $50 Raid Controller >.>
The cheapest I could find was the RocketRAID 2320 at ~ $250 USD
Or 2310 @ $150.

This might be because of the fact that my retailers are limited, as I am only looking at Canadian ones..

Growly

01-14-2007, 02:38 PM

I just wanted to add that in my very, very painful experience, any hardware RAID in which you've paid less than $500 (US, i'm converting) for the card is useless. Taking chances with data on cheap~ish hardware ($300 for a controller may not be cheap, but you'll feel it when your data's gone) is definitely no go.

I bought a relatively expensive and feature-packed Promise controller a couple of years ago, for about NZD$450. The card was good, and fast, and had the features you'd expect, but its class was just not worth the money. I've heard Promise bashed countless times, but nothing else from any other manufacturer in the same class could pretend to have been worthwile. Long story short, a power cut came and I lost 600GB from a RAID 5 array by simultaneously losing two disks from the array.

RAID cards which have features to protect data, which are resilient and reliable, cost money. Areca (my dream) controllers will set you back a fortune, because they will give you very good RAID schlong. The rest I don't even consider anymore.

So, now that you've made it clear you won't be getting said hardware controller goodness, I'd advise you to stay away from a hardware controller entirely.

Use Linux RAID - it's what I do now :)

halo112358

01-14-2007, 02:42 PM

Here's a quick comparison between /dev/sda (first 250gb WD sata drive) and /dev/md1, the ~1tb array of 5 of the same drive.

fortawesome:~# hdparm -t /dev/md1

/dev/md1:
Timing buffered disk reads: 602 MB in 3.00 seconds = 200.57 MB/sec
fortawesome:~# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 226 MB in 3.03 seconds = 74.71 MB/sec
fortawesome:~#

Overall CPU usage is very low, I've never been in a situation (even with a p2 300mhz) where a software raid array was generating any load on the system.

The only downside to installing a linux software raid array is that there is a learning curve to the system software and the raid software. It's not bad if you're willing to read a bit but you should know about it first.

ps:

I second all the other opinions here, stay away from cheap hardware psuedo-raid, my experiences have always been bad with it. I had a machine at work using a p4 intel chipset that had load spikes that I couldn't explain. When I disabled raid on the same machine they went away entirely. I've used windows software raid successfully in the past, it's alright, but it's not as feature filled as linux raid. I'm currently using nforce4 psuedo-hardware raid for a stripe set, it's not as fast or low overhead as I'd like.

uOpt

01-18-2007, 08:22 AM

uOpt

01-18-2007, 08:25 AM

It is not as easy to recover as Linux RAID5, due to dynamic disks.

What? Sorry man, but that sounds like glibberish. There is nothing dynamic about the disks.

uOpt, how would you expand a RAID0/5/10? I know we one can migrate RAID0 > RAID5, but the capacity remains. I didn't really try, but I never saw an option to add a disk to an existing array (for any array that would mean a rebuild).

I didn't mean live expansion.

I mean sitting there with 4 disks and needing more space. With raid-5 you buy another disk and get 33% more space. With raid-10 you have no way to integrate an additional single disk.

alfaunits

01-18-2007, 08:28 AM

Do you know if anything similar is available for Windows?
In any flavor (in pieces, full program etc.)

uOpt

01-18-2007, 08:29 AM

I'd take a $50 hardware raid card over ANY SOFTWARE raid anyday, for the following reasons:
1)Faster rebuilds
2)Superior performance
3)The ability to swap mobos and not have to worry about drivers (windows)
4)$50 raid card is cheaper than a $90 Mobo
5)It supports everything that software raid supports and much more.
6)Higher end cards include on-card ECC DRAM/SRAM, that reduce access latency and increase bandwidth
7)I have never had a single Raid card die on me, ever

You mix up software raid and that onboard sata junk. With software raid you can of course switch mobos as you like.

Whether you had a card die or not is irrelevant. Fact is it can die and then you have to wait for a replacement.

Do you have any numbers comparing how fast a raid-5 rebuilds on software raid and good controller? I am not convinced it is slower.

alfaunits

01-18-2007, 08:30 AM

What? Sorry man, but that sounds like glibberish. There is nothing dynamic about the disks.

Windows XP, IIRC, requires the disks to be dybamic, not basic in order to do the Wndows software RAID5.

I mean sitting there with 4 disks and needing more space. With raid-5 you buy another disk and get 33% more space. With raid-10 you have no way to integrate an additional single disk.
Ah, quite different.

uOpt

01-18-2007, 08:51 AM

Windows XP, IIRC, requires the disks to be dybamic, not basic in order to do the Wndows software RAID5.

Linux doesn't have that concept. Disk is disk. You can integrate any block device into a RAID under Linux software raid. Doesn't matter whether it's a disk, a partition, a USB disk, a file in a filesystem, a network block device, a file inside an encrypted filesystem over a network or your mp3 player. People even made filesystems out of their google mail.

So what's a "dynamic" disk under Windoze?

alfaunits

01-18-2007, 10:33 AM

Look, mate, I said the Windows RAID5 is not as easy to recover as the Linux RAID5 due it requiring dynamic disks, and they are PITA.
You prolly mis understood me.

Dynamic disk under Windowsis a way of not allowing the same disk to be used on more than one Windows ;-) that's all I found it to be, it's not useful in any way).

epion2985

01-19-2007, 01:39 AM

From overall performance stand point raid50 seems to be the best choice. Also if you don't mind having two or more controllers for each mirrored raid5 array in raid50 you can take advantage of duplexing.

Good reads btw:

http://cracauer-forum.cons.org/forum/raid-hints.html

http://cracauer-forum.cons.org/forum/viewtopic.php?t=34

halo112358

01-27-2007, 03:41 PM

It doesn't stop there. With software RAID can also do superstupid tricks like plugging one disk into a USB case and still integrate it into your array with the normally connected drives. Or you can connect the drive to a different machine and use one drive in the RAID over the network.

I did exactly that when I recovered an array from our old crashed file server - for some reason my desktop machine couldn't ever find the IDE disk on boot so I plugged it into a USB enclosure and brought it into the array that way ;)

From overall performance stand point raid50 seems to be the best choice. Also if you don't mind having two or more controllers for each mirrored raid5 array in raid50 you can take advantage of duplexing.
Yum :D I wish work would let me buy more hardware I could build one of those, I'd really like to bench that against raid 6. Actually, how does raid50 compare to raid60?