Setting up some massive storage could use some advice

[tommyshango]

I am planning on adding a lot of storage but I am not sure how to go about doing it. I thought if I made a list of what I have and maybe then I could get some thoughts on what else to get. I don’t have a total dollar amount I want to spend. I guess I would like to look at some options. I might also like the option of building this slowly and getting a few pieces here and there rather than all at once. The goal overall is to put every DVD I own on a raid setup.
Case – lian Li V2100 Plus II http://www.lian-li.com/v2/en/product/product06.php?pr_index=137&cl_index=1&sc_index=38&ss_index=95
I have a total of 12 internal 3.5” bays plus I believe I can get up to 8 more 3.5” drives in my case by using 2 of these http://www.performance-pcs.com/catalog/index.php?main_page=product_info&cPath=130_151_257&products_id=21293
I would then be limited to a single burner but that’s ok.
I have 2 of each of these HDD’s
http://www.newegg.com/Product/Product.aspx?Item=N82E16822148337
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136351
My c: is a OCZ vertex 120gb
The motherboard is this:
http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2831

My thoughts were to try this a few different ways.
1st Try 3 more of the Seagate 1.5tb or 3 more 1.5 tb WD’s and do a raid 5 on the remaining 5 spots on the irch10 controller a long with the vertex. I don’t know if this is possible or not. Then buy a controller card such as an 8 port sata card that high point has such as this one http://www.newegg.com/Product/Product.aspx?Item=N82E16816115022
I would then fill out the remaining 8 slots in 2 more sets of raid 5 so that I could do 4 at a time using more of the western digitals to keep costs down. I thought 2 series of raid 5 so that I can do this in stages.

2nd do similar to the first suggestion but then try a different card and really fill out such as a 16 port areca card like http://www.newegg.com/Product/Product.aspx?Item=N82E16816151025
I was told it might be nice to make one giant raid 5 with a card like that. I might want to do raid 6 if I was going to build such a large array. But I don’t know that I would be able to build it in stages. Would I have to buy all 16 drives at once? That would be quite the upfront cost then or once the raid is built can you rebuild it and add more drives as you go along? (I have never ran more than raid 0)
I have read that high point is good. I have read the Areca is the best. I was told 3ware is good. I was told high point is bad and not a true hardware card. I am concerned about the overall cost of an Areca card and heard their tech support is non-existent. I heard 3ware support is good.
I was told you have to use all the same drives. I really don’t want to waste the drives I have already purchased. If this seems confusing it is. I don’t really know what I am doing here and am open to any suggestions. I would like to keep the initial costs around $1500. That should give me a few drives and whatever card I need but if that’s not possible to do this in stages then I might have to wait. Right now I have data on 2 of the drives and want to be able to build the first raid 5 within a few weeks and then transfer the data over to that. I want to stick with 1.5 TB drives because it looks like a good price point. The 2tb are much more expensive and 1tb just isnt big enough.

Thanks

[tommyshango]

Oh I just realized I will probably need another power supply and run two at once. I currently have an ultra x3 1000 watt but I dont think it can run that many hard drives unless I am wrong. The case says it can hold 2 and I dont want to go crazy spending hundreds on a power supply if I do need one. Probably something that could take a little bit of the load off the Ultra and run quite a few of the drives or something like that. Again I am not sure what would fit this exactly so if anyone has ideas I would appreciate it.

Thanks again!

[mcmeat51]

Oh I just realized I will probably need another power supply and run two at once. I currently have an ultra x3 1000 watt but I dont think it can run that many hard drives unless I am wrong. The case says it can hold 2 and I dont want to go crazy spending hundreds on a power supply if I do need one. Probably something that could take a little bit of the load off the Ultra and run quite a few of the drives or something like that. Again I am not sure what would fit this exactly so if anyone has ideas I would appreciate it.

Hard drives use like ~10 watts of power, Im sure you wont need another power supply!

[tommyshango]

Well I talked with a rep at Ultra and they said no! I was sort of like WTF are you serious but thats what they said. I bet the newer WD green power uses less power than half a dozen older drives.

Should I just get more cabling to hook them up? They make a 4 pin molex that splits into 2 sata connectors. I am sure i could find some combination like that.

Anyone have any advice on anything else? I am really lost here.

[[XC] Oj101]

Hard drives use like ~10 watts of power, Im sure you wont need another power supply!

But during startup?

[XS_Rich]

But during startup?

Say no more than about 30W. A decent 1kW PSU should easily cope with this many hard discs.

[tommyshango]

Well yeah I thought the PSU I have would suffice and even if i did need to purchase another 400-500 watter it wouldnt cost that much.

What about all the other issues? Anyone have any advice? I really want to start the first stage in a couple of weeks. If I can make an initial purchase of 4-8 HDD's and whatever controller card I need. Or even just get 3 HDD's and complete a single raid 5 setup if thats possible using the controller on my motherboard.

[lowfat]

I ran 16 hard drives on a 420W PSU. The whole system rarely consumed more than 200W.

[SoulsCollective]

RAID-5 and onboard with that many drives is just begging for failure.

For large storage arrays, you're looking at a hardware controller, RAID-6 or nested arrays. Invest in a decent 12pt controller, preferably something using the IOP348 chip (dual-core, 1.2GHz) otherwise your write speeds are going to fall down, and rather than the 1.5TB drives, for which you pay a price premium and which have known reliability issues, go with twelve 1TB drives, either F1s or WD drives. That'll give you somewhere around the high 9TB worth of completely redundant (can tolerate failure of two separate drives with no data loss) and fast storage, rather than a piecemeal and slow three arrays of around 3TB. Make sure to buy some of your HDDs from different retailers, too - you want to get drives from as many different batches as possible, to minimise the chances of a simultaneous failure of two drives at once.

I hope you're running all this off a UPS? If not, at least look at getting a BBU (battery backup) for which ever controller you end up getting - as otherwise if you lose power or suddenly reboot, it's fairly likely you're going to get data loss and corruption.

Finally, while a good kilowatt PSU will have no problems powering that many drives, make sure your PSU actually has the cables to support fourteen SATA devices (twelve HDDs, Vertex, ODD), and if I was you I'd look at replacing that Ultra - there're known issues with those PSUs.

Finally, re your question about later expansion, any decent hardware controller will support OCE (online capacity expansion), so you can add in additional drives to a current RAID array. Rebuild times will be slooooow, though. If you don't have the funds to do this all up-front, far better to spend more on the underlying hardware (the controller) and buy fewer drives than half-arse both sides of the equation.

[tommyshango]

The drives just have to be the same capacity and not the same brand correct?

I already have 4 1.5 tb drives and dont really want to just toss them out. I really want the extra capacity as well. I dont think they cost too much more the WD are 129.99. I think paying $40 for an extra half TB is a good deal. I realize the seagates had issues but they have all been fixed with firmware updates so its not really an issue any more.

Also cabling this is going to be a chore and the Ultra does not have that many extra cables so I thought a good but cheap extra PSU might be easier. But again it might also be a problem using more juice for no reason and extra heat from running. I am not upgrading my Ultra at this time and have never had an issue with it.

When I get a raid card what brand would you guys recommend? I am thinking 3ware. Also my board has a x8 slot and a few other slots but if I had 2 cards how would I manage that? I use the x16 currently with my 8800 gtx. or am I better off just getting a single card such as one of the 24 ports or just a single 16 and be happy with 16 1.5 tb drives?

Also rebuilding the array takes time but how much time are we talkign here adding a single drive at a time?

I would do raid 6 if I got one of the larger raid cards.

[SoulsCollective]

The drives just have to be the same capacity and not the same brand correct?Correct, but try and use the same model HDD, otherwise the faster drives will be bottlenecked by the slower. Keep in mind it's not just rotational speed, but also cache (amount and speed), platter size, head technology, etc.

I already have 4 1.5 tb drives and dont really want to just toss them out. I really want the extra capacity as well. I dont think they cost too much more the WD are 129.99. I think paying $40 for an extra half TB is a good deal. I realize the seagates had issues but they have all been fixed with firmware updates so its not really an issue any more.Sure, OK. I wasn't referring to the Seagate drives specifically - just the failure rate of 1.5TB drives in general. More platters at higher density = greater chance per drive of failure. But if you've already got some of the 1.5s, I imagine that's not a particularly compelling reason to switch down - I just really, really wouldn't recommend running anything with a lower redundancy than RAID-6.

Also cabling this is going to be a chore and the Ultra does not have that many extra cables so I thought a good but cheap extra PSU might be easier. But again it might also be a problem using more juice for no reason and extra heat from running. I am not upgrading my Ultra at this time and have never had an issue with it.Two PSUs each running at lower capacity will give you less efficiency and more heat output than one PSU running closer to full load. It'll be louder, too. And if you can't power all those drives from your current PSU, and need to get another anyway, then you may as well replace the Ultra - with that much storage residing on it, I really wouldn't risk it on a PSU with known reliability issues.

When I get a raid card what brand would you guys recommend? I am thinking 3ware. Also my board has a x8 slot and a few other slots but if I had 2 cards how would I manage that? I use the x16 currently with my 8800 gtx. or am I better off just getting a single card such as one of the 24 ports or just a single 16 and be happy with 16 1.5 tb drives?While it's possible to span arrays across multiple controllers, it's a real pain to do. You're better off just getting a 12 or 16pt card - and really, are you actually going to be using more than ~15TB (for the 12pt) or ~20TB (for the 16pt, both assuming 1.5TB drives)?

Re brand, I'd steer clear of 3ware - haven't had good experiences with their cards. Reputable vendors would be Highpoint, Adaptec, LSI, Intel or Areca - Areca are generally considered to make the best products out of those listed.

Also rebuilding the array takes time but how much time are we talkign here adding a single drive at a time?Adding drive = full array rebuild. Depending on the card, you may be able to access the array while it's being rebuilt, but it'll be running in degraded mode and will be sloooooow, particularly for writes.

[lutjens]

You're getting the exact case I have...it's a very nice case, but it does have things to be mindful of.

First and biggest issue is that many power supplies have harnesses that can't reach the board (especially those boards with their ATX connectors and SSI connectors on the bottom half of the board). Try to pick a board with the connectors near the top (CPU end) of the board, as it'll be closest to the supply. Also, to reach some of the bays near the top of the case may require some power extensions. With the power supply down low like it is, reaching those bays way up there is a challenge, so keep that in mind.

My solution was to get a PC Power and Cooling 1200W power supply and have the harness customized by PC Power for the case. It cost a pretty penny, but when I installed it, it slipped into the case like a custom made glove and all connectors reach where they're supposed to easily. The thing has been bulletproof and ultra reliable. Well worth the money spent on it in reliability alone...the convenience itself is a large bonus. For my application, I omitted the third set of PCI-E connectors, the 4 min ATX power connector (not required) and customized the length of the SSI and 24 pin ATX headers to fit the board. The power headers to the drives are SATA only (no 4 pin crap at all) and are spaced tighter than a mass produced supply to lower resistance and provide for a cleaner install. Each drive power rail has 6 connectors, with another header providing 6 connectors to the 5.25" bays that I use for additional drive space and a fourth rail with 4 connectors for the optical drives. I still have the PC Power schematic for it.

The hard drive section main fan is quite out of the way, and you'll likely need a 3 pin extension to reach your board (like I did on my D5400XS). My new board has a connector right on the edge, so the wiring from my Delta just reaches...will depend on your board.

The exhaust fan for the slots does come in very handy, although be prepared to move it around to align properly with the side panel grid (it was a little off by default). The CPU exhaust hood is pretty redundant...I turfed mine right away.:)

All in all, the case is an excellent platform and very high quality...you'll enjoy it...:up:

As for controllers, my Areca 1280ML was rock solid and had no issues, except with my new motherboard.:rolleyes: I just received an e-mail from Areca and there is an issue with the motherboard BIOS...they've informed Tyan and hopefully it's corrected soon. The Adaptec 52445 is good, although both the controller and the expander chips run extremely warm (this is where the slot cooler fan on the case came in very handy). The 52445 does have issues with some drives though. Mine had to be jumpered to 1.5Gbps to work, which is no big deal, just an unknown item that caused problems because I was unaware of it. Adaptec's Online Expansion sucks badly compared to Areca's, so go with Areca if that's on your list of possible activites. Neither of your drives appears on their compatibility report or Areca's HD compatibility list either. Also, AFAIK, there is an issue with the Areca 1680 and the Seagate drives you have, so I don't think that'll work. My advice is to go with an enterprise drive like the WD2002FYPS or Seagate ES.2 1TB. These drives are much more tolerant of RAIDing activities, have geater build quality and even more importantly, are much better supported by RAID card companies, who may or may not give a consumer drive the time of day.

Good luck with your build, whatever you decide to do...:)

[tommyshango]

The reason I am needing such massive storage is that I have actually a pretty big dvd collection and did not realize it was getting so big. I want to transfer the entire collection and store it safely and hopefully store it digitally forever. I am tired of losing dvds to scratches or just plain losing them. I have roughly 1200 dvds and probably even more than that on the shelves and if you take an average 6gb each some are more or less then it comes out to 7200 gb . That combined with a music collection is probably close to half of the 20tb of storage. That leaves me quite a bit of room to grow. In a few years I would probably expand again and build a large NAS or something like that and transfer everything there for a double redundency and hopefully do it cheaper if storage capacities contiue to expand.

I will definitely do raid 6 then and probably get at least a 16 port card. I dont know about a 24 port since I wouldnt really have room ever in my case for 24 devices. I only think I can stuff 20 drives into there and I dont know what I would do with the extra 4 ports. Although 20 is more than 16 so maybe thats something to consider especially since the 24 port card would be better overall and not much more in cost than a 16 port card.

I am still confused about what card to get. I have gotten so many conflicting opinions about what brand to go with. I heard Areca's customer service is non-existant. I heard 3ware's is so much better and even if Areca maybe is arguably slightly faster its not worht the headache if you run into problems. I asked about high point at a computer shop and they told me no way its not a real hardware card and I want to do either 3ware or areca. They were the ones who suggested 3ware.

I realize it would be slow rebuilding the array but it sure would be nice to be able to get the card and get a raid 6 setup initially and then add more drives as I needed more storage. This would keep initial costs down considerably and then I only have to spring for about $300 every time I want to add 3 more tb of space. I could set it up at night or something and let it run or do it before I go to work or something. It wouldnt be that often so I really dont see it as a problem and the alternative is trying to get enough funds to buy 16 drives at once. That would mean over $2000 up front just for the drives. It makes more sense to build as I go and get better prices as they go down.

I really dont want to spend money on another psu. If this one fails i will get another one and try a different brand. When I bought that it was about the only psu on the market that was modular and still 1000 watt+. I am sure that has changed. But I just dont see throwing another $3-400 at that part of my build when its already working.

Lutjens - I was able to get my psu in the case and reach without too much trouble. the only issue was the 5.25" burner at the very top but I could have swapped it and put it at the bottom and then put the drives above it. So far I do love the case. I have never had a better one. It was so easy to work with and having a case on wheels is great.

[3Z3VH]

When dealing with this many drives, it is best to dedicate it to its own machine and run it as a NAS. You will save power in the long run because you aren't leaving 3D cards, high end processor, et all idling. You can run it on a MUCH lower-end machine using MUCH less power, and your main/gaming machine will not take nearly as long to boot or index its storage. Not to mention, it reduces networking bottlenecks when other devices on the network need to access files on that storage. The only down-side to making this a NAS, is you will need to buy a case, and a low-end motherboard/cpu/ram for it. Overall, the results will greatly outweigh the meager additional cost.

Edit: And if you really want to store a LOT of drives in one case, I would look at one of the cases with a ton of 5-1/4" bays, and putting hot-swap chassis in them. The max density you can get with 3.5" drives is five drives into three 5-1/4" bays. In a case like the Antec 1200, you could fit 20 hard drives into its 12 front bays, and they would look good, being in hot-swap chassis.

[tommyshango]

I cant have more than one computer right now that is why its done like this. I used to have many other machines and am limited to using one for now.

[SoulsCollective]

The reason I am needing such massive storage is that I have actually a pretty big dvd collection and did not realize it was getting so big. I want to transfer the entire collection and store it safely and hopefully store it digitally forever. I am tired of losing dvds to scratches or just plain losing them. I have roughly 1200 dvds and probably even more than that on the shelves and if you take an average 6gb each some are more or less then it comes out to 7200 gb . That combined with a music collection is probably close to half of the 20tb of storage. That leaves me quite a bit of room to grow. In a few years I would probably expand again and build a large NAS or something like that and transfer everything there for a double redundency and hopefully do it cheaper if storage capacities contiue to expand.Sure, OK. Good luck with digitising all of that - the process takes ages, I know.

I am still confused about what card to get. I have gotten so many conflicting opinions about what brand to go with. I heard Areca's customer service is non-existant. I heard 3ware's is so much better and even if Areca maybe is arguably slightly faster its not worht the headache if you run into problems. I asked about high point at a computer shop and they told me no way its not a real hardware card and I want to do either 3ware or areca. They were the ones who suggested 3ware.Let me guess - they stock 3ware?

Their opinions simply don't stack up with my experience. Yes, Areca customer support can be a little lacking, but when the controllers really just work and you'll be using it on a standard OS, that's really not an issue. Even the drive support mentioned above I've never run into as an issue. As for Highpoint devices being "not a real hardware card", they're talking out of their arse - Highpoint are a manufacturer, same as 3ware, same as Areca, and they make multiple levels of products. Some Highpoint devices have hardware RAID controllers on them, some are just SATA expansion devices - same as 3ware, same as Areca.

If you can give us a rough budget and a preferred store, we can make some more specific recommendations than just throwing out brand names.

I realize it would be slow rebuilding the array but it sure would be nice to be able to get the card and get a raid 6 setup initially and then add more drives as I needed more storage. This would keep initial costs down considerably and then I only have to spring for about $300 every time I want to add 3 more tb of space. I could set it up at night or something and let it run or do it before I go to work or something. It wouldnt be that often so I really dont see it as a problem and the alternative is trying to get enough funds to buy 16 drives at once. That would mean over $2000 up front just for the drives. It makes more sense to build as I go and get better prices as they go down.I agree - I'm not saying that this is a problem, just making you aware of it.

I really dont want to spend money on another psu. If this one fails i will get another one and try a different brand. When I bought that it was about the only psu on the market that was modular and still 1000 watt+. I am sure that has changed. But I just dont see throwing another $3-400 at that part of my build when its already working.
Corsair HX-1000 (http://www.newegg.com/Product/Product.aspx?Item=N82E16817139007&Tpk=hx1000), $220 after MIR with free shipping. And correct me if I'm wrong, but you've said above multiple times that your current PSU is not capable of powering that many drives - meaning that you're going to need to get either a new PSU or another PSU. If you're going to be getting another PSU, I just don't understand why you wouldn't replace a PSU with known issues that you want to be powering part of an array that is going to be used to store your media collection hopefully forever. :shrug:

[tommyshango]

Well I love newegg so thats the store for me.

The shop wasnt suggesting I buy anything from them just an opinion. They know I shop online. I too heard on this forum high point is good so I was confused.

I dont really have a budget for the card since I realize that no matter what its going to cost an arm and a leg. I would rather have the best no matter the cost since I will hopefully use this card for at least 10 years. (probably 5 knowing me but hey..) I dont want to worry about a hundred dollars difference if it means significant performance or feature loss.

That is a nice psu and that is a nice price as well. I guess they have come down in price a bit lately?

I also notice that the specs say 10 sata connectors. That doesnt seem to be much more than the other psu.

[SoulsCollective]

The shop wasnt suggesting I buy anything from them just an opinion. They know I shop online. I too heard on this forum high point is good so I was confused.Always look to the product, not the company. Some Highpoint cards are great value-for-money, some are crap. Same as with any other OEM.

I dont really have a budget for the card since I realize that no matter what its going to cost an arm and a leg. I would rather have the best no matter the cost since I will hopefully use this card for at least 10 years. (probably 5 knowing me but hey..) I dont want to worry about a hundred dollars difference if it means significant performance or feature loss.
Areca 1680 4G (http://www.newegg.com/Product/Product.aspx?Item=N82E16816151051) - four mini-SAS ports, allowing 16 devices out of the box without use of SAS expanders. Cache can be upgraded to 4GB. Probably the fastest desktop controller for your purposes, featuring the dual-core IOP348, but runs you USD $1,500.

Highpoint RocketRAID 3560 (http://www.newegg.com/Product/Product.aspx?Item=N82E16816115067) - six mini-SAS ports, allowing 24 devices out of the box. Slower than the 1680 above (using the single-core IOP341), and features much less cache (1GB, I believe upgradeable to 2GB). Will run you USD $850.

There are of course other options, but that's what I'd go with out of the Newegg offerings.

That is a nice psu and that is a nice price as well. I guess they have come down in price a bit lately?

I also notice that the specs say 10 sata connectors. That doesnt seem to be much more than the other psu.NewEgg specs are wrong, it's got 12. That's AFAIK the highest amount of SATA connectors available for any desktop PSU - for more, you're either going to need to get customised cables done (not hard - just extend the cable and add more SATA power connectors, the PSU's got more than enough grunt to power them), contact Corsair for more SATA power cables (modular design = win), or rely on molex-SATA adapters. The main reason I'm recommending you switch over is reliability, efficiency and noise, not that there's another kilowatt PSU that will have connectors for 20+ drives.

[tommyshango]

Well that high point sounds like a winner. Half the price is worth it to me. I dont care about overall speeds. Either way the raid 6 would be faster than a single drives performance as I have them set up now. So any improvement is fine. Its more the reliability I worry about rather than speed.

One thing I noticed already is I really know nothing about these types of cards. When I was researching I was looking at the number of sata ports. I did not know that mini-sas connected to sata. I really need to read up more on cards and what they do.

Also to consider is a BBU and the cost involved there. I think considering the total cost of the high point with a BBU would still be under $1000 whereas the Areca might be closer to $2000. (I assume I need cables and what not so am thinking there are other costs involved)

[SoulsCollective]

mSAS ports such as featured on either card above can support up to four SATA devices via splitter cables - so simply multiply the number of ports by four. Each card should come with all the necessary cables included, although I'm not 100% certain so you might want to contact NewEgg.

Will you be running this off a UPS? The BBU will help maintain data integrity within the array, but I'd really get a UPS to protect the drives and the card itself from electrical damage and unexpected shutdowns due to power loss.

[tommyshango]

yeah I could get one. It would be nice to have regardless.

Any suggestions on that? smaller the better so it fits under a desk or as discrete as possible.

[SoulsCollective]

UPSes aren't really my area of expertise, especially as you're on a different continent to me so I'm not sure what's available. I could have a look, but I'd feel much more comfortable leaving this one for someone else to answer.

[lutjens]

I need to report a positive experience with Areca support. I reported the issue I'm having with my 1280ML and my new board to Areca support and they found the issue with my board quite quickly and informed Tyan so that they can work on a fix.

I'm hoping the improvement I noticed is part of a trend and not an isolated incident.:)

[saint-francis]

Yeah, Areca has been good to me too. I have had some very good dialogues with them. They have always been very courteous and informative. I had an issue with one of their cards and a SSD. They contacted me every step of the way including cc'ing me all email correspondence both within the company and with the SSD manufacturer regarding the issue.