post your hard drive benchmarks :)

Quote:

---

Originally Posted by WoLpH

Perhaps a little off-topic, but as I'm facing the decision to get a new system and I have a pretty high-end disk array, I'm wondering what the performance would be using a 'normal' motherboard?

My current workstation (dual opteron 246), has an Areca 1280ML with 16x WD5000AAKS on it and 2GB cache. But as these processors clearly need an upgrade I'm thinking of buying either a fast Core 2 Quad or a (most likely less fast because of the price) Xeon system. So I'm wondering, has anyone around here have experience with a regular desktop system running a high-end disk setup? I haven't found anyone with a similar setup in this topic.
Since workstation motherboards don't offer any overclocking possibilities and are a lot more expensive I find a Core 2 Quad a nice option, if my raid card will still be used to the maximum. I really like the high disk performance I have right now.

PS: I only run Linux and Unix, so some hardware might not be completely compatible/possible.

---

If you're not going to change your disk subsystem (ie, type of drives, or separate your work flows to different spindle groups et al) you're not going to see much at all of an improvement. Basically your computer MB/cpu, et al is generally NOT the bottleneck. It's mainly the drive subsystem that kills you (at least first, there are numerous bottlenecks in a system but that's a much bigger discussion a decent reference would be John Hennessy's/David Patterson's book "computer architecture: a quantitative approach")

Before anyone can really point you in a direction a lot more information would be needed to model your type of access requirements and setup. I've just posted in the storage/raid thread some spreadsheets to do some quantitative analysis but they are only a start.

http://www.xtremesystems.org/forums/...=150176&page=2

Really comes down to is what is your workload type, and what performance level are you seeking. From that then you design the system to match. There's also the issue of disk subsystem utilization curves which come into play (Queuing theory & Markov models like the M/M/1 or M/M/m).

Quote:

---

Originally Posted by stevecs

If you're not going to change your disk subsystem (ie, type of drives, or separate your work flows to different spindle groups et al) you're not going to see much at all of an improvement. Basically your computer MB/cpu, et al is generally NOT the bottleneck. It's mainly the drive subsystem that kills you (at least first, there are numerous bottlenecks in a system but that's a much bigger discussion a decent reference would be John Hennessy's/David Patterson's book "computer architecture: a quantitative approach")

---

My concern is mostly based on previous systems where the raid array would easily do 100MB/s and more but when transferring via the network the PCI bus was immediately completely saturated and because of that limiting the network transfers to 60MB/s. Because of the PCI-E lanes these restrictions obviously no longer apply (the shared PCI bus I mean), but since I haven't had the chance of testing a setup like this with a 'normal' motherboard I wasn't sure what to expect.

Quote:

---

Before anyone can really point you in a direction a lot more information would be needed to model your type of access requirements and setup. I've just posted in the storage/raid thread some spreadsheets to do some quantitative analysis but they are only a start.

http://www.xtremesystems.org/forums/...=150176&page=2

Really comes down to is what is your workload type, and what performance level are you seeking. From that then you design the system to match. There's also the issue of disk subsystem utilization curves which come into play (Queuing theory & Markov models like the M/M/1 or M/M/m).

---

Fantastic information, this is something I can work with. Although the Queueing theory and Markov models usually only come into play when using higher loads, it's still a very interesting read, thank you very much.

As for my workload type, it's mostly transferring big files over the network (big as in 100MiB and more) to about 10 clients simultaneously so the queueing part will be quite simple and will most likely not give problems. My performance concerns are primarily related to the hardware limitations of the system.

It looks like I have enough information to start a proper analysis, perhaps I'll try calculate the Markov chain for it, it could give me some new insights.

Correct on those models require a system in equilibrium and the number of incoming requests is unlimited among other items. but it sounds like you know that.

As for network transferring of large files another item that I've discovered recently is that switches are much more the bottleneck than normally thought for transferring large amounts of data. I've run into some tests with cisco switches (3750's, 3xxxx, 4xxxx et al (haven't tried the various 65xx's yet) that they cannot push full GbE speeds on their nics. They do generally OK in one direction but not in both (send/receive) looks like they are capped at 1GbE total per nic (ie one direction).

Then there is network filesystem overhead (like SMB/CIFS) which just suck at high speed performance (usually capped at around 60MiB/s or so per client connection).

Yes, I have some knowledge about queueing theory. As I'm currently programming with distributed systems I know a bit about multithreading and proper/fast ways to handle queues.

As for network limitations, that is one of the reasons I'm not a big fan of Cisco equipment, I'll take Foundry or Juniper over Cisco any day. Or, if money is a big issue, HP switches have been performing quite good for me aswell, 100-110MiB/s in both directions seems to be possible without the switch giving the least bit of problems (HP ProCurve 2824 or 2848 switches with 24 or 48 1GbE connections). The network won't be the limit here, perhaps the specific network cards though, as the onboard nic of my K8WE only does 75-80MiB/s while the 3c985 fiber channel nic does 120MiB/s easily.

And yes... network filesystems, I'm currently using NFS but I'm indeed not too happy about the performance. Perhaps I should consider switching to a virtual iSCSI system or ATA over Ethernet. But the last time I've tried ATA over Ethernet it was a bit unstable, it caused the kernel to lock up once in a while (and crashes in combination with ATA over Ethernet result in filesystem corruptions) so I'm not that eager to try it again.

Either way... I'm gonna look for a new motherboard with atleast 1x PCI-E 16x for the videocard, 1x PCI-E 8x for the raid card, and preferably PCI-X for the network cards (PCI-E fiber cards are still quite expensive at the moment, and I already have a load of PCI-X network cards lying around), if anyone has a tip I'm open for suggestions ;)
I'm currently looking into Gigabyte but perhaps I'll have a look at Asus, Tyan or SuperMicro aswell.

You may want to try NFS v4 if you're not already due to some of it's optimizations w/ command merging among other items. Bit more of a pain for clients/setup though. I haven't really tested that in a distributed environment yet though.

As for MB's I've used the asus ones (P5W64WS Pro) has 3 8x & 1 4x pci-e, (I've been trying to avoid pci-x as I'm looking more towards 10GbE and pci-X can't handle it). Main item though if you're looking for end-to-end you want an Intel based nic. I haven't found any of the other ones (broadcom, marvell, 3com, et al) able to push full wire speed in full duplex. For workstation classed boards both Tyan & Supermicro have been good in the past for me.

Good luck. and thanks for the tip on the juniper/foundary I'll check out some of those.

Thanks for the NFS v4 tip, I haven't tried switching to it yet. The total enviroment is not really distributed though, just some of the software I'm writing for work :)

10GbE sounds great to me, but it's still way to expensive imho, trunking a few gbit lines (dual or quad gbit connections) should be enough for now.

After your recommendation I've given Asus a good look and I think the P5E64WS Pro might be an option if I'd decide to switch to PCI-E, or the P5E3WS Pro if I'll stay with PCI-X. I'm looking at some Intel PRO/1000 PT dual port adapters right now, it seems they aren't too expensive. I'm actually considering a motherboard that would allow a little overclocking too, and I haven't found a Tyan/SuperMicro motherboard that allows that unfortunately (yeah, under windows with some tools, but I don't run windows ;)).

4 x 150GB Raptors in Raid0, 128KB stripe size, Areca ARC-1210
http://img187.imageshack.us/img187/3...recaarcod2.png
http://img296.imageshack.us/img296/7...recaarckf7.png
http://img259.imageshack.us/img259/8...hmarkarty3.png

Mien looks pretty good

As you can see I have a Raptor. The transfer and burst levels aren't spectacular but, the access times are.Attachment 72027

160Gb SATA @5400 rpm on my

32Mo
http://www.jmax-hardware.com/images/...0/hdtach32.jpg

8Mo
http://www.jmax-hardware.com/images/...00/hdtach8.jpg

Two Hitachi Deskstar 320gb (7200rpm, 16mb cache) RAID 0

Here is a Skulltrail bench with onboard RAID

Vista Ultimate 64 bit
4x 150GB RAID 0

http://fugger.netfirms.com/D5400XS/hdvista.jpg

I need to get that Areca card.

This what mine 4x Samsung F1 32mb 1TB does in RAID0

http://members.lycos.nl/niscoracing/...ng%20RAID0.png

Just a single Seagate 7200.10 and it beats a craptor :D

http://img166.imageshack.us/img166/9...hnj1tx1.th.jpg

Seagate 7200.11 500GB.
http://scoombes15.googlepages.com/seagate.jpg

Nisco, that's impressive.

Nisco, I was looking at those 32M drives, looking even better now.

Fugger

get in touch with the peeps @ fusion-io to get a sample

the iodrive would be a great addition to skulltrail

at least thats what im planning on..

4x Seagate 7200.10 500gb
ICH9R 0+1 64kb stripe Write Cache Enable
P5K-E Q6600 @3.2 2GB
Vista x64


http://aycu25.webshots.com/image/435...2757287_rs.jpg

http://aycu38.webshots.com/image/422...0098377_rs.jpg

Quote:

---

Originally Posted by NiscoRacing

This what mine 4x Samsung F1 32mb 1TB does in RAID0

http://members.lycos.nl/niscoracing/...ng%20RAID0.png

---

I want to set mine up in a striped set but, I can't afford another raptor at the moment. Your access times and transfer are incredible. Those are solid state drives right? Thats what is on my wish list next. Waiting for the price to come down.

Quote:

---

Originally Posted by james bennett

Those are solid state drives right?

---

That is not SSD !

That is :

http://xtreview.com/images/samsung-s...t-F1%20-01.jpg

2xSamsung 500GB 1xWD 500GB

Samsung SpinPoint T166 HD501LJ - 500GB

http://img517.imageshack.us/img517/3...ng500gbct2.jpg

http://img517.imageshack.us/img517/4...amsunghxo5.png

Samsung SpinPoint T166 HD501LJ - 500GB 2nd

http://img352.imageshack.us/img352/7...ng500gbjr2.jpg

http://img87.imageshack.us/img87/744...amsunghwa1.png

WD5000AAKS

http://img151.imageshack.us/img151/3...dcwd500lk4.png

http://img141.imageshack.us/img141/4...3102007yn5.jpg

All tested on same rig:

Asus p5k-e wifi
4x1gb Geil 1ghz@1130
2140@3ghz

Quote:

---

Originally Posted by Pudi

That is not SSD !

That is :

http://xtreview.com/images/samsung-s...%20-01.jpg

---

I can't believe the transfer rates and the access time. It has to be reading it in a striped set right? (from the three disks). How else can it get such high transfer rates. I think I might be pricing this INSTEAD of solid states.

Two 320GB 7200.10 Seagate drives. 100GB Intel Matrix partition.

http://i159.photobucket.com/albums/t...henator/hd.jpg

Maxtor 300GB | SATAII | 16MB

nothing too spectacular here:
31.7 MB/s average
0.7ms access time ;)

eee's internal ssd