Project: My first FreeNAS server! 2.25tb of storage...drool.

Hey everyone...here's my latest project!

=== skip this section if you don't care about the background on how i got this hardware ===

Recently my old boss called me to ask for help with his server. I had built him a file server awhile back. He wanted the option of using it as a workstation as well (mainly internet/email stuff) so I put it together and loaded XP. Long story short, I put the thing together really well and it even had a fully automated backup system. The storage part of the system consisted of (4) 500gb SATA hard drives. The drives were configured into (2) hardware RAID-0 arrays with software mirroring between arrays (one terabyte of storage + one terabyte of backup).

Shortly after I left that job, one of the 500gb hard drives went bad. No big deal - the system was built to lose a drive. I went by one day and pulled the drive, gave it to my boss along with an RMA number and instructions to mail it out to Western Digital for a replacement. Long story short - he never sent it out. A few months later, I get a call from him saying that the storage drive is no longer showing on the server. I go over to check out the server, and someone has tampered with the RAID configuration, effectively killing it. Since my boss never sent out the bad drive, and it was never replaced, they were running without a backup.

My boss offered to give me the whole machine (with the drives) if I could recover all his data. I told him to order a 1tb RAID1 (1tb + 1tb) NAS drive, which he did, and then I used RAID recovery software to extract all of the data off of the old server.

=== end backstory ===

So my boss got all his data back, and I got a free computer!

Here are the specs on the machine:
- ABIT KN9 AM2 NVIDIA nForce4 Ultra ATX AMD Motherboard
- AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
- 512mb DDR2 RAM
- (3) WD 500gb SATA HDDs (there were 4, but one locked up)
- (1) WD 80gb IDE HDD
- DVD+/-RW IDE Drive
- EVGA 8600GT PCI-e Video Card
- Enermax Liberty 500w Modular PSU
- Generic Black Mid-Tower Steel Case (pretty nice actually...120mm fans, etc.)

Besides the free hardware, I also had the following sitting around:
- (1) Hitachi 750gb SATA HDD
- (1) WD 500gb SATA HDD

Then I ordered:
- (1) Hitachi 750gb SATA HDD
- (1) WD 500gb SATA HDD
- SYBA SD-SATA2-4IR PCI SATA II Controller Card RAID 0/1/5/10 JBOD
- LIAN LI PC-A17B Black Aluminum ATX Mid Tower Computer Case
- (2) Lian Li EX-34NB Expansion Hard Drive Kit - Black

Stock photos (until I can take my own):

New case...coming Thursday:
http://www.pccasegear.com/UserFiles/a17011.jpghttp://www.pccasegear.com/UserFiles/a17005.jpghttp://www.pccasegear.com/UserFiles/a17025.jpg

HDD racks, ordered two, also coming on Thursday:
http://c1.neweggimages.com/NeweggIma...988-006-03.jpg



Anyway, I wanted to give FreeNAS a try. After downloading and installing the latest build (currently using RC2 build), I began to mount up my drives and create RAID arrays.

In total I have:
(1) 80gb IDE HDD
(2) 750gb SATA HDDs
(5) 500gb SATA HDDs

Currently I have them configured like so:

(1) 80gb IDE HDD - Primary, FreeNAS OS + Internal Storage
(2) 750gb SATA HDDs - RAID1, 750gb usable space
(2) 500gb SATA HDDs - RAID1, 500gb usable space
(3) 500gb SATA HDDs - RAID5, 1tb usable space

2.25tb of usable space! Woowoo!!!

I have currently I have the box configured as network storage, FTP server, HTTP server, BitTorrent client, and iTunes server. I am using Dynamic DNS so I can access the server from any computer (including my Windows Mobile phone) connected to the internet.

http://img206.imageshack.us/img206/9480/freenasfl9.jpg

Through my Linksys WRT54G 10/100 Wireless Router (running DD-WRT), I only get about 8mb/s transfer speed, however when connected directly to my desktop via 10/100/1000, I am seeing up to 50mb/s transfer speed.

Reasons why I like my new FreeNAS box:
1) Most of the hardware was FREE!
2) 2.25tb of storage! All of it in either RAID1 or RAID5, so I don't have to worry about losing data in the event of drive failure.
3) Drives go to sleep after 10min of being idle. Hopefully this saves on my power bill
4) BitTorrent client, FTP server, HTTP server, SSH, etc. etc. etc.
5) Now I only have one hard drive in my desktop - leaving plenty of room for the soon-to-come watercooling setup.

The new Lian-Li case & HDD racks come on Thursday - I'll post pics when I get it all set up again!

Cheers! :up:

Nice setup, looking forward to the pics.

Rememeber to get a UPS to protect your data and your machine, its pretty much a must for always on machines (even those sleeping) especially with RAID.

Get a nice APC 1000 smartups from ebay, they are much cheaper than retail and supported by freebsd.

is FreeNas, just a program for windows?
for example my server runs 64bit Xp Pro
I like this because I'll be getting a windows mobile phone soon.

freenas is a whole os based on freebsd.

FreeNAS is an operating system based on FreeBSD, so no, it is not a program for windows.

I love my FreeNAS box, and if you are looking to save just a little bit of power with that setup, i would look into a compact flash card and and ide converter for the OS drive. i have my FreeNAS OS installed on a 4gb CF card and it runs beautifully.

I have a 1tb FreeNAS machine and a 3.2tb OpenFiler machine and I had a hard time deciding which to stick with, so I continue to use both, as they both have their strong points.

for some reason the case came today (tracking number said it was coming tomorrow...?)

i moved the hardware into the new case, but for now i'm still using the stock hdd bay + the hdd bay from the old case. anyway, here are some pics. i will take better ones once the lian-li hdd racks come.

http://img143.imageshack.us/img143/3017/dsc1007hy7.jpg

http://img143.imageshack.us/img143/3189/dsc1018yq7.jpg

the case is really beautiful in person. exceptional quality, as expected. the only thing i dont like is the silver aluminum inside. i might pull everything and paint it all matte-black on the inside.

do you really need that 8600gt in server?

it just make some heat..

Quote:

---

Originally Posted by theorie

3) Drives go to sleep after 10min of being idle. Hopefully this saves on my power bill

---

i've been told that that ^ is a big no-no - it means that ur HDDs stop/start more often, thus more wear'n'tear & thus decrease life :down:

I would suggest not doing it..
plus, if u have enterprise class HDDs, they are specifically made to be 'on' 24/7 :yepp:

:up:

Quote:

---

Originally Posted by pcaddict

do you really need that 8600gt in server?

it just make some heat..

---

yes you are right. i actually had a fanless geforce 6200LE pci-e video card...but i gave it to a friend.

i think this is it:
http://www.pricebat.ca/productimage/normal/10071228.jpg

(long story short - he had two evga 8600GT cards, for some reason his asus mobo would not work with both cards, so we put the 6200LE in as the second card and it worked fine. he's not using his machine for gaming - just financial stuff, so he doesn't care what card it is)

i am going to get another fanless low-end video card the next time i place an order with newegg.

Quote:

---

Originally Posted by tiro_uspsss

i've been told that that ^ is a big no-no - it means that ur HDDs stop/start more often, thus more wear'n'tear & thus decrease life :down:

I would suggest not doing it..
plus, if u have enterprise class HDDs, they are specifically made to be 'on' 24/7 :yepp:

:up:

---

hmm really? well i don't want to have a huge ass power bill so i think i'll risk it. all of the data has redundancy (only using raid1 or raid5), so if a drive goes bad i'll simply replace it.

i wonder what the difference in power consumption is...how can i figure that out (besides experimenting with my power bill...haha)

HDD racks came today...

http://tfrascone.net/myphotos/g2data...1/DSC_1020.jpg

http://tfrascone.net/myphotos/g2data...1/DSC_1023.jpg

http://tfrascone.net/myphotos/g2data...1/DSC_1028.jpg

http://tfrascone.net/myphotos/g2data...1/DSC_1026.jpg

http://tfrascone.net/myphotos/g2data...1/DSC_1025.jpg

Nice cable management!

How is the airflow over the drives in the cages? Are you able to monitor the drives with SMART? If so, do temps stay below 35-38? Thanks.

I have a similar Lian-li full tower hosting a soon to be large array and will be needing a cage or two.

Quote:

---

Originally Posted by theorie

i wonder what the difference in power consumption is...how can i figure that out (besides experimenting with my power bill...haha)

---

well 1 HDD takes *roughly* 8W @ idle.. yes thats right, a mesely 8W.. so u got 8 HDDs, thats 8x8= 64W ... now @ load HDDs *roughly* take only an extra 1W or 2.. so an absolute max of 80W for all 8 HDDs @ load - & ur willing to risk it for max 80W? :/

thats a completely overkill set up for FreeNAS....


I LOVE IT

I'll have to chalk a Free-Nas project on my To Do list.. along with a Smoothwall project sometime next year.

Quote:

---

Originally Posted by [XC] riptide

I'll have to chalk a Free-Nas project on my To Do list.. along with a Smoothwall project sometime next year.

---

i had fun with a little project recently

http://raqcop.com/

i picked up a raq3 for 33 bucks shipped from ebay
there are several raq4's going for around 50-60 shipped on there...

i like to consider it geek fun

Quote:

---

Originally Posted by [XC] riptide

I'll have to chalk a Free-Nas project on my To Do list.. along with a Smoothwall project sometime next year.

---

If you change the smoothwall to IPCOP, I can help you with that Riptide (just a different distro). Really nice Freenas box. I've always wanted to make on, but I always do other stuff. Maybe over holiday break . . .

Quote:

---

Originally Posted by [XC] moddolicous

If you change the smoothwall to IPCOP, I can help you with that Riptide (just a different distro). Really nice Freenas box. I've always wanted to make on, but I always do other stuff. Maybe over holiday break . . .

---

Or he could stay with the dark side (freeBSD) and try out pf-sense

Wow! Very clean

I thought that mobo have integrated graphics.. Awesome cable management! :up:

thanks for all the feedback guys!

Quote:

---

Originally Posted by rogard

Or he could stay with the dark side (freeBSD) and try out pf-sense

---

yes, i have been eying pf-sense lately...:yepp: just gotta figure out what i would want to use for hardware first...

If pf-sense is anything like IPCOP (I'm gonna guess it is) than lower end hardware should be more than enough (P3 and 512mb ram and a small hard drive unless you want to run some weird stuff).

pfsense rocks, but i like to change up my setup every now and then...
smoothwall has lots of kickass features and addons
ipcop is minimal, but does the job its designed to do. my new raqcop box is pretty cool

ended up getting a used FANLESS NVIDIA GEFORCE 6200 TC 256MB PCI-E card off ebay for wicked cheap.

even matches the color of the mobo!

http://www.retekdirect.com/images/eb...56PCIEred1.jpg

Quote:

---

Originally Posted by Gogeta

Nice cable management!

How is the airflow over the drives in the cages? Are you able to monitor the drives with SMART? If so, do temps stay below 35-38? Thanks.

I have a similar Lian-li full tower hosting a soon to be large array and will be needing a cage or two.

---

http://img440.imageshack.us/img440/538/hddtempsgr7.jpg

I've also noticed that network speeds have been a little...well...slow. :shakes:

Most likely because I'm using a 10/100 router. While it is a very nice router (see my other thread with the mods I've done to it...), it's not gigabit so when transferring large files over the network, it tends to be a little slow.

What's really cool is that I am going to setup my FreeNAS box & Primary RIG to use Link Aggregation Control Protocol (LACP), meaning each computer will be connected to the switch via TWO cat-6 network cables & 2 NIC cards. This should definitely speed up my network... :D

To solve this problem, I just ordered the following:

+ Cisco SMB SLM2008 10/100/1000Mbps Switch (8 x RJ45 8K MAC Address Table)
http://www.smallnetbuilder.com/image...2008_front.jpg

+ LAN Cable Crimper + Cable Tester + Striper + Impact Tool
http://www.kc-kwan.com/online/pics/o...g_tools_01.jpg

+ (50) 8P8C Network Cable Modular Connectors
http://imgs.inkfrog.com/pix/besteworld/1004.jpg

+ (100) RJ45 Plug Boots - Red
http://img.auctiva.com/imgdata/1/0/4...3349137_tp.jpg

+ 100ft CAT6 Network CABLE - Black
http://content.glidesociety.com/imag...f-ebcc6d5a2960

Most of the network cable parts are coming from China...so that might take a week or two to get here, but the router will be here within a few days. Looking forward to having a gigabit network with my new FreeNAS box!

What's really cool is that I am going to setup my FreeNAS box to use Link Aggregation Control Protocol (LACP), meaning it will be connected to the switch via TWO cat-6 network cables & 2 NIC cards. i think i can also do this with my primary computer using the nvidia system tools. This should definitely speed up my network... :D

Ditch the lackluster onboard NICs for an Intel dual port adapter.

Money well spent, no question.

Quote:

---

Originally Posted by Gogeta

Ditch the lackluster onboard NICs for an Intel dual port adapter.

Money well spent, no question.

---

actually, right now i'm waiting on a pair of pci-e 1x gigabit nic's. hopefully they come soon.

just installed my new cisco slm2008 switch. works great so far. i have my PC connected to it via LACP (using both mobo NIC's), and the FreeNAS box will be connected via LACP also once the new NIC cards come. :)

The two PCie 1x Gigabit NIC's came. I could be wrong, but I'm pretty sure they are some sort of Hong Kong knock-off of this Rosewill adapter. The two NIC's I bought came off eBay, and were under $15 each, shipped.

Turns out these things are plug & play with FreeNAS - no extra drivers to load or configure.

Currently using both NIC's with LACP teaming.

All I'm waiting for now is my cable crimper so I can replace all my cat-5 network cable with custom-made cat-6.

Heres a shot of the box with the new eBay PCIe 1x Gigabit NIC's + the new fanless video card that I installed a few weeks ago. You can also see the CF card + CF --> IDE adapter (mounted on floor of the case):
http://tfrascone.net/myphotos/g2data...4/DSC_2106.jpg

try getting an intel card that uses the e1000 driver (fxp) on BSD systems they work very well with the 6.4 kernel and the 7+ kernels.

drag & drop transfer speeds between the PC & the FreeNAS box:

http://img25.imageshack.us/img25/8285/lacpspeedsqj9.jpg

not bad!

what the processor utilization like? you should be getting more than that. Try and get about 100Mbyte/sec

Very nice FreeNAS project! I am a privately contracted computer engineer and I often get to bring home some really nice goodies as well (when the job is done, of course). I recently acquired a single shelf NetAPP FAS2020 with 300GB SAS drives and I love it. Having your own NAS to play with is so nice isn't it?

Quote:

---

Originally Posted by rogard

what the processor utilization like? you should be getting more than that. Try and get about 100Mbyte/sec

---

yes but i think the SATA-II controllers are somewhat of a bottleneck.

on the reviews for the controllers, i read that someone else was getting 75MB/s transferring to a single drive, so i would imagine that the software RAID would slow it down as well.

...actually, now that i'm looking at it, i'm noticing that the MTU is set to 1500. i believe it should be set to about 9014. i think i need to do some tweaking...

Ya... I got 75Megs/sec from Raptor 74x2 Raid 0 through a Sweex (lol) gigabit switch. I ask it to copy 1 large files to 3 other machines on the switch simultaneously. So basically you should be getting a lot moar. My MTU was set 576 , but that was for other reasons.

one reason why living in manhattan is awesome:

it's really easy to come across free computer hardware just sitting on the street.

i don't know why, but people throw out perfectly good computer hardware all the time! full desktops, monitors, server equipment, etc.

today while biking home i saw an awesome 19" rack server cabinet just sitting on the curb. it's about 5' tall and in good shape, except it's missing one of the side panels (no big deal).

i didn't have any way to get it home today, but on saturday i have rented a truck (moving from nyc to brooklyn) so i'll be able to pick it up...if it's still there. i locked it to a light pole using my massive kryptonite bike lock, and left a sign on it asking nyc sanitation that i would be picking it up, so i think it should still be there by saturday.

this thing will be perfect for storing all my computer stuff. i plan on using it to house my modem/router/switch, freenas server tower, an additional workstation tower, plus a rack-mounted untangle firewall. i guess i'll make new side panels for it, and perhaps some sort of mesh or glass door for the front. either way, this saves me about $150 - $300 that i would have spent on a similar used cabinet (shipping is a !*@#$% on these things).

well, fingers crossed that it's still there!

http://img4.imageshack.us/img4/1555/19inrackstreet.jpg

BTW with teaming, i think your switch has to support 802.3ad to create an equivalent 2 Gbit/sec connection. Otherwise, it's load balancing.

Quote:

---

Originally Posted by Fuji

BTW with teaming, i think your switch has to support 802.3ad to create an equivalent 2 Gbit/sec connection. Otherwise, it's load balancing.

---

yes, my switch supports 802.3ad - check out the specs here:
http://www.smallnetbuilder.com/content/view/30226/51/

Quote:

---

Originally Posted by theorie

one reason why living in manhattan is awesome:

it's really easy to come across free computer hardware just sitting on the street.

i don't know why, but people throw out perfectly good computer hardware all the time! full desktops, monitors, server equipment, etc.

today while biking home i saw an awesome 19" rack server cabinet just sitting on the curb. it's about 5' tall and in good shape, except it's missing one of the side panels (no big deal).

i didn't have any way to get it home today, but on saturday i have rented a truck (moving from nyc to brooklyn) so i'll be able to pick it up...if it's still there. i locked it to a light pole using my massive kryptonite bike lock, and left a sign on it asking nyc sanitation that i would be picking it up, so i think it should still be there by saturday.

this thing will be perfect for storing all my computer stuff. i plan on using it to house my modem/router/switch, freenas server tower, an additional workstation tower, plus a rack-mounted untangle firewall. i guess i'll make new side panels for it, and perhaps some sort of mesh or glass door for the front. either way, this saves me about $150 - $300 that i would have spent on a similar used cabinet (shipping is a !*@#$% on these things).

well, fingers crossed that it's still there!

http://img4.imageshack.us/img4/1555/19inrackstreet.jpg

---

When I lived in Milwaukee we found all kinds of goodies like that. Of course I'm sure not as kick ass as Manhattan. One time we found a tower walking home from the bar that had a bad video card on it, so I dug up my old ti 4200 and stuck in there, and dude had a s--t ton of itunes tracks and all kinds of stuff I wouldn't want others to see... Honestly, I formatted it on the spot and my friends are still using to this day. It's crazy what people throw away.

well...i finally got a third Lian-LI EX-34NB.

apparently there are two versions of the unit, which i found out the hard way. the only way to tell them apart (without opening the box) is to check the UPC label. here is the difference:
http://tfrascone.net/misc/ex-34nb/ex-34nb_old.jpg
http://tfrascone.net/misc/ex-34nb/ex-34nb_new.jpg

as you can see, the two that i already had were the "old style". i ordered another unit and the one that came was a "new style" so i had to return it, then search around for a reseller that still carried the old style. i finally found one "old style" unit at sundialmicro.com (they said it was their last one!).

here's the setup now with all three EX-34NB's installed, plus some pics of the pcie 1x gigabit cards.

oh, and still unpacking at my new apt, so i haven't had a chance to setup anything in the 19" rack cabinet yet.

http://tfrascone.net/myphotos/g2data...4/DSC_2289.jpg

http://tfrascone.net/myphotos/g2data...4/DSC_2291.jpg

http://tfrascone.net/myphotos/g2data...4/DSC_2287.jpg

http://tfrascone.net/myphotos/g2data...4/DSC_2286.jpg

http://tfrascone.net/myphotos/g2data...4/DSC_2283.jpg