Switches / 10Gbe / SFP

[Molotox]

Hi, I need to upgrade the network at work to have more server to client bandwidth and was wondering about 10Gbe ethernet. I read a bit about it, but info is hard to come by. So far, I know I'd need a switch with 24-48 1Gb ports with SFP expansion modules and use a CX-4.

Are all SFP expansion slots made equal? If not what kind of SFP expansion slot would I need on a switch to use CX-4 10Gbe modules?

I'm trying to get an idea of the cost of this small setup (only 1 server involved in this environment, and only up to 30ish clients). I know I could use port aggregation to achieve greater bandwidth (over multiple clients, wich is the scenario here), but I'm trying to get an idea of the cost of the better (imo) solution.

[Serra]

Cisco WS-C3560X-24 or WS-C3560X-48 would both suit your needs. PoE versions are also available if you would get value from that for access points or VoIP phones. I can get you a part number on the SFP expansion module as well. I would have done so before finishing this post but my computer decided I hit (?something?) that said "post this now"...


Edit:
Okay. So you're going to have two options for 10Gb, fiber or copper. Personally I'm going to recommend fiber... the prices are typically similar or cheaper and you get more distance out of it, and better availability.

On the Cisco side, if you just want a 24-48 port switch you would probably want something in the 3560X series. There is going to be a bit of a cost difference depending on features you're looking for, but I'm going to take a stab here.

Assumptions:
- 48 ports
- Layer 3 switching (routing) is desired
- Do not require "advanced" L3 features like BGP, OSPF... basically, RIP is the only dynamic routing protocol... maybe EIGRP stub mode as well.
- No PoE requirement (though I would opt for a PoE model... VoIP is the bees knees)
- Redundant Power Supplies are not required (for pricing, you can get an additional PSU for these)
- No vendor support required (again, available, just not priced out here)
- Only 2x 10GbE SFP ports are required
- You do not require the ability to stack this switch for redundancy or performance (that would be a slightly more expensive chassis)

Switch:
Model - WS-C3560X-48T-S
Price - $7500 list - ~$4700 actual

10Gbe Expansion Module (not interchangeable with other vendors equipment):
Model - C3KX-NM-10G
Price - $2500 list - ~$1600 actual

10Gb SFP+
Model - SFP-10G-SR= (multimode)
Price - $1500 list - ~$975 actual

Total: ~$7,275 (actual)


At least, that's the Cisco solution. I can't comment on other vendors.

[Molotox]

I just checked out the price on those models

Any cheaper solutions that can support 10Gbe modules? I doubt my boss is ready to dish out that kind of money on the network upgrade (I figured 1500$-2000$ for a switch would be the max he'd be ready to pay). Looks like I'll be relying more on aggregating ports togheter to achieve higher bandwidth.

Thanks for the quick answer Serra, I knew I'd be getting an answer from you by posting here

[Serra]

Yeah, it's still not quite that cheap. You can save money by dropping to layer-2 only and only going with 24 ports, but then you're also stuck putting the server on the same subnet as the clients and you still won't hit the $2000 mark.

Again I can only speak to Cisco with any authority, but I don't think HP has anything in that range either. HP is probably Cisco's main competition in that cost space, so if neither of them has it out I doubt anyone else does.

If nothing else you can use up to 8x ports aggregated together on a single link for up to 8Gb/s distributed throughput. Of course at that point you're adding in so many NICs at that point that the server starts to become a bit of a frankenstein.

Anyway, before you start making port channels you should make sure your switch is non-blocking (many/most switches are designed for oversubscription), or if it is oversubscribed how that works. For example, a common oversubscription model would be to tie ports 1-4 to one ASIC, 4-8 to another, etc.... though with that said I have also seen switches with some truly bizarre and unexpected interface mappings. The result is that if you want full bandwidth from two different ports you should actually make sure they're on separate ASIC's.

[Molotox]

Yeah, it's still not quite that cheap. You can save money by dropping to layer-2 only and only going with 24 ports, but then you're also stuck putting the server on the same subnet as the clients and you still won't hit the $2000 mark.

Again I can only speak to Cisco with any authority, but I don't think HP has anything in that range either. HP is probably Cisco's main competition in that cost space, so if neither of them has it out I doubt anyone else does.

Well, I knew it was not cheap, but I had no idea how much exactly. I don't need L3 on the switch, it always nice to have but not a must. Don't need much in terms of security either (considering the environment : a small PC repair shop, can have up to +/- 30 clients on the network).

If nothing else you can use up to 8x ports aggregated together on a single link for up to 8Gb/s distributed throughput. Of course at that point you're adding in so many NICs at that point that the server starts to become a bit of a frankenstein.

10Gbe would have been nice only for *futureproofing* the network side of the shop, as the current server can't even feed that much with it's storage. I probably won't need 8x ports aggregated, but will surely go for 4-6.

Anyway, before you start making port channels you should make sure your switch is non-blocking (many/most switches are designed for oversubscription), or if it is oversubscribed how that works. For example, a common oversubscription model would be to tie ports 1-4 to one ASIC, 4-8 to another, etc.... though with that said I have also seen switches with some truly bizarre and unexpected interface mappings. The result is that if you want full bandwidth from two different ports you should actually make sure they're on separate ASIC's.

ASIC? You lost me there. I'm currently studying networking (done TCP/IP, OSI model stuff, now doing a basic Cisco class, +/- CCNA level) so lots of stuff I don't know


In all cases, any new solution will be better than the current setup of the shop. We have the server connected to a 24 ports 10/100 switch, in wich we have 2 or 3 8 ports switches cascaded to make up for the lack of ports... I'm planning to deploy a WDS solution to ease / automate OS installations and the current network infrastructure won't cut it. The same server is used to store client backups, so a lot of large transfers is going on on the NIC as well. So far, 1 client can saturate the server's throughput, so it slows down the work a lot.

Thanks for your insight

[Serra]

ASIC? You lost me there. I'm currently studying networking (done TCP/IP, OSI model stuff, now doing a basic Cisco class, +/- CCNA level) so lots of stuff I don't know


In all cases, any new solution will be better than the current setup of the shop. We have the server connected to a 24 ports 10/100 switch, in wich we have 2 or 3 8 ports switches cascaded to make up for the lack of ports... I'm planning to deploy a WDS solution to ease / automate OS installations and the current network infrastructure won't cut it. The same server is used to store client backups, so a lot of large transfers is going on on the NIC as well. So far, 1 client can saturate the server's throughput, so it slows down the work a lot.

Thanks for your insight

ASIC stands for Application Specific Integrated Circuit. I'm not quite up to talking about the specifics, but basically it's what you call the piece of hardware that makes some of the actual forwarding decisions on the switch. In a typical design you'll see a flow kind of like the following:
- PC sends data to the switch (ASIC)
- The ASIC determines whether the destination is on the same ASIC or not
- If the destination is on the same ASIC it is forwarded out the correct port, typically at line rate
- If the destination is not on the same ASIC the request is forwarded to some other type of circuit/fabric which forwards it to the correct ASIC. This forwarding process is where one will most likely and generally see a bottleneck.

[Molotox]

Ok.. So it is basically the switching within the switching? Neat

[rogard]

check out extreme or force 10 - both might be good alternatives if cisco for you.

and if your doing alot of lab stuff check out gns

one of my better labs is here:
http://www.networking-forum.com/blog/?p=246

it might also we worth checking out ebay they have some great ccna/p/ie lab kits already to go

[Molotox]

Thanks for the labs link, those will come in handy