the distro kunundrum...

[jarod-ajLrJawYSntWk0Htik3J/w at public.gmane.org (Jarod Wilson)]

On Nov 16, 2009, at 1:27 PM, Stephen Adler wrote:

> Well, here's what I did.
> 
> I have a program called tcpBlaster which I use to test network speeds. 
> it basically opens up a socket connection and dumps bits over it as fast 
> it can go. There is no disk io involved.
> 
> The rates that I'm getting are 7MBytes/sec into the system, 15MBytes/sec 
> out of the system when running RHEL 5.4.
> 
> The Dell came with broadcom netxtreme cards.

Specific BCM model required here. I think some early NetXtreme cards still use the tg3 driver, later ones use the bnx driver, and later still models use the bnx2 and/or bnx2x driver.

> The first thing I did was 
> pull over the latest driver from broadcom.

Which may or may not actually be the latest... And per the above, are you sure you got the right one? :)

> Same transfer rates. I then 
> plugged in an intel nic ard which uses the e1000 driver. Same result.

Could potentially be an MSI problem. Message Signaled Interrupts are still rather problematic for many systems, there are constantly fixes for drivers, chipsets, etc. going upstream... Can't recall if they're enabled or not by default in RHEL5. You might try booting with pci=msi and pci=nomsi, and see if either one makes a difference.

> I then booted from the fedora 11 live CD, and the data rates sprang to 
> 94MBytes/sec which is what I would expect for gigabit ethernet.
> 
> I also used ethtool to look at the various options paying careful 
> attention to the offload parameters. Turning them on and off didn't 
> change the data rate problem. The other odd bit, is that an excessive 
> amount of CPU is being used, makes me think that it's not servicing 
> interrupts properly or something like that. Since I didn't change the 
> cables or switches when I got the data rates to go from 7/15 to 95 
> MBytes/sec, I'm ruling out broken cables/switches.... although 
> autoconfiging could still be a problem.

That seems like a possibility as well.

-- 
Jarod Wilson
jarod-ajLrJawYSntWk0Htik3J/w at public.gmane.org