[cod3fr3ak]

After verifying that may raid card is supported in OBSD I decided to move off of win2k3, and use OBSD + Samba for all my network storage sharing stuff. I moved all my data, about 360Gb of movies, music and documents over to a spare server, and installed OBSD 3.6 on the prime server. When I initally moved the data from win2k3 to my temp OBSD box, I was really frustrated with the file transfer speed. I have a Intel Pro 1000GB (SX) Server card in each system. Both boxes are connected back to back via fiber cable. The network is tiny using just 2 ips on the whole subnet (255.255.255.252). With the default MTU setting on both machines max thruput topped out at about 3mb (3000kb). which is correct for ether net running a default MTU of 1500 and having a full-dux connection speed. My buddy told me to set up jumbo frames and give that a spin to decrease file transfer times. So i did the ifconfig <addr> mtu 9000 on the OBSD box. On the win2k3 box things got dicey, as the default windows drivers do not support jumbo frames. So i had to download another set of drivers directly from Intel. After doing this and setting jumbo frames to intel's preset or 9016, I tried slinging a few fat packets to the OBSD box. It worked no fragments just 8000 byte packets coming into the OBSD box. So then I fire up sftp to start transfering files and I acually get LESS bandwidth than before. So I chalk it up to stupid windows and set everything back to the default MTU of 1500, slog thru 3 days of sftp transfers, wipe the win2k3 box in install OBSD. This time I use ifconfig on both servers to set the MTU to 9032 (added 32 for header overhead). Slung a few packets back and forth to make sure no fragmentation occured and started my scp.
Max download is now 3.5mb. What am I doing wrong? These are Gigabit fiber network adapters. I can see no reason why they should not be able to get at least 250mb using this config. Any suggestions or ideas would be greatly appreciated.
Oh and heres what the ifconfig says:

SERVER-
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9032
address: 00:d0:b7:82:27:27
media: Ethernet 1000baseSX full-duplex
status: active
inet 10.16.0.1 netmask 0xfffffffc broadcast 10.16.0.3
inet6 fe80::2d0:b7ff:fe82:2727%em0 prefixlen 64 scopeid 0x2

CLIENT-
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9032
address: 00:03:47:25:35:50
media: Ethernet 1000baseSX full-duplex
status: active
inet 10.16.0.2 netmask 0xfffffffc broadcast 10.16.0.3
inet6 fe80::203:47ff:fe25:3550%em0 prefixlen 64 scopeid 0x1


And here is what the netstat -rn says:

SERVER-
10.16.0.0/30 link#2 UC 2 0 - em0
10.16.0.2 0:3:47:25:35:50 UHLc 2 1031554 - em0
10.16.0.3 link#2 UHLc 2 67 - em0

CLIENT-
10.16.0.0/30 link#1 UC 2 0 - em0
10.16.0.1 0:d0:b7:82:27:27 UHLc 2 1112362 - em0
10.16.0.2 0:3:47:25:35:50 UHLc 0 8 - lo0


Thanks in advance.

[elmore]

Huh this guy shows a similar problem:

http://readlist.com/lists/openbsd.org/misc/0/616.html

No replies...

This post says Jumobo frames on em(4) are now reliable:

http://obsd.17slon.org/pipermail/obsd/2004-October/000909.html

I'll keep looking around but that's all I can see right now.

[cod3fr3ak]

Thanks E.

[elmore]

Here's Ryan McBride's answer:

http://www.monkey.org/openbsd/archive/misc/0407/msg01546.html

[cod3fr3ak]

Yeah I saw this when I was snooping around the Net too. But I am using 3.6 so I wonder if the issue still exists. I am too tired to load both machines up with SuSE 9.0 so I guess it'll have to wait from me to test later. I'll keep you all informed.

[schotty]

So does that mean that non broadcom chipsets can do large frames reliably and speedily?

[elmore]

Change you jumbo size to 9216 that's my switch is set to by default. I played them today and did not have an issue.

[cod3fr3ak]

I'll try this tonight. Thanks.

[frisco]

You're transferring files using sftp, which isn't the fastest way to transfer files and definitely not the best way to judge network performance since it adds your computer's encryption performance into the mix, plus I seem to recall there being other issues with sftp that slows down transfers as well - check the mailing list.

If you just want to see how fast your network is, use a tool other than sftp.

[cod3fr3ak]

Check. I was just using sftp to do a basic test. Not really a benchmark. I am making the changes to the box right now. And I'll get a benchmarking app to see the real speed soon.

[cod3fr3ak]

I set the packet size to 9216 and had some serious issues. I think the problem is a WindowsXP + OpenBSD problem. I plan on doing further tests but I need to get the system up and running for a party I have next weekend.