Modify

Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#7454 closed enhancement (wontfix)

CPUs are to weak to exhaust USB data rates => enable jumbo frames for GBit Devices

Reported by: markus_kubler@… Owned by: developers
Priority: lowest Milestone: Backfire 10.03.1
Component: kernel Version: Backfire 10.03
Keywords: support jumbo frames Cc:

Description

With WR1043ND iperf measures a max data rate of 160MBit/s when accessing the device itself. This is very far from the theoretical possible 1000MBit/s. The switch works fine and also bridging performs very well! Just the routing+accessing does not.
Also reading/writing from/to USB-Device is max 80/40MBit/s.

Not sure whether this is so much of a CPU-Power issue or more of a IRQ-issue, however "Jumbo Frames" should help with this.

# ifconfig eth0 mtu 1501
ifconfig: SIOCSIFMTU: Invalid argument

# ifconfig eth0.1 mtu 1501
ifconfig: SIOCSIFMTU: Numerical result out of range

Is this a driver issue? Does the driver for eth0 not support frames>1500?

# ifconfig wlan0 mtu 2275
ifconfig: SIOCSIFMTU: Invalid argument

2274 gives no error message. However somebody needs to test whether this really results in frames that big. I have no other wireless to test this.

Attachments (0)

Change History (9)

comment:1 follow-up: Changed 8 years ago by dirtyfreebooter <openwrt@…>

In my experience this entirely CPU based. I have the Linksys WRT610Nv2 and D-LINK DIR825-B1. Both gigabit, usb2 devices. The 610 is 480MHz, the 825 is 680MHz. I not been able to top 6MB/s on the 610 and 9MB/s on the 825 from USB disk. These CPUs are weak, low powered embedded processors. I think you are expecting too much. I did too when I first switched from using a desktop linux box as a router to one these linux-based routers.

I've even overclocked my 610 to 533MHz only to get about 1.5MB/s more out of it. Even using the factory firmwares getting more than 10MB/s on any consumer router using the USB port is wishful.

You can improve your throughput by using a less CPU intensive protocol, Samba3 is a CPU hog, and even using ext2 over ext3 helps, but you lose the journaling safety factor.

comment:2 in reply to: ↑ 1 Changed 8 years ago by anonymous

Replying to dirtyfreebooter <openwrt@…>:

In my experience this entirely CPU based. [...] These CPUs are weak, low powered embedded processors. I think you are expecting too much.

Especially if the read/write rates from/to USB are CPU limited (more precisly IRQ/IO limited), jumbo frames do make a lot of sense! This is the very reason why J-Frames were introduced in the first place. Zero-copy would not help, since we do access the USB-Drive. Check with top how much is utilized by usr+sys and how much by irq+i/o. Question is not, if J-Frames would help, question is how much!

Even using the factory firmwares getting more than 10MB/s on any consumer router using the USB port is wishful.

Maybe some marketing people like to establish this 10MB/s frontier for consumer routers... I don't need a NAS which consumes 30W (and costs a load of money). I'd simply like to get more out of this little devices.

You can improve your throughput by using a less CPU intensive protocol, Samba3 is a CPU hog, and even using ext2 over ext3 helps, but you lose the journaling safety factor.

FTP with ext2 would be the most CPU-gentle solution. Yes. It is also a good idea to disable connection tracking for this ;-) But the issue here are Jumbo packets.

I do not know, if the gain is worth the effort, but it does make sense to use J-Frames in certain situations. I already wrote to the TP-Link support to find out, if the NIC supports J-Frames at all. Let us see, how long it will take them to answer. If it's a simple fix do it, if not, dump it.

comment:3 Changed 7 years ago by eymert@…

I am also encoutering the problem that jumbo frames are not supported.

While the rtl8366rb should support frames up to 9k, I also get the problem that no MTU setting higher than 1500 will be accepted (I have tried this for the br-lan interface as well as eth0, as well as eth0.10 (which is the interface for my LAN vlan).

Is there any chance this will be implemented? It is not only for accessing the TP-linkWR1043ND itself but also when it has to bridge from, eg, my pc to my Synology nas who both have 7k jumbo frames set. Or at least, not they have 1500, but with 7k frames set on the Synology, traffic to it is not coming through anymore.

comment:4 Changed 7 years ago by Wicher Minnaard <wicher@…>

This may be related to #7537 .
Also, from http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge :
"All devices share the same maximum packet size (MTU). The bridge doesn't fragment packets."

Set the MTU before assembling the bridge, then?

comment:5 Changed 7 years ago by anonymous

Actually, my RouterStation running Backfire 10.03 can read/write my USB HDD at 40Mb/sec which is very close to theoretical USB 2.0 limits. CPU load is ~40% only. Technically, RS haves AR7161 @ 680 MHz (full specs are on ubnt.com and their wiki). That's just HDD reading, though. Keep in mind that this is partially depends on IC used as USB <-> SATA HDD box and other stuff. Also, stuff like SMB or NFS can put certain overhead. Maybe used protocols are too bloated for CPUs?

You can: 1) Try to read raw device and measure speed of just reading (i.e. smth like timed run of dd if = /dev/sdX-your-usb-drive-here of=/dev/null). 2) Then, try to copy file from filesystem on /dev/sdX to /dev/null and measure time. Then compare speed. If difference is large, you're chosen bad filesystem for your file. 3) Then try to serve file with something lightweight like lighttpd or nginx and compare with previous results. Lighttpd is light enough to keep HTTP handling overhead very low. Compare speed again and make conclusion if network usage lowers throughput. It's nice idea to see top and figure out what uses most of CPU (iowait, irqs, some task or what?). If you're lucky and found bottleneck, you can improve your speeds, one way or another.

comment:6 Changed 7 years ago by anonymous

USB 2.0 is 480Mbits per second. Real world performance is more around 200 Mbps or (25 MB/s). My DIR825 680 MHz can do nowhere near 25 MB/s, 10MB/s max, trying Samba, NFS, FTP, and AFP (Netatalk, Apple).

The transfers are CPU bound, thus the point of this ticket, is to try and reduce CPU usage in the network stack by using jumbo frames... but networking, filesystem, and protocol (SMB,FTP,NFS,AFP,etc).. they all contribute to the load. Reducing either will theoretically help.

comment:7 Changed 7 years ago by KanjiMonster

As a FYI, the ar71xx SoCs (that includes the ar724x and ar913x) do not support jumbo frames. This is a hardware limitation.

comment:8 Changed 7 years ago by nbd

  • Resolution set to wontfix
  • Status changed from new to closed

comment:9 Changed 7 years ago by sKAApGIF

For those simply looking for Jumbo Frames functionality in the switch see #7977 (This won't help if you use your TL-WR1043ND as a NAS but would help if you simply use it as a switch between your computer an a separate NAS).

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.