Modify

Opened 10 years ago

Closed 10 years ago

#2921 closed defect (fixed)

dst cache overflow causes router to lose all network connectivity

Reported by: tardfree Owned by: developers
Priority: normal Milestone: Kamikaze 7.09
Component: kernel Version:
Keywords: Cc:

Description

After a short duration of heavy traffic (both p2p and web surfing) I've observed a recurring dst cache overflow which causes the router to lose network connectivity. Even the lan ports on the switch lose the ability to communicate to other ports on the same vlan.

This does not happen to me when using the 7.09 release tagged in svn, but does happen with both r9928 and r10005.

To monitor the occurance I've been looking at ip_dst_cache in /proc/slabinfo and when it hits 8192 I immediately get the following logged to syslog (logged to another ip so i can see it before it loses network)

Dec 29 03:56:36 openwrt.lan kernel: dst cache overflow
Dec 29 03:56:36 openwrt.lan kernel: MASQUERADE: No route: Rusty's brain broke!
Dec 29 03:56:36 openwrt.lan kernel: dst cache overflow
Dec 29 03:56:36 openwrt.lan kernel: MASQUERADE: No route: Rusty's brain broke!
Dec 29 03:56:37 openwrt.lan kernel: dst cache overflow
Dec 29 03:56:37 openwrt.lan kernel: MASQUERADE: No route: Rusty's brain broke!

The MASQUERADE error doesn't always occur, but the dst cache overflow does.

I find when using bittorrent the dst cache increases faster, but not using it does not prevent it from going up. From power on to losing connectivity (with BT running) can be under 2 hours.

Issuing the following to flush the route cache doesn't always have an impact, sometimes the cache drops in size, but quickly regains it's previous level.

echo 1 > /proc/sys/net/ipv4/route/flush

This is one of the commands I have been using to dump info while I monitor, this is one dumped just before network is lost.

root@OpenWrt:/# uptime ; grep ip_dst_cache /proc/slabinfo ; echo rtcache: ; cat /proc/net/rt_cache | wc -l ; echo contrack, contrack_max ; cat /proc/net/ip_conntrack | wc -l ; cat /proc/sys/net/ipv4/ip_conntrack_max ; free
 02:29:40 up  1:44, load average: 0.07, 0.05, 0.00
ip_dst_cache        8192   8200    192  410  410    1
rtcache:
852
contrack, contrack_max
388
5953
              total         used         free       shared      buffers
  Mem:        30512        16020        14492            0         1436
 Swap:            0            0            0
Total:        30512        16020        14492
root@OpenWrt:/#

I am using pppoe for wan connection and wireless is disabled. QOS is installed with default config. I have been testing with both a WRT54GS and WRT54GL and it occurs on both (the GL doesn't run out of free memory).

This forum topic http://forum.openwrt.org/viewtopic.php?id=10706 is for the same issue under a third party WhiteRussian image.

I'm willing and able to help diagnose this issue.

Attachments (0)

Change History (3)

comment:1 Changed 10 years ago by Weedy <weedy2887@…>

r10065

Dec 31 01:39:14 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:14 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:14 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:14 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:14 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:15 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:15 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:15 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:15 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:15 OpenWrt user.warn kernel: dst cache overflow
Dec 31 01:39:19 OpenWrt user.warn kernel: NET: 287 messages suppressed.
Dec 31 01:39:19 OpenWrt user.warn kernel: dst cache overflow

comment:2 Changed 10 years ago by Weedy <weedy2887@…>

r9801 is what causes it. im going to try a later build with mac filtering removed.

comment:3 Changed 10 years ago by nbd

  • Resolution set to fixed
  • Status changed from new to closed

fixed in r10132

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.