Modify

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#20163 closed defect (worksforme)

CC 15.05-rc3: packets sent out on a second (wired) WAN are being dropped somewhere in OpenWRT

Reported by: braveheart_leo@… Owned by: developers
Priority: high Milestone: Chaos Calmer 15.05
Component: other Version: Trunk
Keywords: Cc:

Description

I reconfigured my WZR-HP-AG300H, running CHAOS CALMER (15.05-rc3, r46163), for a dual (wired) WAN setup, opting to tranform LAN port 4 (switch port 1) of the router as a second WAN.

I have two ADSL2+ lines. This router's primary WAN port (eth1) is connected to a modem that is configured as a bridge. The secondary WAN port (LAN port 4) is connected to the LAN of another router running DD-WRT, which is in turn connected to another modem configured as bridge. The secondary WAN port is configured in a DMZ on the DD-WRT router.

This is my network configuration:

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'
	option force_link '1'

config interface 'lan'
	option proto 'static'
	option ipaddr '192.168.101.1'
	option netmask '255.255.255.0'
	option ifname 'eth0.1'

config interface 'wan'
	option ifname 'eth1'
	option proto 'dhcp'
	option defaultroute '1'
	option metric '10'
	option hostname 'WANHOST1'

config interface 'modem'
	option ifname '@wan'
	option proto 'static'
	option ipaddr '192.168.0.2'
	option netmask '255.255.255.0'

config interface 'wan2'
	option auto '1'
	option ifname 'eth0.2'
	option proto 'static'
	option ipaddr '192.168.1.2'
	option netmask '255.255.255.0'
	option gateway '192.168.1.1'
	option metric '20'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0t 2 3 4'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0t 1'

And here is the relevant firewall configuration options:

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'wan'
        option input 'DROP'
        option output 'ACCEPT'
        option forward 'DROP'
        option masq '1'
        option mtu_fix '0'
        list network 'wan'
        list network 'wan2'

config forwarding
        option src 'lan'
        option dest 'wan'

This is the AG300H's routing table given the above configuration:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         49.x.x.x        0.0.0.0         UG    10     0        0 eth1
0.0.0.0         192.168.1.1     0.0.0.0         UG    20     0        0 eth0.2
49.x.x.0        0.0.0.0         255.255.224.0   U     10     0        0 eth1
49.x.x.1        0.0.0.0         255.255.255.255 UH    10     0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     20     0        0 eth0.2
192.168.101.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0.1

As you can see, I have two default gateways corresponding to the two WAN interfaces I have setup, each having different metrics. However, testing shows that while the primary WAN connection works, the secondary WAN (wan2) does not:

AG300H:~# ping -c 2 -I eth1 google.com
PING google.com (122.2.129.167): 56 data bytes
64 bytes from 122.2.129.167: seq=0 ttl=60 time=33.508 ms
64 bytes from 122.2.129.167: seq=1 ttl=60 time=32.863 ms

--- google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 32.863/33.185/33.508 ms

AG300H:~# ping -c 2 -I eth0.2 google.com
PING google.com (122.2.129.167): 56 data bytes

--- google.com ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

Of course I verified that the other router running DD-WRT is properly connected to the net, so telnetting there and doing the ping test:

DD-WRT:~# ping -c 1 google.com
PING google.com (58.71.25.89): 56 data bytes
64 bytes from 58.71.25.89: seq=0 ttl=59 time=35.203 ms

--- google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 35.203/35.203/35.203 ms

I can also ping the AG300H from the DD-WRT router as well:

DD-WRT:~# ping -c 2 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56 data bytes
64 bytes from 192.168.1.2: seq=0 ttl=64 time=1.739 ms
64 bytes from 192.168.1.2: seq=1 ttl=64 time=1.600 ms

--- 192.168.1.2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.600/1.669/1.739 ms

tcpdump shows that packets are arriving at eth0.2, but are being dropped somewhere on the AG300H:

AG300H:~# tcpdump -ni eth0.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.2, link-type EN10MB (Ethernet), capture size 65535 bytes
08:48:16.083965 IP 113.196.212.123.14731 > 192.168.1.2.13754: UDP, length 20
08:48:18.086424 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46
08:48:18.086481 ARP, Reply 192.168.1.2 is-at 00:24:a5:ef:8e:ae, length 28
08:48:19.865809 IP 114.26.145.92.9789 > 192.168.1.2.13754: UDP, length 20
08:48:22.542439 IP 183.6.172.36.12894 > 192.168.1.2.13754: UDP, length 20
08:48:22.909185 IP 119.98.147.104.12150 > 192.168.1.2.13754: UDP, length 20
08:48:31.424329 IP 58.208.66.234.62278 > 192.168.1.2.13754: UDP, length 20
08:48:42.132547 IP 221.6.97.74.12980 > 192.168.1.2.13754: UDP, length 20

I even captured the replies to the pings I sent out on eth0.2 but never reached the application, thus resulting in a ping timeout:

AG300H:~# ping -c 2 -I eth0.2 google.com
PING google.com (122.2.129.168): 56 data bytes

--- google.com ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

AG300H:~# tcpdump -ni eth0.2
08:51:47.783759 IP 192.168.1.2 > 122.2.129.168: ICMP echo request, id 2970, seq 0, length 64
08:51:47.819554 IP 122.2.129.168 > 192.168.1.2: ICMP echo reply, id 2970, seq 0, length 64
08:51:48.784100 IP 192.168.1.2 > 122.2.129.168: ICMP echo request, id 2970, seq 1, length 64
08:51:48.818954 IP 122.2.129.168 > 192.168.1.2: ICMP echo reply, id 2970, seq 1, length 64

As you can see, the packets arrived at the interface, but never traversed the upper network stack, indicating that it is being dropped somewhere in OpenWRT.

Attachments (0)

Change History (6)

comment:1 Changed 3 years ago by braveheart_leo@…

I can ping the DD-WRT router from Openwrt via wan2, but any other than that and it results in a request timed out:

AG300H:~# ping -c 2 -I eth0.2 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: seq=0 ttl=64 time=1.498 ms
64 bytes from 192.168.1.1: seq=1 ttl=64 time=1.042 ms

--- 192.168.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.042/1.270/1.498 ms

AG300H:~# ping -c 2 -I eth1 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes

--- 192.168.1.1 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

The second ping test fails because the address is not along its upstream path, which is correct.

There is a bug in there somewhere, but I can't put my hands on it, nor point out where the issue may be.

comment:2 Changed 3 years ago by braveheart_leo@…

I just want to add some further observations and tests I have conducted.

As previously mentioned, packets are arriving at the wan2 (eth0.2) interface, seen from a tcpdump, but they are not hitting iptables, indicating that the packets are being dropped early on:

Chain delegate_input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   18  1754 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
  189 14240 input_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* user chain for input */
   10  2993 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
   28  1300 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
   93  5012 syn_flood  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x17/0x02
   19  1139 zone_lan_input  all  --  eth0.1 *       0.0.0.0/0            0.0.0.0/0           
    0     0 zone_wan_input  all  --  eth0.2 *       0.0.0.0/0            0.0.0.0/0           
  132  8808 zone_wan_input  all  --  eth1   *       0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Also, if I issue ifdown wan, then wan2 starts working, forwarding packets to and fro.

comment:3 Changed 3 years ago by braveheart_leo@…

I found the culprit.

I changed the openwrt default sysctl.conf and activated the rp_filter sysctl that interfered with routing packets. It was fine to have it on (for added security) when I had a single WAN setup, but it got complicated when I tried to have two WANs.

Setting net.ipv4.conf.{default,all}.rp_filter=0, which is the default in openwrt, makes both my WANs function normally.

The same behavior is observed in this post: http://nrocco.github.io/2013/04/13/disable-rp-filter.html

comment:4 Changed 3 years ago by jow

  • Resolution set to worksforme
  • Status changed from new to closed

comment:5 follow-up: Changed 3 years ago by anonymous

instead of setting rp_filter to 0 try setting it to 2

quotes from
https://access.redhat.com/solutions/53031

0 - No source validation.

1 - Strict mode as defined in RFC3704 Strict Reverse Path

Each incoming packet is tested against the FIB and if the interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.

2 - Loose mode as defined in RFC3704 Loose Reverse Path

Each incoming packet's source address is also tested against the FIB
and if the source address is not reachable via any interface
the packet check will fail.

Current recommended practice in RFC3704 is to enable strict mode
to prevent IP spoofing from DDos attacks. If using asymmetric routing
or other complicated routing, then loose mode is recommended.

/quotes

comment:6 in reply to: ↑ 5 Changed 3 years ago by braveheart_leo@…

Replying to anonymous:

instead of setting rp_filter to 0 try setting it to 2

quotes from
https://access.redhat.com/solutions/53031

0 - No source validation.

1 - Strict mode as defined in RFC3704 Strict Reverse Path

Each incoming packet is tested against the FIB and if the interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.

2 - Loose mode as defined in RFC3704 Loose Reverse Path

Each incoming packet's source address is also tested against the FIB
and if the source address is not reachable via any interface
the packet check will fail.

Current recommended practice in RFC3704 is to enable strict mode
to prevent IP spoofing from DDos attacks. If using asymmetric routing
or other complicated routing, then loose mode is recommended.

/quotes

yes, i have opted to set rp_filter=2 after testing it with 0 (the default) and verified that it was indeed the culprit.

i wonder now if the devs would consider activating this by default as well, for added security and as per RF3704 recommendation. would there be exceptional scenarios where this is desired to be turned off?

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.