Modify

Opened 6 years ago

Closed 6 years ago

Last modified 4 years ago

#10096 closed defect (fixed)

NoDogSplash in combination with qos-scripts blocks all traffic

Reported by: Tobias Wolf <towolf@…> Owned by: developers
Priority: normal Milestone: Barrier Breaker 14.07
Component: packages Version: Trunk
Keywords: Cc: towolf@…

Description

I’ve been using qos-scripts in combination with a NoDogsplash captive portal for a long time without changing the configuration.

The splash page is only intended to make the guests click a button to acknowledge a message. The traffic control in NDS is disabled.

Now I’ve compiled OpenWRT trunk regularly and since a few weeks ago the router blocks all traffic if both packages are loaded.

/etc/init.d/qos alone works properly and /etc/init.d/nodogsplash alone works too. As soon as both are started (no matter in which order) everything is blocked from LAN to WAN.

I’ve been looking around for weeks inspecting things like iptables -tmangle -L trying to disentangle where the conflict is, but all this is over my head.

At first, when I logged NDS with full verbosity, I found that it futzes around with tc even though I explicitly disabled TrafficControl

[7][Tue Sep 13 17:42:52 2011][4493](fw_iptables.c:591) Destroying our tc hooks
[7][Tue Sep 13 17:42:52 2011][4493](tc.c:69) Executing command: tc qdisc del dev imq0 root
[7][Tue Sep 13 17:42:52 2011][4493](tc.c:69) Executing command: tc qdisc del dev imq1 root
[7][Tue Sep 13 17:42:52 2011][4493](tc.c:216) Executing command: ip link set imq0 down
[7][Tue Sep 13 17:42:52 2011][4493](tc.c:219) Executing command: ip link set imq0 down
[7][Tue Sep 13 17:43:11 2011][4493](tc.c:69) Executing command: tc qdisc del dev imq0 root
[7][Tue Sep 13 17:43:11 2011][4493](tc.c:69) Executing command: tc qdisc del dev imq1 root
[7][Tue Sep 13 17:43:11 2011][4493](tc.c:216) Executing command: ip link set imq0 down
[7][Tue Sep 13 17:43:11 2011][4493](tc.c:219) Executing command: ip link set imq0 down

I went looking in the code and it does this upon startup, even though traffic_control is false.

In fw_iptables.c:380 it looks at the setting

  /* Set up for traffic control */
  if(traffic_control) {
    rc |= tc_init_tc();
  }

But in fw_iptables.c:593 it touches tc anyway and always:

/** Remove the firewall rules 
 * This is used when we do a clean shutdown of nodogsplash, 
 * and when it starts, to make sure there are no rules left over from a crash 
 */ 
int 
iptables_fw_destroy(void) { 
  fw_quiet = 1; 
 
  debug(LOG_DEBUG, "Destroying our tc hooks"); 
 
  tc_destroy_tc();

Since imq is deprecated I thought that was it and commented that shit out. But that didn’t make it work. There must be some other conflict.

Perhaps someone could pore over these. Below are the things that are run during the starutp of both

qos-scripts

cat Desktop/qosboot.log 
+ iptables -t mangle -F
+ iptables -t mangle -X
+ insmod ipt_multiport
+ insmod ipt_CONNMARK
+ insmod ipt_length
+ iptables -t mangle -N Default
+ iptables -t mangle -N Default_ct
+ iptables -t mangle -A Default_ct -m mark --mark 0 -m tcp -p tcp -m multiport --ports 22,53 -j MARK --set-mark 1
+ iptables -t mangle -A Default_ct -m mark --mark 0 -p udp -m udp -m multiport --ports 22,53 -j MARK --set-mark 1
+ iptables -t mangle -A Default_ct -m mark --mark 0 -p tcp -m tcp -m multiport --ports 20,21,25,80,110,443,993,995 -j MARK --set-mark 3
+ iptables -t mangle -A Default_ct -m mark --mark 0 -m tcp -p tcp -m multiport --ports 5190 -j MARK --set-mark 2
+ iptables -t mangle -A Default_ct -m mark --mark 0 -p udp -m udp -m multiport --ports 5190 -j MARK --set-mark 2
+ iptables -t mangle -A Default_ct -j CONNMARK --save-mark
+ iptables -t mangle -A Default -j CONNMARK --restore-mark
+ iptables -t mangle -A Default -m mark --mark 0 -j Default_ct
+ iptables -t mangle -A Default -m mark --mark 1 -m length --length 400: -j MARK --set-mark 0
+ iptables -t mangle -A Default -m mark --mark 2 -m length --length 800: -j MARK --set-mark 0
+ iptables -t mangle -A Default -m mark --mark 0 -p udp -m length --length :500 -j MARK --set-mark 2
+ iptables -t mangle -A Default -p icmp -j MARK --set-mark 1
+ iptables -t mangle -A Default -m mark --mark 0 -m tcp -p tcp --sport 1024:65535 --dport 1024:65535 -j MARK --set-mark 4
+ iptables -t mangle -A Default -m mark --mark 0 -p udp -m udp --sport 1024:65535 --dport 1024:65535 -j MARK --set-mark 4
+ iptables -t mangle -A Default -p tcp -m length --length :128 -m mark ! --mark 4 -m tcp --tcp-flags ALL SYN -j MARK --set-mark 1
+ iptables -t mangle -A Default -p tcp -m length --length :128 -m mark ! --mark 4 -m tcp --tcp-flags ALL ACK -j MARK --set-mark 1
+ iptables -t mangle -A OUTPUT -o pppoe-wan -j Default
+ iptables -t mangle -A FORWARD -o pppoe-wan -j Default

nodogsplash

$ grep Executing Desktop/nds.log | grep -o iptables.*
iptables -t mangle -F ndsTRU
iptables -t mangle -F ndsBLK
iptables -t mangle -F ndsALW
iptables -t mangle -F ndsOUT
iptables -t mangle -F ndsINC
iptables -t mangle -X ndsTRU
iptables -t mangle -X ndsBLK
iptables -t mangle -X ndsALW
iptables -t mangle -X ndsOUT
iptables -t mangle -X ndsINC
iptables -t nat -F ndsOUT
iptables -t nat -X ndsOUT
iptables -t filter -F ndsRTR
iptables -t filter -F ndsNET
iptables -t filter -F ndsAUT
iptables -t filter -F ndsTRU
iptables -t filter -F ndsTRT
iptables -t filter -X ndsRTR
iptables -t filter -X ndsNET
iptables -t filter -X ndsAUT
iptables -t filter -X ndsTRU
iptables -t filter -X ndsTRT
iptables -t mangle -I PREROUTING 1 -j MARK --or-mark 0x100
iptables -t mangle -D PREROUTING 1
iptables -t filter -I FORWARD 1 -m mark --mark 0x100/0x700 -j REJECT
iptables -t filter -D FORWARD 1
iptables -t mangle -N ndsTRU
iptables -t mangle -N ndsBLK
iptables -t mangle -N ndsINC
iptables -t mangle -N ndsOUT
iptables -t mangle -I PREROUTING 1 -i br-lan -s 0.0.0.0/0 -j ndsOUT
iptables -t mangle -I PREROUTING 2 -i br-lan -s 0.0.0.0/0 -j ndsBLK
iptables -t mangle -I PREROUTING 3 -i br-lan -s 0.0.0.0/0 -j ndsTRU
iptables -t mangle -I POSTROUTING 1 -o br-lan -d 0.0.0.0/0 -j ndsINC
iptables -t nat -N ndsOUT
iptables -t nat -I PREROUTING -i br-lan -s 0.0.0.0/0 -j ndsOUT
iptables -t nat -A ndsOUT -m mark --mark 0x200/0x700 -j ACCEPT
iptables -t nat -A ndsOUT -m mark --mark 0x400/0x700 -j ACCEPT
iptables -t nat -A ndsOUT -d 0.0.0.0/0 -p tcp --dport 53 -j ACCEPT
iptables -t nat -A ndsOUT -d 0.0.0.0/0 -p udp --dport 53 -j ACCEPT
iptables -t nat -A ndsOUT -d 10.0.0.1 -p tcp --dport 80 -j ACCEPT
iptables -t nat -A ndsOUT -d 10.0.0.10 -p tcp --dport 80 -j ACCEPT
iptables -t nat -A ndsOUT -p tcp --dport 80 -j DNAT --to-destination 10.0.0.1:2050
iptables -t nat -A ndsOUT -j ACCEPT
iptables -t filter -N ndsNET
iptables -t filter -N ndsRTR
iptables -t filter -N ndsAUT
iptables -t filter -N ndsTRU
iptables -t filter -N ndsTRT
iptables -t filter -I INPUT -i br-lan -s 0.0.0.0/0 -j ndsRTR
iptables -t filter -A ndsRTR -m mark --mark 0x100/0x700 -j DROP
iptables -t filter -A ndsRTR -m state --state INVALID -j DROP
iptables -t filter -A ndsRTR -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -t filter -A ndsRTR -p tcp --tcp-flags SYN SYN --tcp-option \! 2 -j  DROP
iptables -t filter -A ndsRTR -p tcp --dport 2050 -j ACCEPT
iptables -t filter -A ndsRTR -m mark --mark 0x200/0x700 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p udp --dport 53 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p tcp --dport 53 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p udp --dport 67 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p tcp --dport 49326 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p tcp --dport 80 -j ACCEPT
iptables -t filter -A ndsRTR -d 0.0.0.0/0 -p tcp --dport 443 -j ACCEPT
iptables -t filter -A ndsRTR -j REJECT --reject-with icmp-port-unreachable
iptables -t filter -I FORWARD -i br-lan -s 0.0.0.0/0 -j ndsNET
iptables -t filter -A ndsNET -m mark --mark 0x100/0x700 -j DROP
iptables -t filter -A ndsNET -m state --state INVALID -j DROP
iptables -t filter -A ndsNET -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
iptables -t filter -A ndsNET -m mark --mark 0x200/0x700 -j ACCEPT
iptables -t filter -A ndsNET -m mark --mark 0x400/0x700 -j ndsAUT
iptables -t filter -A ndsAUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -t filter -A ndsAUT -d 10.0.0.0/25 -j REJECT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 53 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p udp --dport 53 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 80 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 443 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 22 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 995 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 993 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 465 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 5222 -j ACCEPT
iptables -t filter -A ndsAUT -d 0.0.0.0/0 -p tcp --dport 5298 -j ACCEPT
iptables -t filter -A ndsAUT -j REJECT --reject-with icmp-port-unreachable
iptables -t filter -A ndsNET -d 0.0.0.0/0 -p tcp --dport 53 -j ACCEPT
iptables -t filter -A ndsNET -d 0.0.0.0/0 -p udp --dport 53 -j ACCEPT
iptables -t filter -A ndsNET -d 10.0.0.1 -p tcp --dport 80 -j ACCEPT
iptables -t filter -A ndsNET -d 10.0.0.10 -p tcp --dport 80 -j ACCEPT
iptables -t filter -A ndsNET -j REJECT --reject-with icmp-port-unreachable
iptables -t mangle -D PREROUTING 3
iptables -t mangle -D PREROUTING 2
iptables -t mangle -D PREROUTING 1
iptables -t mangle -D POSTROUTING 1
iptables -t mangle -F ndsTRU
iptables -t mangle -F ndsBLK
iptables -t mangle -F ndsALW
iptables -t mangle -F ndsOUT
iptables -t mangle -F ndsINC
iptables -t mangle -X ndsTRU
iptables -t mangle -X ndsBLK
iptables -t mangle -X ndsALW
iptables -t mangle -X ndsOUT
iptables -t mangle -X ndsINC
iptables -t nat -D PREROUTING 1
iptables -t nat -F ndsOUT
iptables -t nat -X ndsOUT
iptables -t filter -D INPUT 1
iptables -t filter -D FORWARD 1
iptables -t filter -F ndsRTR
iptables -t filter -F ndsNET
iptables -t filter -F ndsAUT
iptables -t filter -F ndsTRU
iptables -t filter -F ndsTRT
iptables -t filter -X ndsRTR
iptables -t filter -X ndsNET
iptables -t filter -X ndsAUT
iptables -t filter -X ndsTRU
iptables -t filter -X ndsTRT

Attachments (3)

mask-qos-marks.patch (2.9 KB) - added by Tobias Wolf <towolf@…> 6 years ago.
restrict marking to lower 8 bits.
sh -x color-diff.png (728.3 KB) - added by Tobias Wolf <towolf@…> 6 years ago.
overview over changed rules
mask-qos-marks.2.patch (3.0 KB) - added by towolf <towolf@…> 6 years ago.
updated patch

Download all attachments as: .zip

Change History (11)

comment:1 Changed 6 years ago by jow

See if you have "ip_queue" in "lmod". If yes, unload it.

comment:2 Changed 6 years ago by jow

The above should've read in "lsmod".

comment:3 Changed 6 years ago by Tobias Wolf <towolf@…>

No, I don’t.

I’m now comparing what you are doing in your luci-splash vs what NDS is doing. This is all over my head, I’m afraid. I can’t parse all this.

root@winder:~# lsmod | grep ip_queue
root@winder:~# lsmod | cut -f1 -d' '
Module
sch_red
sch_sfq
sch_hfsc
cls_fw
sch_ingress
act_mirred
act_connmark
em_u32
cls_flow
cls_u32
ifb
ledtrig_usbdev
nf_nat_irc
nf_conntrack_irc
nf_nat_ftp
nf_conntrack_ftp
xt_HL
xt_hl
ipt_ECN
xt_CLASSIFY
xt_time
xt_tcpmss
xt_statistic
xt_mark
xt_length
ipt_ecn
xt_DSCP
xt_dscp
xt_string
xt_layer7
ipt_MASQUERADE
iptable_nat
nf_nat
xt_recent
xt_helper
xt_connmark
xt_connbytes
xt_conntrack
xt_NOTRACK
iptable_raw
xt_state
nf_conntrack_ipv4
nf_defrag_ipv4
nf_conntrack
ipt_REJECT
xt_TCPMSS
ipt_LOG
xt_comment
xt_multiport
xt_mac
xt_limit
iptable_mangle
iptable_filter
ip_tables
xt_tcpudp
x_tables
nfsd
nfs
ppp_async
loop
vfat
fat
lockd
sunrpc
isofs
ath9k
ath9k_common
ath9k_hw
ath
mac80211
ts_fsm
ts_bm
ts_kmp
exportfs
crc_ccitt
cfg80211
compat
arc4
aes_generic
crypto_algapi
pppoe
pppox
ppp_generic
slhc
usb_storage
ohci_hcd
ehci_hcd
sd_mod
ext4
jbd2
mbcache
usbcore
scsi_mod
nls_base
crc16
leds_gpio
button_hotplug
gpio_keys_polled
input_polldev
input_core

comment:4 Changed 6 years ago by Tobias Wolf <towolf@…>

I think I figured it out. I wonder if there are any side effects. Please have a good look at my naive proposal.

I think qos-scripts was messing with the fwmark packet marks set by NoDogSplash.
NDS inserts itself at the beginning and only uses the second byte of the mark. Then generate.sh goes in and overwrites those. I’m not sure which part did this exactly, it’s all bohemian villages to me.

I’ve prepared a patch that makes generate.sh only touch the lower 8 bits and leave the other ones alone. NDS occupies mask 0x700.

Changed 6 years ago by Tobias Wolf <towolf@…>

restrict marking to lower 8 bits.

Changed 6 years ago by Tobias Wolf <towolf@…>

overview over changed rules

comment:5 Changed 6 years ago by Tobias Wolf <towolf@…>

Relevant reading reveals that 2.4 might not support this. So you might not want this patch.

0.9_beta9.9.1

*** Enable optional functionality for authenticated packets to 
RETURN from ndsNET chain instead of jumping to
ndsAUT chain, thus hitting pre-existing (non-nodogsplash) firewall
rules in the FORWARD chain of the filter table.  This is configured by
specifying an empty FirewallRuleset authenticated-users.

*** Nodogsplash iptables rules now attempts to use mark masking if the
kernel supports it.  (2.6 kernels should support it; stock 2.4 kernels do
not.)  This should permit nodogsplash to coexist with other packages
that mark packets, if they also use mark masking (i.e. --or-mark instead
of --set-mark when marking, and using an appropriate mask when reading
marks).

http://kokoro.ucsd.edu/nodogsplash/CHANGELOG

Changed 6 years ago by towolf <towolf@…>

updated patch

comment:6 Changed 6 years ago by towolf <towolf@…>

Since you namespaced the qos chains, it would be good to also restrict setting the marks with a bitmask. I updated the patch.

comment:7 Changed 6 years ago by jow

  • Resolution set to fixed
  • Status changed from new to closed

Committed in r28731 - thanks!

comment:8 Changed 4 years ago by jow

  • Milestone changed from Attitude Adjustment 12.09 to Barrier Breaker 14.07

Milestone Attitude Adjustment 12.09 deleted

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.