Modify

Opened 3 years ago

Last modified 3 years ago

#18455 reopened defect

random reboots in TP-LINK 3500

Reported by: tgayoso@… Owned by: developers
Priority: normal Milestone: Chaos Calmer 15.05
Component: kernel Version: Trunk
Keywords: unhandled paging requests Cc:

Description

I'm getting random reboots at least twice a day on my TP-LINK 3500 due to unhandled paging requests. Full trace in the attachment.

CPU 0 Unable to handle kernel paging request at virtual address 60000000, epc == 87040040, ra == 8706a52c
<4>[ 2473.984000] Oops[#1]:

Attachments (1)

tplink_3500_trace.txt (16.3 KB) - added by tgayoso@… 3 years ago.
unhandled paging trace

Download all attachments as: .zip

Change History (20)

Changed 3 years ago by tgayoso@…

unhandled paging trace

comment:1 Changed 3 years ago by bittorf@…

are you using ipv6? how is your setup?

comment:2 Changed 3 years ago by tgayoso@…

hi Bittorf,

It's a custom build to support SQM shapping with ECN on egress+ingress and custom CFLAGS to squeeze some throughput improvement.

I use GCC 4.9 with the following CFLAGS

CONFIG_TARGET_OPTIMIZATION="-O2 -march=74kc -mtune=74kc -mdspr2 -mno-branch-likely -pipe"

WAN and LAN hosts all configured and working with IPV4. NO IPV6 address appears on LUCI's web interface. When I query via ifconfig in command line I DO SEE IPV6 configured on each interface automagically.

Let me know what output/config may help address this and how may I contribute.

Kernel Modules:

root@netcat:~# lsmod 
act_ipt                 2592  0 
act_mirred              2432  1 
act_police              3296  0 
act_skbedit             1696  0 
arc4                    1296  4 
ath                    17717  3 ath9k
ath9k                  92849  0 
ath9k_common           11453  1 ath9k
ath9k_hw              318766  2 ath9k
cfg80211              203859  4 ath9k
cls_basic               3168  0 
cls_flow                4976  0 
cls_fw                  3344  7 
cls_route               5136  0 
cls_tcindex             4144  0 
cls_u32                 5792  4 
compat                  2139  5 ath9k
crc_ccitt               1019  1 ppp_async
crypto_blkcipher       10280  1 arc4
ehci_hcd               32171  1 ehci_platform
ehci_platform           2512  0 
em_cmp                   752  0 
em_meta                 4480  0 
em_nbyte                 736  0 
em_text                 1312  0 
em_u32                   576  0 
gpio_button_hotplug     4496  0 
ifb                     2704  0 
ip6_tables              9201  3 ip6table_raw
ip6t_REJECT             2016  2 
ip6table_filter          656  1 
ip6table_mangle         1072  1 
ip6table_raw             576  1 
ip_tables               9293  4 iptable_nat
ipt_ECN                 1376  0 
ipt_MASQUERADE          1104  1 
ipt_REJECT              1888  2 
iptable_filter           720  1 
iptable_mangle           928  1 
iptable_nat             1728  1 
iptable_raw              656  1 
ipv6                  253809 26 nf_conntrack_ipv6
ledtrig_usbdev          1936  0 
mac80211              369079  1 ath9k
nf_conntrack           46091 17 iptable_nat
nf_conntrack_ftp        5200  1 nf_nat_ftp
nf_conntrack_ipv4       4560  8 
nf_conntrack_ipv6       4864  3 
nf_conntrack_irc        2800  1 nf_nat_irc
nf_defrag_ipv4           806  1 nf_conntrack_ipv4
nf_defrag_ipv6          9143  1 nf_conntrack_ipv6
nf_nat                  8681  7 iptable_nat
nf_nat_ftp              1200  0 
nf_nat_ipv4             2494  1 iptable_nat
nf_nat_irc               976  0 
nls_base                4960  1 usbcore
ohci_hcd               21545  1 ohci_platform
ohci_platform           1712  0 
ppp_async               6208  0 
ppp_generic            20162  3 pppoe
pppoe                   7776  0 
pppox                   1322  1 pppoe
sch_codel               4192  0 
sch_dsmark              3568  0 
sch_gred                6208  0 
sch_hfsc               13504  0 
sch_htb                12912  2 
sch_ingress              944  1 
sch_prio                3088  0 
sch_red                 4416  0 
sch_sfq                 8160  0 
sch_tbf                 5152  0 
sch_teql                3744  0 
slhc                    4283  1 ppp_generic
ts_bm                   1392  0 
ts_fsm                  2512  0 
ts_kmp                  1312  0 
usb_common              1160  1 usbcore
usbcore               112367  5 ledtrig_usbdev
x_tables               10997 43 ipt_MASQUERADE
xt_CLASSIFY              576  0 
xt_CT                   2240  0 
xt_DSCP                 1504  4 
xt_HL                   1328  0 
xt_LOG                  9408  0 
xt_REDIRECT             1056  0 
xt_TCPMSS               2592  2 
xt_comment               480 59 
xt_connbytes            1472  0 
xt_connlimit            2560  0 
xt_connmark             1040  0 
xt_conntrack            2176 10 
xt_dscp                  992 12 
xt_ecn                  1280  0 
xt_helper                848  0 
xt_hl                    800  0 
xt_id                    480  0 
xt_length                688  0 
xt_limit                 992 20 
xt_mac                   608  0 
xt_mark                  672 16 
xt_multiport            1120  2 
xt_nat                  1072  8 
xt_recent               6576  0 
xt_state                 688  0 
xt_statistic             784  0 
xt_string                784  0 
xt_tcpmss                992  0 
xt_tcpudp               1648 14 
xt_time                 1664  0 
root@netcat:~# 

comment:3 Changed 3 years ago by bittorf@…

please show us

ip address show

and

for table in mangle filter nat raw; do ip6tables -t $table -nxvL; done

comment:4 Changed 3 years ago by nbd

please test if the same happens in a build where you didn't mess with the compiler/cflags config.

comment:5 Changed 3 years ago by tgayoso@…

@NBD, acknowledged and will try with defaul CFLAGS. I follow your commits closely and rebuild regularly when you submit code to the AR71XX target. Thanks.

@Bittortf, as requested:

root@netcat:~# 
root@netcat:~# 
root@netcat:~# date
Mon Dec  1 16:26:30 GMT 2014
root@netcat:~# ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP group default qlen 1000
    link/ether a0:f3:c1:46:25:7b brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000
    link/ether a0:f3:c1:46:25:7e brd ff:ff:ff:ff:ff:ff
    inet 24.232.217.36/24 brd 24.232.217.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a2f3:c1ff:fe46:257e/64 scope link 
       valid_lft forever preferred_lft forever
4: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32
    link/ether 72:41:30:6d:cb:0b brd ff:ff:ff:ff:ff:ff
5: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32
    link/ether a6:37:56:b6:27:eb brd ff:ff:ff:ff:ff:ff
6: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void 
9: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether a0:f3:c1:46:25:7b brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fd5a:4d01:9e97::1/60 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::a2f3:c1ff:fe46:257b/64 scope link 
       valid_lft forever preferred_lft forever
10: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether a0:f3:c1:46:25:7d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a2f3:c1ff:fe46:257d/64 scope link 
       valid_lft forever preferred_lft forever
12: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether a0:f3:c1:46:25:7c brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a2f3:c1ff:fe46:257c/64 scope link 
       valid_lft forever preferred_lft forever
13: ifb4eth1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN group default qlen 32
    link/ether fa:cd:48:58:61:bd brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f8cd:48ff:fe58:61bd/64 scope link 
       valid_lft forever preferred_lft forever
root@netcat:~# 
root@netcat:~# 
root@netcat:~# for table in mangle filter nat raw; do ip6tables -t $table -nxvL; done
Chain PREROUTING (policy ACCEPT 129 packets, 9778 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 MARK       tcp      vtun+  *       ::/0                 ::/0                 MARK xset 0x2/0xff
       0        0 DSCP       all      eth1   *       ::/0                 ::/0                 DSCP match !0x00 DSCP set 0x00
     129     9778 fwmark     all      *      *       ::/0                 ::/0                

Chain INPUT (policy ACCEPT 75 packets, 5890 bytes)
    pkts      bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 mssfix     all      *      *       ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 123 packets, 12678 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
      35     6174 DSCP       udp      *      *       ::/0                 ::/0                 multiport ports 123,53 DSCP set 0x24

Chain POSTROUTING (policy ACCEPT 177 packets, 16566 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 QOS_MARK_eth1  all      *      eth1    ::/0                 ::/0                [goto]  mark match 0x0/0xff

Chain QOS_MARK_eth1 (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 MARK       all      *      *       ::/0                 ::/0                 MARK xset 0x2/0xff
       0        0 MARK       all      *      *       ::/0                 ::/0                 DSCP match 0x08 MARK xset 0x3/0xff
       0        0 MARK       all      *      *       ::/0                 ::/0                 DSCP match 0x30 MARK xset 0x1/0xff
       0        0 MARK       all      *      *       ::/0                 ::/0                 DSCP match 0x2e MARK xset 0x1/0xff
       0        0 MARK       all      *      *       ::/0                 ::/0                 DSCP match 0x24 MARK xset 0x1/0xff
       0        0 MARK       all      *      *       ::/0                 ::/0                 tos match0x10/0x3f MARK xset 0x1/0xff

Chain fwmark (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain mssfix (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 TCPMSS     tcp      *      eth1    ::/0                 ::/0                 tcp flags:0x06/0x02 /* wan (mtu_fix) */ TCPMSS clamp to PMTU
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
      75     5890 delegate_input  all      *      *       ::/0                 ::/0                

Chain FORWARD (policy DROP 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 delegate_forward  all      *      *       ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
     123    12678 delegate_output  all      *      *       ::/0                 ::/0                

Chain delegate_forward (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 forwarding_rule  all      *      *       ::/0                 ::/0                 /* user chain for forwarding */
       0        0 ACCEPT     all      *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
       0        0 zone_lan_forward  all      br-lan *       ::/0                 ::/0                
       0        0 zone_wan_forward  all      eth1   *       ::/0                 ::/0                
       0        0 reject     all      *      *       ::/0                 ::/0                

Chain delegate_input (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 ACCEPT     all      lo     *       ::/0                 ::/0                
      75     5890 input_rule  all      *      *       ::/0                 ::/0                 /* user chain for input */
       0        0 ACCEPT     all      *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
       0        0 syn_flood  tcp      *      *       ::/0                 ::/0                 tcp flags:0x17/0x02
      75     5890 zone_lan_input  all      br-lan *       ::/0                 ::/0                
       0        0 zone_wan_input  all      eth1   *       ::/0                 ::/0                

Chain delegate_output (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 ACCEPT     all      *      lo      ::/0                 ::/0                
     123    12678 output_rule  all      *      *       ::/0                 ::/0                 /* user chain for output */
      35     6174 ACCEPT     all      *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
      88     6504 zone_lan_output  all      *      br-lan  ::/0                 ::/0                
       0        0 zone_wan_output  all      *      eth1    ::/0                 ::/0                

Chain forwarding_lan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain forwarding_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain forwarding_wan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain input_lan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain input_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain input_wan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain output_lan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain output_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain output_wan_rule (1 references)
    pkts      bytes target     prot opt in     out     source               destination         

Chain reject (3 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 REJECT     tcp      *      *       ::/0                 ::/0                 reject-with tcp-reset
       0        0 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-port-unreachable

Chain syn_flood (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 RETURN     tcp      *      *       ::/0                 ::/0                 tcp flags:0x17/0x02 limit: avg 25/sec burst 50
       0        0 DROP       all      *      *       ::/0                 ::/0                

Chain zone_lan_dest_ACCEPT (2 references)
    pkts      bytes target     prot opt in     out     source               destination         
      88     6504 ACCEPT     all      *      br-lan  ::/0                 ::/0                

Chain zone_lan_forward (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 forwarding_lan_rule  all      *      *       ::/0                 ::/0                 /* user chain for forwarding */
       0        0 zone_wan_dest_ACCEPT  all      *      *       ::/0                 ::/0                 /* forwarding lan -> wan */
       0        0 zone_lan_dest_ACCEPT  all      *      *       ::/0                 ::/0                

Chain zone_lan_input (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
      75     5890 input_lan_rule  all      *      *       ::/0                 ::/0                 /* user chain for input */
      75     5890 zone_lan_src_ACCEPT  all      *      *       ::/0                 ::/0                

Chain zone_lan_output (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
      88     6504 output_lan_rule  all      *      *       ::/0                 ::/0                 /* user chain for output */
      88     6504 zone_lan_dest_ACCEPT  all      *      *       ::/0                 ::/0                

Chain zone_lan_src_ACCEPT (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
      75     5890 ACCEPT     all      br-lan *       ::/0                 ::/0                

Chain zone_wan_dest_ACCEPT (2 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 ACCEPT     all      *      eth1    ::/0                 ::/0                

Chain zone_wan_dest_REJECT (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 reject     all      *      eth1    ::/0                 ::/0                

Chain zone_wan_forward (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 forwarding_wan_rule  all      *      *       ::/0                 ::/0                 /* user chain for forwarding */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 128 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 129 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 1 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 2 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 3 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 4 code 0 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 4 code 1 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Forward */
       0        0 zone_wan_dest_REJECT  all      *      *       ::/0                 ::/0                

Chain zone_wan_input (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 input_wan_rule  all      *      *       ::/0                 ::/0                 /* user chain for input */
       0        0 ACCEPT     udp      *      *       fe80::/10            fe80::/10            udp spt:547 dpt:546 /* Allow-DHCPv6 */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 128 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 129 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 1 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 2 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 3 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 4 code 0 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 4 code 1 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 133 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 135 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 134 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 ACCEPT     icmpv6    *      *       ::/0                 ::/0                 ipv6-icmptype 136 limit: avg 1000/sec burst 5 /* Allow-ICMPv6-Input */
       0        0 zone_wan_src_REJECT  all      *      *       ::/0                 ::/0                

Chain zone_wan_output (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 output_wan_rule  all      *      *       ::/0                 ::/0                 /* user chain for output */
       0        0 zone_wan_dest_ACCEPT  all      *      *       ::/0                 ::/0                

Chain zone_wan_src_REJECT (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 reject     all      eth1   *       ::/0                 ::/0                
ip6tables v1.4.21: can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.
Chain PREROUTING (policy ACCEPT 129 packets, 9778 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
     129     9778 delegate_notrack  all      *      *       ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 123 packets, 12678 bytes)
    pkts      bytes target     prot opt in     out     source               destination         

Chain delegate_notrack (1 references)
    pkts      bytes target     prot opt in     out     source               destination         
root@netcat:~# 



comment:6 Changed 3 years ago by bittorf@…

unsure. you have an IPv6-address from your provider and you have some IPv6-related rules. can you please disable the DSCP-rules just for testing and report if it happens again?

comment:7 Changed 3 years ago by tgayoso@…

no joy, disabling the IPV6 related rules and it crashed anyways.

comment:8 Changed 3 years ago by tgayoso@…

just noticed this crash was linked to dnsmasq instead of swapper

<6>[   31.752000] br-lan: port 3(wlan1) entered forwarding state
<6>[ 2109.368000] nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead.
<1>[48518.855000] CPU 0 Unable to handle kernel paging request at virtual address 60000000, epc == 87040040, ra == 8706a52c
<4>[48518.866000] Oops[#1]:
<4>[48518.866000] CPU: 0 PID: 1546 Comm: dnsmasq Not tainted 3.14.25 #1
<4>[48518.866000] task: 86e84e58 ti: 86eda000 task.ti: 86eda000
<4>[48518.866000] $ 0   : 00000000 00000000 60000000 858b651c
<4>[48518.866000] $ 4   : 86e94a00 86c580c0 00002000 0000009c
<4>[48518.866000] $ 8   : 0000318c 00000000 00000000 872de930
<4>[48518.866000] $12   : 00000000 00000000 80000000 20000000
<4>[48518.866000] $16   : 86c580c0 871fa0c0 86e94540 86e94a00
<4>[48518.866000] $20   : 00000000 87938700 20000000 00000006
<4>[48518.866000] $24   : 00000002 873612a0                  
<4>[48518.866000] $28   : 86eda000 86edb9a0 8034b7a0 8706a52c
<4>[48518.866000] Hi    : 00000000
<4>[48518.866000] Lo    : 000003f0
<4>[48518.866000] epc   : 87040040 ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[48518.866000]     Not tainted
<4>[48518.866000] ra    : 8706a52c ipv6_frag_exit+0x27d0/0x2f4c [ipv6]
<4>[48518.866000] Status: 1100fc03      KERNEL EXL IE 
<4>[48518.866000] Cause : 00800008
<4>[48518.866000] BadVA : 60000000
<4>[48518.866000] PrId  : 0001974c (MIPS 74Kc)
<4>[48518.866000] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 mac80211 ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_string xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_id xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_DSCP xt_CT xt_CLASSIFY ts_kmp ts_fsm ts_bm slhc nf_nat_irc nf_nat_ftp nf_nat nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_irc nf_conntrack_ftp nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_REJECT ipt_ECN ip_tables crc_ccitt compat sch_teql sch_tbf sch_sfq sch_red sch_prio sch_htb sch_gred sch_dsmark sch_codel em_text em_nbyte em_meta em_cmp cls_basic act_police act_ipt act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_ingress ledtrig_usbdev ip6t_REJECT ip6table_raw ip6table_mangle ip6table_filter ip6_tables x_tables ifb ipv6 arc4 crypto_blkcipher ohci_platform ohci_hcd ehci_platform ehci_hcd gpio_button_hotplug usbcore nls_base usb_common
<4>[48518.866000] Process dnsmasq (pid: 1546, threadinfo=86eda000, task=86e84e58, tls=7700e440)
<4>[48518.866000] Stack : 871fa0c0 86d1a000 00000601 86edba20 86edb9b8 80335a00 80106f68 80106bec
<4>[48518.866000]         00000000 800ae090 80330000 871fa0c0 86e94540 00000000 80341000 858b651c
<4>[48518.866000]         8039b150 8706a944 80335a00 871fa0c0 00000001 0000000a 00000000 801d40d4
<4>[48518.866000]         0000825a 80000000 803359f8 871fa0c0 00000035 86edba20 871fa0c0 86d66800
<4>[48518.866000]         87076210 803369e8 fffffffe 30000000 20000000 00000006 00000006 87045bf0
<4>[48518.866000]         ...
<4>[48518.866000] Call Trace:
<4>[48518.866000] [<87040040>] ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[48518.866000] [<8706a52c>] ipv6_frag_exit+0x27d0/0x2f4c [ipv6]
<4>[48518.866000] 
<4>[48518.866000] 
<4>[48518.866000] Code: 00671821  88620000  98620003 <88430000> 98430003  7c62d800  10400005  00000000  8c830040 
<4>[48519.130000] ---[ end trace 335e9f6e70e1b288 ]---

comment:9 Changed 3 years ago by Namidairo

Happens on latest trunk, both snapshot and own.

<1>[  892.570000] CPU 0 Unable to handle kernel paging request at virtual address 60000000, epc == 870c0040, ra == 870e9970
<4>[  892.580000] Oops[#1]:
<4>[  892.580000] CPU: 0 PID: 0 Comm: swapper Not tainted 3.14.25 #3
<4>[  892.580000] task: 80341158 ti: 80330000 task.ti: 80330000
<4>[  892.580000] $ 0   : 00000000 00000001 60000000 86a519c0
<4>[  892.580000] $ 4   : 863fdcd8 86a9ac00 863fdb80 03000000
<4>[  892.580000] $ 8   : 0015002a 00000000 00000015 0015002a
<4>[  892.580000] $12   : 0002b936 00000000 00000000 00000000
<4>[  892.580000] $16   : 86a9ac00 861ad180 86a51500 86a519c0
<4>[  892.580000] $20   : 00000000 8793aee0 00000006 803ac410
<4>[  892.580000] $24   : 00000000 870c6404
<4>[  892.580000] $28   : 80330000 80331998 8034ad30 870e9970
<4>[  892.580000] Hi    : 00000098
<4>[  892.580000] Lo    : 00000009
<4>[  892.580000] epc   : 870c0040 ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[  892.580000]     Not tainted
<4>[  892.580000] ra    : 870e9970 ipv6_frag_exit+0x1b70/0x2f38 [ipv6]
<4>[  892.580000] Status: 1100fc03      KERNEL EXL IE
<4>[  892.580000] Cause : 00800008
<4>[  892.580000] BadVA : 60000000
<4>[  892.580000] PrId  : 00019750 (MIPS 74Kc)
<4>[  892.580000] Modules linked in: pppoe ppp_async iptable_nat ath9k pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 ipt_MASQUERADE ath9k_common xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_id xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_CT ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda slhc nf_nat_irc nf_nat_ftp nf_nat nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_irc nf_conntrack_ftp nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_REJECT ip_tables crc_ccitt ath9k_hw ath10k_pci ath10k_core ath mac80211 cfg80211 compat ledtrig_usbdev ip6t_REJECT ip6table_raw ip6table_mangle ip6table_filter ip6_tables x_tables sit tunnel4 ip_tunnel vfat fat ntfs ipv6 arc4 crypto_blkcipher usb_storage ohci_platform ohci_hcd ehci_platform ehci_hcd sd_mod scsi_mod gpio_button_hotplug usbcore nls_base usb_common
<4>[  892.580000] Process swapper (pid: 0, threadinfo=80330000, task=80341158, tls=00000000)
<4>[  892.580000] Stack : 861ad180 87358000 00000601 80331a18 80331a18 80333a00 861ad180 801d17c0
<4>[  892.580000]         00000000 000086dd 80333190 861ad180 86a51500 00000000 8033fd70 863fdcd8
<4>[  892.580000]         00007cb6 870e9d8c 861ad180 00000001 80333a00 0000000a 87358000 801d136c
<4>[  892.580000]         870c5930 80000000 803339f8 861ad180 00000035 80331a18 861ad180 86889c00
<4>[  892.580000]         870f6260 00000006 fffffffe 803349e8 30000000 20000000 00000006 870c5b04
<4>[  892.580000]         ...
<4>[  892.580000] Call Trace:
<4>[  892.580000] [<870c0040>] ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[  892.580000] [<870e9970>] ipv6_frag_exit+0x1b70/0x2f38 [ipv6]
<4>[  892.580000]
<4>[  892.580000]
<4>[  892.580000] Code: 00c42021  88820000  98820003 <88440000> 98440003  7c84d800  10800005  00000000  8c640040
<4>[  892.830000] ---[ end trace 5b9e128f1d665786 ]---

comment:10 Changed 3 years ago by tgayoso@…

what I did to mitigate the effects and be able to get more than 24 hours of uptime in trunk, though it beats the purpose of the OpenWrt distribution, is to remove de WAN6 interface, remove all pre-defened IPV6 iptables rules, flush ip6tables from the command line and edited sysctl.conf with this:

net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.forwarding=0
net.ipv6.conf.all.forwarding=0

Did you experience the crash with DNSMasq also?

comment:11 Changed 3 years ago by tgayoso@…

To be able to get more than 24 hours of uptime in trunk, though it beats the purpose of the OpenWrt distribution, I removed de WAN6 interface, remove all pre-defened IPV6 iptables rules, flush ip6tables from the command line and edited sysctl.conf with this and rebooted:

net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.forwarding=0
net.ipv6.conf.all.forwarding=0

Does DNSMASQ crash for you too?

comment:12 Changed 3 years ago by nbd

  • Resolution set to fixed
  • Status changed from new to closed

fixed in r43560, make sure you clean your kernel tree after updating.

comment:13 Changed 3 years ago by robstoon

  • Resolution fixed deleted
  • Status changed from closed to reopened

I think I have been seeing the same problem on my Archer C7 v2. I just updated to r43584 but I got another crash. This one happened when running the ICSI Netalyzr utility. So it appears this problem still exists.

<1>[ 1163.500000] CPU 0 Unable to handle kernel paging request at virtual address 60000000, epc == 870c0040, ra == 870e9970
<4>[ 1163.510000] Oops[#1]:
<4>[ 1163.510000] CPU: 0 PID: 6034 Comm: dnsmasq Not tainted 3.14.26 #1
<4>[ 1163.510000] task: 86b25cb0 ti: 86a0e000 task.ti: 86a0e000
<4>[ 1163.510000] $ 0   : 00000000 00000000 60000000 86af2440
<4>[ 1163.510000] $ 4   : 86a18050 86ad7b40 86a18000 03000000
<4>[ 1163.510000] $ 8   : 00000001 00098a5e 00000000 0000002d
<4>[ 1163.510000] $12   : 00038cf8 80000379 00000000 00000000
<4>[ 1163.510000] $16   : 86ad7b40 86ad7780 86af1f80 86af2440
<4>[ 1163.510000] $20   : 00000000 8793e7e0 00000323 00000000
<4>[ 1163.510000] $24   : 00000000 80210850                  
<4>[ 1163.510000] $28   : 86a0e000 86a0fd08 00000000 870e9970
<4>[ 1163.510000] Hi    : 00000298
<4>[ 1163.510000] Lo    : 00000003
<4>[ 1163.510000] epc   : 870c0040 ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[ 1163.510000]     Not tainted
<4>[ 1163.510000] ra    : 870e9970 ipv6_frag_exit+0x1b80/0x2f48 [ipv6]
<4>[ 1163.510000] Status: 1100fc03	KERNEL EXL IE 
<4>[ 1163.510000] Cause : 00800008
<4>[ 1163.510000] BadVA : 60000000
<4>[ 1163.510000] PrId  : 00019750 (MIPS 74Kc)
<4>[ 1163.510000] Modules linked in: ifb pppoe ppp_async iptable_nat ath9k pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 ipt_MASQUERADE ath9k_common xt_time xt_tcpudp xt_tcpmss xt_string xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_id xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_DSCP xt_CT xt_CLASSIFY ts_kmp ts_fsm ts_bm slhc nf_nat_irc nf_nat_ftp nf_nat nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_irc nf_conntrack_ftp iptable_raw iptable_mangle iptable_filter ipt_REJECT ipt_ECN ip_tables crc_ccitt ath9k_hw fuse em_cmp em_nbyte sch_teql sch_dsmark act_ipt sch_gred sch_htb cls_basic sch_prio em_meta act_police em_text sch_codel sch_tbf sch_sfq sch_red sch_esfq act_connmark nf_conntrack act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_ingress ath10k_pci ath10k_core ath mac80211 cfg80211 compat ledtrig_usbdev ip6t_rt ip6t_frag ip6t_hbh ip6t_eui64 ip6t_mh ip6t_ah ip6t_ipv6header ip6t_REJECT ip6table_raw ip6table_mangle ip6table_filter ip6_tables x_tables msdos sit ip6_tunnel tunnel6 tunnel4 ip_tunnel vfat fat ipv6 arc4 crypto_blkcipher ohci_platform ohci_hcd ehci_platform ehci_hcd gpio_button_hotplug ext4 crc16 jbd2 mbcache usbcore nls_base usb_common crypto_hash [last unloaded: ifb]
<4>[ 1163.510000] Process dnsmasq (pid: 6034, threadinfo=86a0e000, task=86b25cb0, tls=775cf440)
<4>[ 1163.510000] Stack : 00000000 802814c4 803b2720 87320780 86b28a80 000000d0 87802680 000006e3
<4>[ 1163.510000] 	  00000323 86af2034 00000000 86af1f80 86ad7780 86ad7900 80340000 803136f8
<4>[ 1163.510000] 	  00000001 8021a55c 00000323 00000000 00000000 800ad234 00000321 86af1f80
<4>[ 1163.510000] 	  86b28a80 00000323 00000323 0083acd3 00000000 80278ff4 803395d0 800676f4
<4>[ 1163.510000] 	  7fffffff 80340000 00000594 00000000 000000a8 7fffffff 00000594 00000000
<4>[ 1163.510000] 	  ...
<4>[ 1163.510000] Call Trace:
<4>[ 1163.510000] [<870c0040>] ipv6_opt_accepted+0x40/0xb0 [ipv6]
<4>[ 1163.510000] [<870e9970>] ipv6_frag_exit+0x1b80/0x2f48 [ipv6]
<4>[ 1163.510000] 
<4>[ 1163.510000] 
<4>[ 1163.510000] Code: 00c42021  88820000  98820003 <88440000> 98440003  7c84d800  10800005  00000000  8c640040 
<4>[ 1163.800000] ---[ end trace 4dd9dd005ba9147b ]---

comment:14 Changed 3 years ago by robstoon

So it appears that 902-unaligned_access_hacks.patch, which nbd changed in the last fix, does touch that ipv6_opt_accepted function. I'm not exactly sure what difference the patch is doing to the code generation, but it's clearly something, as the generated assembly in that function is different with and without the patch on that file. From what I can see in the disassembly, it appears the compiler may be reordering some of the checks in the if statement in that function, which might be something to do with why it's blowing up.

I'm currently testing a build with the hunk of the patch that touches af_inet6.c removed to see if that fixes the crash.

comment:15 Changed 3 years ago by nbd

oh, right. i overlooked that one. this will probably fix the crash, i committed the change in r43586

comment:16 Changed 3 years ago by robstoon

Seems solid after the last fix. This can likely be marked as fixed.

comment:17 Changed 3 years ago by nbd

  • Resolution set to fixed
  • Status changed from reopened to closed

comment:18 Changed 3 years ago by pieter.smith@…

  • Resolution fixed deleted
  • Status changed from closed to reopened

Removing this chunk of the patch does reduce the likelihood of the oops occuring, but it does not really fix the issue. If I remove the chunk, the issue occurs once a day on our network. If I keep the chunk, the issue is reproducible with every boot. If I apply r43586, clean and rebuild the kernel, I still get the following on occasion:

procd: - init complete -
[   23.580000] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   23.580000] device eth0 entered promiscuous mode
[   23.590000] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   24.660000] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[   25.440000] random: nonblocking pool is initialized
[   25.950000] eth0: link up (100Mbps/Full duplex)
[   25.950000] br-lan: port 1(eth0) entered forwarding state
[   25.960000] br-lan: port 1(eth0) entered forwarding state
[   25.960000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   25.970000] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   27.960000] br-lan: port 1(eth0) entered forwarding state
[   88.400000] CPU 0 Unable to handle kernel paging request at virtual address 60000000, epc == 83300048, ra == 833292f0
[   88.410000] Oops[#1]:
[   88.410000] CPU: 0 PID: 0 Comm: swapper Not tainted 3.14.0 #13
[   88.410000] task: 803a4e88 ti: 8039e000 task.ti: 8039e000
[   88.410000] $ 0   : 00000000 00000000 60000000 82f8d480
[   88.410000] $ 4   : 82f9d050 82cf9300 82f9d000 03000000
[   88.410000] $ 8   : 00000001 000018e5 82cf9c00 00000000
[   88.410000] $12   : 00000000 29f31635 00000000 00000000
[   88.410000] $16   : 82cf9300 82cf9a80 82f8cfc0 82f8d480
[   88.410000] $20   : 00000000 83386000 00000006 8041d0f8
[   88.410000] $24   : 00000000 8007d0fc                  
[   88.410000] $28   : 8039e000 8039fba8 803bb4ec 833292f0
[   88.410000] Hi    : 00000009
[   88.410000] Lo    : 00000001
[   88.410000] epc   : 83300048 ipv6_opt_accepted+0x48/0xbc [ipv6]
[   88.410000]     Not tainted
[   88.410000] ra    : 833292f0 ipv6_frag_exit+0x1b80/0x2f48 [ipv6]
[   88.410000] Status: 1000fc03 KERNEL EXL IE 
[   88.410000] Cause : 00800008
[   88.410000] BadVA : 60000000
[   88.410000] PrId  : 00019374 (MIPS 24Kc)
[   88.410000] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv4 mac80211 ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comh
[   88.410000] Process swapper (pid: 0, threadinfo=8039e000, task=803a4e88, tls=00000000)
[   88.410000] Stack : 82cf9a80 82f28000 00000601 8039fc28 8039fbc0 803a1a30 8026e2d8 8026e2a0
          00000000 80275ac4 803a11c0 82cf9a80 82f8cfc0 00000000 803b9d50 82f9d050
          0000eaf7 8332970c 82cf9a80 00000001 803a1a30 0000000a 00000000 802a1280
          833056f0 80000000 803a1a28 82cf9a80 00000035 8039fc28 82cf9a80 82f62000
          83335be0 00000006 fffffffe 803a2a0c 30000000 20000000 00000006 833058c4
          ...
[   88.410000] Call Trace:
[   88.410000] [<83300048>] ipv6_opt_accepted+0x48/0xbc [ipv6]
[   88.410000] [<833292f0>] ipv6_frag_exit+0x1b80/0x2f48 [ipv6]
[   88.410000] 
[   88.410000] 
Code: 00c42021  88820000  98820003 <88440000> 98440003  7c84d800  10800005  00000000  8c620040 
[   88.650000] ---[ end trace ec68717a9ff185b1 ]---
[   88.660000] Kernel panic - not syncing: Fatal exception in interrupt
[   88.660000] Rebooting in 3 seconds..

If I don't apply r43586, the oops is reproducible on our network on every boot.

comment:19 Changed 3 years ago by nbd

please try latest trunk and see if the oops still occurs.

Add Comment

Modify Ticket

Action
as reopened .
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.