Modify

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#18130 closed defect (fixed)

wlan stops working after random time

Reported by: anonymous Owned by: developers
Priority: normal Milestone:
Component: packages Version: Trunk
Keywords: Cc:

Description

router: wndr3800
version: r42707

config:

config wifi-device 'radio0'
	option type 'mac80211'
	option path 'pci0000:00/0000:00:11.0'
	list ht_capab 'SHORT-GI-40'
	list ht_capab 'TX-STBC'
	list ht_capab 'RX-STBC1'
	list ht_capab 'DSSS_CCK-40'
	option country 'DE'
	option hwmode '11g'
	option channel '11'
	option txpower '17'
	option distance '30'

config wifi-iface
	option device 'radio0'
	option mode 'ap'
	option ssid '...'
	option encryption 'psk2+ccmp'
	option network 'lan'
	option key '...'

Clients lose connection after some hours and wont get new dhcp IP. Restarting wlan via luci resolves the issue for a few hours or even a few days. Theres nothing to see in logread or dmesg.

Attachments (0)

Change History (22)

comment:1 Changed 3 years ago by bittorf@…

i have the same issue on TP-LINK TL-WR1043ND with r42884.
i use adhoc+ap on one radio - adhoc is working fine in this situation,
only 1 message in klog:

[39466.900000] ath: phy0: Failed to stop TX DMA, queues=0x004!

the router can hear other stations/laptops end responds for DHCP-requests
but the laptops dont receive them. restarting dnsmasq does not help, but
doing a 'wifi' - everything works normal again.

Thu Oct 16 17:02:20 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:20 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:23 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:23 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:25 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:25 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:02:49 2014 daemon.info hostapd: wlan0-1: STA 00:21:6a:32:7c:1c IEEE 802.11: authenticated
Thu Oct 16 17:02:49 2014 daemon.info hostapd: wlan0-1: STA 00:21:6a:32:7c:1c IEEE 802.11: associated (aid 2)
Thu Oct 16 17:02:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPREQUEST(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:02:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPACK(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c X301
Thu Oct 16 17:02:53 2014 daemon.info dnsmasq-dhcp[2456]: DHCPREQUEST(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:02:53 2014 daemon.info dnsmasq-dhcp[2456]: DHCPACK(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c X301
Thu Oct 16 17:02:57 2014 daemon.info dnsmasq-dhcp[2456]: DHCPREQUEST(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:02:57 2014 daemon.info dnsmasq-dhcp[2456]: DHCPACK(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c X301
Thu Oct 16 17:03:02 2014 user.notice root: debug: phy: phy0 interval: 60 avg: 64 avg_overall: 58 change: -9%
Thu Oct 16 17:03:05 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:05 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:07 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:07 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:16 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:16 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:03:30 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:03:30 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:03:32 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:03:32 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:03:34 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:03:34 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:02 2014 user.notice root: debug: phy: phy0 interval: 60 avg: 62 avg_overall: 59 change: -4%
Thu Oct 16 17:04:13 2014 daemon.info hostapd: wlan0-1: STA 00:21:6a:32:7c:1c IEEE 802.11: authenticated
Thu Oct 16 17:04:13 2014 daemon.info hostapd: wlan0-1: STA 00:21:6a:32:7c:1c IEEE 802.11: associated (aid 2)
Thu Oct 16 17:04:13 2014 daemon.info dnsmasq-dhcp[2456]: DHCPREQUEST(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:04:13 2014 daemon.info dnsmasq-dhcp[2456]: DHCPACK(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c X301
Thu Oct 16 17:04:17 2014 daemon.info dnsmasq-dhcp[2456]: DHCPREQUEST(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c 
Thu Oct 16 17:04:17 2014 daemon.info dnsmasq-dhcp[2456]: DHCPACK(wlan0-1) 10.63.22.67 00:21:6a:32:7c:1c X301
Thu Oct 16 17:04:39 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:39 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:41 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:41 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:42 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:42 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:04:44 2014 authpriv.info dropbear[10655]: Child connection from 10.63.48.65:34334
Thu Oct 16 17:04:48 2014 authpriv.notice dropbear[10655]: Password auth succeeded for 'root' from 10.63.48.65:34334
Thu Oct 16 17:05:02 2014 user.notice root: debug: phy: phy0 interval: 60 avg: 60 avg_overall: 59 change: -1%
Thu Oct 16 17:05:48 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:05:48 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:05:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:05:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:05:52 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) ac:18:26:ee:4e:ac 
Thu Oct 16 17:05:52 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.80 ac:18:26:ee:4e:ac 
Thu Oct 16 17:06:02 2014 user.notice root: debug: phy: phy0 interval: 60 avg: 59 avg_overall: 59 change: 0%
Thu Oct 16 17:06:11 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: authenticated
Thu Oct 16 17:06:11 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: associated (aid 2)
Thu Oct 16 17:06:11 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: disassociated
Thu Oct 16 17:06:12 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: authenticated
Thu Oct 16 17:06:12 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: associated (aid 2)
Thu Oct 16 17:06:19 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:19 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.88 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:23 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:23 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.88 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:40 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:40 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.88 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:42 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: disassociated
Thu Oct 16 17:06:42 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: authenticated
Thu Oct 16 17:06:42 2014 daemon.info hostapd: wlan0-1: STA 78:d6:f0:45:eb:55 IEEE 802.11: associated (aid 2)
Thu Oct 16 17:06:43 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:43 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.88 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPDISCOVER(wlan0-1) 78:d6:f0:45:eb:55 
Thu Oct 16 17:06:50 2014 daemon.info dnsmasq-dhcp[2456]: DHCPOFFER(wlan0-1) 10.63.22.88 78:d6:f0:45:eb:55 

before executing 'wifi', these are the stats:

root@DG-Fries36-Wohnzimmer:~ for W in reset interrupt queues;do for F in /sys/kernel/debug/ieee80211/phy*/ath9k/$W; do echo "### $F"; cat $F; d
one; done
### /sys/kernel/debug/ieee80211/phy0/ath9k/reset
    Baseband Hang:  0
Baseband Watchdog:  0
   Fatal HW Error:  0
      TX HW error:  0
 Transmit timeout:  0
     TX Path Hang:  1
      PLL RX Hang:  0
         MAC Hang:  0
     Stuck Beacon:  1
        MCI Reset:  0
Calibration error:  0
### /sys/kernel/debug/ieee80211/phy0/ath9k/interrupt
                   RX:    5501261
                RXEOL:          0
                RXORN:          0
                   TX:    1710078
                TXURN:          0
                  MIB:          0
                RXPHY:          0
                RXKCM:          0
                 SWBA:    3649303
                BMISS:          0
                  BNR:          0
                  CST:       5017
                  GTT:          0
                  TIM:          0
               CABEND:          0
             DTIMSYNC:          0
                 DTIM:          0
               TSFOOR:          0
                  MCI:          0
             GENTIMER:          0
                TOTAL:    9798722
SYNC_CAUSE stats:
             Sync-All:          0
              RTC-IRQ:          0
              MAC-IRQ:          0
EEPROM-Illegal-Access:          0
          APB-Timeout:          0
    PCI-Mode-Conflict:          0
          HOST1-Fatal:          0
           HOST1-Perr:          0
       TRCV-FIFO-Perr:          0
          RADM-CPL-EP:          0
  RADM-CPL-DLLP-Abort:          0
   RADM-CPL-TLP-Abort:          0
    RADM-CPL-ECRC-Err:          0
     RADM-CPL-Timeout:          0
    Local-Bus-Timeout:          0
            PM-Access:          0
            MAC-Awake:          0
           MAC-Asleep:          0
     MAC-Sleep-Access:          0
### /sys/kernel/debug/ieee80211/phy0/ath9k/queues
(VO):  qnum: 0 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(VI):  qnum: 1 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(BE):  qnum: 2 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(BK):  qnum: 3 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(CAB): qnum: 8 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0

comment:2 Changed 3 years ago by anonymous

Thank you so much @bittorf for proving that I am not crazy. I had opened a few tickets about this before, but with the false idea it was a dnsmasq/dhcp problem, and no one ever got back to me. The wlan is still "there", but clients will somehow hang during getting ip/dhcp auth. Restarting dnsmasq doesnt solve it, but restarting wifi does (for a few hours or days). Hope someone can figure out whats causing it.

comment:3 Changed 3 years ago by bittorf@…

next you see the prob (i forgot it too), you can raise the athk-debug level:

echo 0x4409 > /sys/kernel/debug/ieee80211/phy0/ath9k/debug
or
echo '0xffffffff' > /sys/kernel/debug/ieee80211/phy0/ath9k/debug

comment:4 Changed 3 years ago by anonymous

So any idea whats causing it or how to solve it?

comment:5 Changed 3 years ago by anonymous

Would be nice hearing from the devs something about than nothing. I ran into it on my Ultrabook now too because I wanted to copy 30gb of data over 5ghz Wlan and the connection dropped after a minute or so.

comment:6 Changed 3 years ago by anonymous

I cant rebug btw:

root@wndr:~# echo 0x4409 > /sys/kernel/debug/ieee80211/phy0/ath9k/debug
-ash: can't create /sys/kernel/debug/ieee80211/phy0/ath9k/debug: Permission deni
ed
root@wndr:~# echo '0xffffffff' > /sys/kernel/debug/ieee80211/phy0/ath9k/debug
-ash: can't create /sys/kernel/debug/ieee80211/phy0/ath9k/debug: Permission deni
ed
root@wndr:~#

comment:7 Changed 3 years ago by anonymous

I think this is the same problem: /ticket/15320.html

comment:8 Changed 3 years ago by anonymous

Not sure though because clients dont get new ip via dhcp until wlan is restarted from the router.

comment:9 Changed 3 years ago by anonymous

root@wrt201:~# cat /sys/kernel/debug/ieee80211/phy0/ath9k/reset
    Baseband Hang:  0
Baseband Watchdog:  0
   Fatal HW Error:  0
      TX HW error:  0
 Transmit timeout:  0
     TX Path Hang:  0
      PLL RX Hang:  0
         MAC Hang: 15
     Stuck Beacon:  1
        MCI Reset:  0
Calibration error:  0

Is this helpful? What does MAX Hang und Stuck Beacon mean? This is the current status after I wanted to copy those files, I still can connect to the WiFi but the connection is limited says Windows. And I cant ping anything.

comment:10 Changed 3 years ago by bittorf@…

please test r42952 or later

comment:11 Changed 3 years ago by nbd

please show the output of these commands while it's hung:

cat /sys/kernel/debug/ieee80211/phy0/ath9k/queues
cat /sys/kernel/debug/ieee80211/phy0/queues

comment:12 Changed 3 years ago by anonymous

Another hang 10 minutes ago, build r42960. So the fixes dont work :( There are 5 connected clients shown under DHCP Leases, but I cannot ping any of them anymore from the router, and connection is broken on the clients too.

root@wndr:~# cat /sys/kernel/debug/ieee80211/phy0/ath9k/queues
(VO):  qnum: 0 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(VI):  qnum: 1 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(BE):  qnum: 2 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(BK):  qnum: 3 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
(CAB): qnum: 8 qdepth:  0 ampdu-depth:  0 pending:   0 stopped: 0
root@wndr:~# cat /sys/kernel/debug/ieee80211/phy0/queues
00: 0x00000000/0
01: 0x00000000/0
02: 0x00000001/1
03: 0x00000000/0
04: 0x00000000/0
05: 0x00000000/0
06: 0x00000000/0
07: 0x00000000/0
08: 0x00000000/0
09: 0x00000000/0

comment:14 Changed 3 years ago by anonymous

How do I apply it, can I just do a clean svn clone and copy the file 999-ath9k-Enable-HW-queue-control-only-for-MCC.patch into /home/openwrt/trunk/package/kernel/mac80211/patches and it will include the patch automatically?

comment:15 Changed 3 years ago by nbd

apply it with patch -p1 on top of your existing trunk directory

comment:16 Changed 3 years ago by anonymous

openwrt@Server:~/trunk$ patch -p1 0001-Add-a-HW-queue-control-patch.patch

That doesnt do anything and I have to ctrl+c to quit it.

comment:17 Changed 3 years ago by anonymous

openwrt@Server:~/trunk$ patch -p1 < 0001-Add-a-HW-queue-control-patch.patch
patching file package/kernel/mac80211/patches/999-ath9k-Enable-HW-queue-control-only-for-MCC.patch
openwrt@Server:~/trunk$

That seemed to work, compiling now and will test.

comment:18 Changed 3 years ago by nbd

  • Resolution set to fixed
  • Status changed from new to closed

fix committed in r43011

comment:19 Changed 3 years ago by anonymous

  • Resolution fixed deleted
  • Status changed from closed to reopened

compiled with r43011
Wed Oct 22 16:14:06 2014 kern.err kernel: [17855.500000] ath: phy1: Failed to stop TX DMA, queues=0x004!

cat /sys/kernel/debug/ieee80211/phy0/ath9k/reset
Baseband Hang:  0
Baseband Watchdog:  0
   Fatal HW Error:  1
      TX HW error:  0
 Transmit timeout:  0
     TX Path Hang:  0
      PLL RX Hang:  0
         MAC Hang:  0
     Stuck Beacon: 128
        MCI Reset:  0
Calibration error:  0
Wed Oct 22 16:06:03 2014 kern.err kernel: [17371.860000] INFO: rcu_sched self-detected stall on CPU { 0}  (t=186030 jiffies g=180403 c=180402 q=15924)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.49 #1
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] Backtrace: 
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0019e64>] (dump_backtrace+0x0/0x114) from [<c001a080>] (show_stack+0x18/0x1c)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r6:c03bd0e4 r5:00003e34 r4:c03b86c0 r3:00000000
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c001a068>] (show_stack+0x0/0x1c) from [<c01384e8>] (dump_stack+0x20/0x28)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c01384c8>] (dump_stack+0x0/0x28) from [<c0073258>] (rcu_check_callbacks+0x1dc/0x694)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c007307c>] (rcu_check_callbacks+0x0/0x694) from [<c00323b8>] (update_process_times+0x48/0x74)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0032370>] (update_process_times+0x0/0x74) from [<c005f7c8>] (tick_sched_timer+0x1c0/0x204)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r7:c03fdf70 r6:c03bbe08 r5:00000fcc r4:b2f30e5c
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c005f608>] (tick_sched_timer+0x0/0x204) from [<c00466d4>] (__run_hrtimer.isra.34+0x6c/0x10c)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0046668>] (__run_hrtimer.isra.34+0x0/0x10c) from [<c0046ec4>] (hrtimer_interrupt+0x148/0x2d8)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r7:00000001 r6:c0603440 r5:00000fcc r4:b2f30e5c
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0046d7c>] (hrtimer_interrupt+0x0/0x2d8) from [<c001c36c>] (twd_handler+0x38/0x4c)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c001c334>] (twd_handler+0x0/0x4c) from [<c006e28c>] (handle_percpu_devid_irq+0x88/0xa8)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c006e204>] (handle_percpu_devid_irq+0x0/0xa8) from [<c006a908>] (generic_handle_irq+0x30/0x44)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r8:c03bbf18 r7:0000001d r6:c03ba000 r5:c03b8b44 r4:0000001d
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] r3:c006e204
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c006a8d8>] (generic_handle_irq+0x0/0x44) from [<c00173e4>] (handle_IRQ+0x84/0xb8)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r8:00000000 r7:fb004100 r6:c03bbe08 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] r3:00000010
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] Exception stack(0xc03bbe08 to 0xc03bbe50)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] be00:                   00000001 c03ba010 00000000 00000000 00000040 c03b8b44
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] be20: c03ba000 c03bc040 00000000 c0603760 c03ba000 c03bbe9c c03f9b60 c03bbe50
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] be40: 0000000a c002c54c 20000113 ffffffff
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r7:c03bbe3c r6:ffffffff r5:20000113 r4:c002c54c
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c002c4ac>] (__do_softirq+0x0/0x1e4) from [<c002c738>] (do_softirq+0x50/0x5c)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c002c6e8>] (do_softirq+0x0/0x5c) from [<c002c9b8>] (irq_exit+0x7c/0x94)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r4:c03bd2c8 r3:00000040
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c002c93c>] (irq_exit+0x0/0x94) from [<c00173ec>] (handle_IRQ+0x8c/0xb8)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r8:60000013 r7:fb004100 r6:c03bbf18 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] r3:00000010
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] Exception stack(0xc03bbf18 to 0xc03bbf60)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] bf00:                                                       c0603760 00000000
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] bf20: 00e9f3bf 00000000 00000000 00000000 c03b8760 0024b000 60000013 c0603760
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] bf40: c03ba000 c03bbf8c c03bbf60 c03bbf60 c00727c0 c00727c4 60000013 ffffffff
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r7:c03bbf4c r6:ffffffff r5:60000013 r4:c00727c4
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c007271c>] (rcu_idle_exit+0x0/0xbc) from [<c00563f4>] (cpu_startup_entry+0xb8/0x130)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c005633c>] (cpu_startup_entry+0x0/0x130) from [<c000c518>] (rest_init+0x68/0x80)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000]  r7:c0600ca0 r3:00000000
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c000c4b0>] (rest_init+0x0/0x80) from [<c038fab8>] (start_kernel+0x2a8/0x310)
Wed Oct 22 16:06:03 2014 kern.warn kernel: [17371.860000] [<c038f810>] (start_kernel+0x0/0x310) from [<20008070>] (0x20008070)
Wed Oct 22 16:09:03 2014 kern.err kernel: [17551.890000] INFO: rcu_sched self-detected stall on CPU { 0}  (t=204033 jiffies g=180403 c=180402 q=17062)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.49 #1
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] Backtrace: 
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0019e64>] (dump_backtrace+0x0/0x114) from [<c001a080>] (show_stack+0x18/0x1c)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r6:c03bd0e4 r5:000042a6 r4:c03b86c0 r3:00000000
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c001a068>] (show_stack+0x0/0x1c) from [<c01384e8>] (dump_stack+0x20/0x28)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c01384c8>] (dump_stack+0x0/0x28) from [<c0073258>] (rcu_check_callbacks+0x1dc/0x694)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c007307c>] (rcu_check_callbacks+0x0/0x694) from [<c00323b8>] (update_process_times+0x48/0x74)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0032370>] (update_process_times+0x0/0x74) from [<c005f7c8>] (tick_sched_timer+0x1c0/0x204)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r7:c03fdf70 r6:c03bbe08 r5:00000ff6 r4:9d92d347
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c005f608>] (tick_sched_timer+0x0/0x204) from [<c00466d4>] (__run_hrtimer.isra.34+0x6c/0x10c)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0046668>] (__run_hrtimer.isra.34+0x0/0x10c) from [<c0046ec4>] (hrtimer_interrupt+0x148/0x2d8)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r7:00000001 r6:c0603440 r5:00000ff6 r4:9d92d347
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0046d7c>] (hrtimer_interrupt+0x0/0x2d8) from [<c001c36c>] (twd_handler+0x38/0x4c)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c001c334>] (twd_handler+0x0/0x4c) from [<c006e28c>] (handle_percpu_devid_irq+0x88/0xa8)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c006e204>] (handle_percpu_devid_irq+0x0/0xa8) from [<c006a908>] (generic_handle_irq+0x30/0x44)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r8:c03bbf18 r7:0000001d r6:c03ba000 r5:c03b8b44 r4:0000001d
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] r3:c006e204
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c006a8d8>] (generic_handle_irq+0x0/0x44) from [<c00173e4>] (handle_IRQ+0x84/0xb8)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r8:00000000 r7:fb004100 r6:c03bbe08 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] r3:00000010
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] Exception stack(0xc03bbe08 to 0xc03bbe50)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] be00:                   00000001 c03ba010 00000000 00000000 00000040 c03b8b44
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] be20: c03ba000 c03bc040 00000000 c0603760 c03ba000 c03bbe9c c03f9b60 c03bbe50
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] be40: 0000000a c002c54c 20000113 ffffffff
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r7:c03bbe3c r6:ffffffff r5:20000113 r4:c002c54c
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c002c4ac>] (__do_softirq+0x0/0x1e4) from [<c002c738>] (do_softirq+0x50/0x5c)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c002c6e8>] (do_softirq+0x0/0x5c) from [<c002c9b8>] (irq_exit+0x7c/0x94)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r4:c03bd2c8 r3:00000040
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c002c93c>] (irq_exit+0x0/0x94) from [<c00173ec>] (handle_IRQ+0x8c/0xb8)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r8:60000013 r7:fb004100 r6:c03bbf18 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] r3:00000010
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] Exception stack(0xc03bbf18 to 0xc03bbf60)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] bf00:                                                       c0603760 00000000
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] bf20: 00e9f3bf 00000000 00000000 00000000 c03b8760 0024b000 60000013 c0603760
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] bf40: c03ba000 c03bbf8c c03bbf60 c03bbf60 c00727c0 c00727c4 60000013 ffffffff
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r7:c03bbf4c r6:ffffffff r5:60000013 r4:c00727c4
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c007271c>] (rcu_idle_exit+0x0/0xbc) from [<c00563f4>] (cpu_startup_entry+0xb8/0x130)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c005633c>] (cpu_startup_entry+0x0/0x130) from [<c000c518>] (rest_init+0x68/0x80)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000]  r7:c0600ca0 r3:00000000
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c000c4b0>] (rest_init+0x0/0x80) from [<c038fab8>] (start_kernel+0x2a8/0x310)
Wed Oct 22 16:09:03 2014 kern.warn kernel: [17551.890000] [<c038f810>] (start_kernel+0x0/0x310) from [<20008070>] (0x20008070)
Wed Oct 22 16:10:01 2014 cron.info crond[1239]: crond: USER root pid 3099 cmd /usr/sbin/check-cpu
Wed Oct 22 16:10:02 2014 cron.info crond[1239]: crond: USER root pid 3100 cmd /usr/sbin/check-kacd
Wed Oct 22 16:12:03 2014 kern.err kernel: [17731.920000] INFO: rcu_sched self-detected stall on CPU { 0}  (t=222036 jiffies g=180403 c=180402 q=18844)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.49 #1
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] Backtrace: 
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0019e64>] (dump_backtrace+0x0/0x114) from [<c001a080>] (show_stack+0x18/0x1c)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r6:c03bd0e4 r5:0000499c r4:c03b86c0 r3:00000000
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c001a068>] (show_stack+0x0/0x1c) from [<c01384e8>] (dump_stack+0x20/0x28)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c01384c8>] (dump_stack+0x0/0x28) from [<c0073258>] (rcu_check_callbacks+0x1dc/0x694)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c007307c>] (rcu_check_callbacks+0x0/0x694) from [<c00323b8>] (update_process_times+0x48/0x74)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0032370>] (update_process_times+0x0/0x74) from [<c005f7c8>] (tick_sched_timer+0x1c0/0x204)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r7:c03fdf70 r6:c03bbe08 r5:00001020 r4:88327c85
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c005f608>] (tick_sched_timer+0x0/0x204) from [<c00466d4>] (__run_hrtimer.isra.34+0x6c/0x10c)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0046668>] (__run_hrtimer.isra.34+0x0/0x10c) from [<c0046ec4>] (hrtimer_interrupt+0x148/0x2d8)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r7:00000001 r6:c0603440 r5:00001020 r4:88327c85
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0046d7c>] (hrtimer_interrupt+0x0/0x2d8) from [<c001c36c>] (twd_handler+0x38/0x4c)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c001c334>] (twd_handler+0x0/0x4c) from [<c006e28c>] (handle_percpu_devid_irq+0x88/0xa8)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c006e204>] (handle_percpu_devid_irq+0x0/0xa8) from [<c006a908>] (generic_handle_irq+0x30/0x44)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r8:c03bbf18 r7:0000001d r6:c03ba000 r5:c03b8b44 r4:0000001d
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] r3:c006e204
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c006a8d8>] (generic_handle_irq+0x0/0x44) from [<c00173e4>] (handle_IRQ+0x84/0xb8)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r8:00000000 r7:fb004100 r6:c03bbe08 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] r3:00000010
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] Exception stack(0xc03bbe08 to 0xc03bbe50)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] be00:                   00000001 c03ba010 00000000 00000000 00000040 c03b8b44
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] be20: c03ba000 c03bc040 00000000 c0603760 c03ba000 c03bbe9c c03f9b60 c03bbe50
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] be40: 0000000a c002c54c 20000113 ffffffff
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r7:c03bbe3c r6:ffffffff r5:20000113 r4:c002c54c
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c002c4ac>] (__do_softirq+0x0/0x1e4) from [<c002c738>] (do_softirq+0x50/0x5c)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c002c6e8>] (do_softirq+0x0/0x5c) from [<c002c9b8>] (irq_exit+0x7c/0x94)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r4:c03bd2c8 r3:00000040
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c002c93c>] (irq_exit+0x0/0x94) from [<c00173ec>] (handle_IRQ+0x8c/0xb8)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r4:c03bd2c8 r3:000000a0
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0017360>] (handle_IRQ+0x0/0xb8) from [<c00084dc>] (gic_handle_irq+0x44/0x64)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r8:60000013 r7:fb004100 r6:c03bbf18 r5:c03bd3f0 r4:fb00410c
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] r3:00000010
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c0008498>] (gic_handle_irq+0x0/0x64) from [<c0008aa0>] (__irq_svc+0x40/0x54)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] Exception stack(0xc03bbf18 to 0xc03bbf60)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] bf00:                                                       c0603760 00000000
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] bf20: 00e9f3bf 00000000 00000000 00000000 c03b8760 0024b000 60000013 c0603760
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] bf40: c03ba000 c03bbf8c c03bbf60 c03bbf60 c00727c0 c00727c4 60000013 ffffffff
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r7:c03bbf4c r6:ffffffff r5:60000013 r4:c00727c4
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c007271c>] (rcu_idle_exit+0x0/0xbc) from [<c00563f4>] (cpu_startup_entry+0xb8/0x130)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c005633c>] (cpu_startup_entry+0x0/0x130) from [<c000c518>] (rest_init+0x68/0x80)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000]  r7:c0600ca0 r3:00000000
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c000c4b0>] (rest_init+0x0/0x80) from [<c038fab8>] (start_kernel+0x2a8/0x310)
Wed Oct 22 16:12:03 2014 kern.warn kernel: [17731.920000] [<c038f810>] (start_kernel+0x0/0x310) from [<20008070>] (0x20008070)
Wed Oct 22 16:14:06 2014 kern.err kernel: [17855.500000] ath: phy1: Failed to stop TX DMA, queues=0x004!

comment:20 Changed 3 years ago by nbd

  • Resolution set to fixed
  • Status changed from reopened to closed

That's a different issue. I'm assuming you're using MiniPCI cards on a Gateworks GW2388? If so, you probably have the old revision with the PCI clock hardware problem.

comment:21 Changed 3 years ago by anonymous

yes,
I redo the test with rev6 (external PCI clock)
Thank you

comment:22 Changed 3 years ago by anonymous

Gateworks GW2388 rev 2, not working --> PCI clock hardware problem
Gateworks GW2388 rev 3, not working --> PCI clock hardware problem
Gateworks GW2388 rev 4, not working --> PCI clock hardware problem
Gateworks GW2388 rev 5, not working --> PCI clock hardware problem
Gateworks GW2388 rev 6 OK
Gateworks GW2388 rev 7 OK

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.