Modify

Opened 4 years ago

Closed 3 years ago

#16363 closed defect (fixed)

DDns Client nslookup does not work correctly

Reported by: openwrt-bugtracker@… Owned by: developers
Priority: normal Milestone: Chaos Calmer 15.05
Component: packages Version: Trunk
Keywords: Cc:

Description

I use DDns Client in conjunction with duckdns.org. This works fine but when testing I see updates are always done even if the correct IP is already set. I think /usr/lib/ddns/dynamic_dns_updater.sh
doesn't do the nslookup correct. The Variable registered_ip is always empty so the update is always triggered
The code in question is:

 registered_ip=$(echo $(nslookup "$domain" 2>/dev/null) |  grep -o "Name:.*" | grep -o "$ip_regex")

I don't understand that grep for Name: and it doesn't work in my case (and probably other DuckDNS users) Here is what the nslookup for my Host looks like:

root@OpenWrt:/etc/ssl/certs# nslookup genesis-ka.duckdns.org 2>/dev/null
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      genesis-ka.duckdns.org
Address 1: 79.227.133.57 p4FE38539.dip0.t-ipconnect.de
root@OpenWrt:/etc/ssl/certs#

I don't know in which cases Name might be useful but I would suggest to add a Regex or for Adress:

 registered_ip=$(echo $(nslookup "$domain" 2>/dev/null) |  grep -o "Name:\|Address.*" | grep -o "$ip_regex")


Attachments (0)

Change History (23)

comment:1 follow-up: Changed 4 years ago by ryts

The proposed change outputs localhost and Wan IPs

comment:2 in reply to: ↑ 1 Changed 4 years ago by openwrt-bugtracker@…

Replying to ryts:

The proposed change outputs localhost and Wan IPs

You are right. my tests on this were flawed but I can only think of an pretty unelegant solution: removing localhost with another grep

root@OpenWrt:/etc/ssl/certs# export ip_regex="[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\
{1,3\}\.[0-9]\{1,3\}"
root@OpenWrt:/etc/ssl/certs# nslookup "genesis-ka.duckdns.org" 2>/dev/null|grep -v localhost | grep -o "Name:\|Address.*" | grep -o "$ip_regex"
79.219.104.179
root@OpenWrt:/etc/ssl/certs#

Sorry but I don't see a good way with the busybox nslookup. The "correct" way would be calling the libc routine gethostbyname() but I dont see a way with the standard openwrt base. On full Linux installations I tend to use getent for scripting since this tool also hornors /etc/nsswitch.conf which is important if the system also uses radius/ldap/yellow pages/etc. for name resolution. This also applies to /etc/host entries so I think it is the most compatible way but obviously no option for openwrt. Example:

xxx@raspberrypi ~ $ getent hosts genesis-ka.duckdns.org
79.219.104.179  genesis-ka.duckdns.org

comment:3 Changed 4 years ago by openwrt-bugtracker@…

I think I'm going to adress the elephant in the room here. why does the busybox nslookup output localhost in the first place? I couldn't find a case where it is only printing the IP of the requested hostname

root@OpenWrt:/etc/ssl/certs# nslookup google.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      google.com
Address 1: 2a00:1450:4008:c01::64 bk-in-x64.1e100.net
Address 2: 173.194.69.139 bk-in-f139.1e100.net
Address 3: 173.194.69.138 bk-in-f138.1e100.net
Address 4: 173.194.69.100 bk-in-f100.1e100.net
Address 5: 173.194.69.101 bk-in-f101.1e100.net
Address 6: 173.194.69.113 bk-in-f113.1e100.net
Address 7: 173.194.69.102 bk-in-f102.1e100.net
root@OpenWrt:/etc/ssl/certs#
root@OpenWrt:/etc/ssl/certs# nslookup yahoo.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      yahoo.com
Address 1: 98.138.253.109 ir1.fp.vip.ne1.yahoo.com
Address 2: 98.139.183.24 ir2.fp.vip.bf1.yahoo.com
Address 3: 206.190.36.45 ir1.fp.vip.gq1.yahoo.com
root@OpenWrt:/etc/ssl/certs#

I don't know if busybox shows this behaviour only on openwrt (and if so: why?) or if it is a general problem und should be fixed upstream

comment:4 Changed 4 years ago by ryts

No, the behaviour is present on the Nokia n900 (Busybox v1.22) BUUUT that output format occurs also on nslookup (ie. dnsutils) on Ubuntu 14.04, so....

None of this invalidates your original point but the script is flawed in other ways: with your amended regex try the CLI:

usr/lib/ddns/dynamic_dns_updater.sh duckdns

Output shows that registered_ip fails, hence the constant updates.

comment:5 Changed 4 years ago by openwrt-bugtracker@…

I think I found a better way. But first: let's make clear what happens. nslookup outputs the nameserver which was used for name resolution and does a lookup on the server ip. So the grep -v won't work if another server than localhost is used. The first two lines returned will never be the requested name but always some information regarding the nameserver. In my opinion the safest solution is to skip the first two lines by adding a sed statement:

root@OpenWrt:/etc/ssl/certs# export ip_regex="[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\
{1,3\}\.[0-9]\{1,3\}"
root@OpenWrt:/etc/ssl/certs# nslookup "genesis-ka.duckdns.org" 2>/dev/null  | se
d   '1,2d' | grep -o "Name:\|Address.*" | grep -o "$ip_regex"
79.227.154.88
root@OpenWrt:/etc/ssl/certs#

Also tested this on a Debian wheezy with the dnsutils nslookup - works fine

comment:6 Changed 4 years ago by ryts

Acknowledged. But, is registered_ip reflected in the output of manually running the script as shown in above comment? It is not in my case with an amended regex.

comment:7 Changed 4 years ago by openwrt-bugtracker@…

You are right it was still broken. I removed the echo which seems redundant and now it works on my router with this code

registered_ip=$(nslookup "$domain" 2>/dev/null | sed  '1,2d'|  grep -o "Name:\|Address.*" | grep -o "$ip_regex")

This is the Result when I now execute it manually:
Obfuscated the URL since I had to hardcode the credentials because duckdns only accepts the subdomain and not the FQDN which won't work with nslookup

root@OpenWrt:/etc/ssl/certs# /usr/lib/ddns/dynamic_dns_updater.sh duckdns
update_url=https://www.duckdns.org/update?xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=[IP]
force seconds = 259200
check seconds = 600
old process id (if it exists) = ""
time_since_update = 0 hours
Running IP check...
current system ip = 80.143.246.92
registered domain ip = 80.143.246.92
update unnecessary
time since last update = 0 hours
the time is now Tue May 13 11:19:51 CEST 2014
^C
root@OpenWrt:/etc/ssl/certs#

comment:8 Changed 4 years ago by ryts

Great. As you will gather, I don't code. This needs a patch.

To summarise: as I read it, and you seem to think, the script is broken for all dynamic dns services, due to if [ "$current_ip" != "$registered_ip" ]. Unchanged, dynamic_dns_updater.sh cannot determine registered_ip (grep -o "Name:.*" in output of nslookup), so it updates regardless. As the ddns service just absorbs that, the error might be seen as trivial but it does invalidate much of what the script is designed to do.

Secondly, without a work-around, the ddns client system is broken for duckdns as the duckdns wants just hostname without parent domain and the script needs FQDN to submit to nslookup. The result is an update regardless of having a corrected dynamic_dns_updater.sh.

Please correct if my summary is wrong.

comment:9 Changed 4 years ago by openwrt-bugtracker@…

The summary is correct. I just want to add a few words. If I remember correctly the old free dyn.org service would ban your account from updates for a certain time if you make to many updates for a hostname. This was considered abusive and maybe some other services do this too, duckdns just absorbs it though just as you said. Anyway it will increase their load on that free service needlessly.So I think this is worth fixing. Regarding the FQDN issue: this isn't the fault of ddns Client. the duckdns update script should accept the FQDN like other DDNS Services do and the workaround for is in /etc/config/ddns is quite simple. Set option domain to the FQDN and hardcode the &domains value in the update_url option instead of using the [DOMAIN] macro

comment:10 Changed 4 years ago by anonymous

For info, second half of report:

/ticket/7820.html

comment:11 follow-up: Changed 4 years ago by mattbunce

Just to add another caveat to this. The latest version of the script does include a regex to extract the IP addresses from the nslookup. Unfortunately, my ISP uses a rather annoying domain notation which matches as an IP address, for example:

Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      dynamic.mydomain.co.uk
Address 1: 91.125.216.140 140.216.125.91.dyn.plus.net

As you can see, PlusNet switch the order of the IP address and include it with within the reverse DNS domain name - The current regex used matches this IP address too, so my registered ip is stored as "91.125.216.140 140.216.125.91" and therefore never matches the current IP.

This issue won't affect most users since the common way for ISP's to include customer IP addresses in the reverse DNS is to use hyphens. For example, this is what BT use:

Name:      dynamic2.mydomain.co.uk
Address 1: 86.139.31.49 host86-139-31-49.range86-139.btcentralplus.com

So my question is:
How can the regex be changed so only the actual IP address is matched and not an IP address which just happens to appear within a domain name?

comment:12 in reply to: ↑ 11 ; follow-up: Changed 4 years ago by openwrt-bugtracker@…

Replying to mattbunce:

Just to add another caveat to this. The latest version of the script does include a regex to extract the IP addresses from the nslookup. Unfortunately, my ISP uses a rather annoying domain notation which matches as an IP address, for example:

Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      dynamic.mydomain.co.uk
Address 1: 91.125.216.140 140.216.125.91.dyn.plus.net

As you can see, PlusNet switch the order of the IP address and include it with within the reverse DNS domain name - The current regex used matches this IP address too, so my registered ip is stored as "91.125.216.140 140.216.125.91" and therefore never matches the current IP.

This issue won't affect most users since the common way for ISP's to include customer IP addresses in the reverse DNS is to use hyphens. For example, this is what BT use:

Name:      dynamic2.mydomain.co.uk
Address 1: 86.139.31.49 host86-139-31-49.range86-139.btcentralplus.com

So my question is:
How can the regex be changed so only the actual IP address is matched and not an IP address which just happens to appear within a domain name?

Rather unfortunate but there's a way to get it working for every case. First let's take a look at what the current stuff does

root@OpenWrt:~# export ip_regex="[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]
\{1,3\}"
root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | gr
ep -o "$ip_regex"
91.125.216.140
140.216.125.91
root@OpenWrt:~#

Since the IP is matched correctly and in the domain name it is stored twice in the variable which will fail the comparison. I don't think a regex is the best way of doing this (and I can't think of any which will cover your case) so what does the output look like before the IP regex stuff:

root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"
Name:
Address 1: 91.125.216.140 140.216.125.91.dyn.plus.net
root@OpenWrt:~#

The first column contains Name or Adress the second the IP and the third contains the domain name. So only displaying the third column should solve this:

root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

91.125.216.140

Also works for the BT name you provided and for my case:

root@OpenWrt:~# export domain=host86-139-31-49.range86-139.btcentralplus.com
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

86.139.31.49
root@OpenWrt:~# export domain=genesis-ka.duckdns.org
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

79.227.164.77

so awk '{print $3}' instead of grep -o "$ip_regex" in /usr/lib/ddns/dynamic_dns_updater.sh should do the trick for everyone

comment:13 in reply to: ↑ 12 Changed 4 years ago by openwrt-bugtracker@…

Replying to openwrt-bugtracker@…:

Rather unfortunate but there's a way to get it working for every case. First let's take a look at what the current stuff does

root@OpenWrt:~# export ip_regex="[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]
\{1,3\}"
root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | gr
ep -o "$ip_regex"
91.125.216.140
140.216.125.91
root@OpenWrt:~#

Since the IP is matched correctly and in the domain name it is stored twice in the variable which will fail the comparison. I don't think a regex is the best way of doing this (and I can't think of any which will cover your case) so what does the output look like before the IP regex stuff:

root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"
Name:
Address 1: 91.125.216.140 140.216.125.91.dyn.plus.net
root@OpenWrt:~#

The first column contains Name or Adress the second the IP and the third contains the domain name. So only displaying the third column should solve this:

root@OpenWrt:~# export domain=140.216.125.91.dyn.plus.net
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

91.125.216.140

Also works for the BT name you provided and for my case:

root@OpenWrt:~# export domain=host86-139-31-49.range86-139.btcentralplus.com
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

86.139.31.49
root@OpenWrt:~# export domain=genesis-ka.duckdns.org
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '{print $3}'

79.227.164.77

so awk '{print $3}' instead of grep -o "$ip_regex" in /usr/lib/ddns/dynamic_dns_updater.sh should do the trick for everyone

After re-reading this I saw a mistake I did. completely removing the grep will add blank newlines to the result which will also lead to a failed match A solution would be to match for numbers within awk. example:

root@OpenWrt:~# export domain=host86-139-31-49.range86-139.btcentralplus.com
root@OpenWrt:~# nslookup $domain| sed   '1,2d' |grep -o "Name:\|Address.*"  | aw
k '/[0-9.]/{print $3}'
86.139.31.49
root@OpenWrt:~#

so awk '/[0-9.]/{print $3}' instead of grep -o "$ip_regex" in /usr/lib/ddns/dynamic_dns_updater.sh should actually work

comment:14 Changed 4 years ago by bittorf@…

you can do something like this with 'internals'

DOMAIN='host86-139-31-49.range86-139.btcentralplus.com'

here we go:

set -- $( nslookup "$DOMAIN" )

while shift
do
  case "$1" in
    Name:)
      PARSE='true'
    ;;
    Address)
      [ "$PARSE" = true ] || continue
      IP="$3"
      break
    ;;
  esac
done

output with:

echo "$IP"

Remember, that you must filter out IPv6-addresses if needed within the case-statement.

comment:15 Changed 4 years ago by jow

This shell hackery around "nslookup" is getting out of hand. Imho it is better to use the "resolveip" package for ddns-scripts.

comment:16 Changed 4 years ago by mattbunce

Thanks for all the help, especially with the awk and grep commands which I must admit are a little beyond my comprehension, so I take my hat off to you for being able to devise these solutions.

I have just got the opportunity to apply a fix to my router and I must admit I agree with jow - it seems things are getting a little tricky and since I'm not short of storage space I am going to install the resolveip package and use this within ddns-scripts.

Thanks once again

Matt

comment:17 Changed 3 years ago by earlyriser

Sorry to re-open this discussion but I have a similar problem to the original one where nslookup fails.

This is the output of nslookup:

Server:    127.0.0.1
Address 1: 127.0.0.1 localhost

Name:      cm-84.208.129.99.getinternet.no.
Address 1: 84.208.129.99 cm-84.208.129.99.getinternet.no

I think you can guess what the output of the 'grep -o "$ip_regex"' command results in.

Can you please just move over to using resolveip and be done with this horrible nslookup parsing hack?

comment:18 Changed 3 years ago by mattbunce

I gave up on regex in the end and used "resolveip" instead (You'll need to do a "opkg install resolveip" to install this first!).

I made the following patch to tweak the DDNS script:

119c119
<         retrieve_prog="/usr/bin/curl "
---
>         retrieve_prog="/usr/bin/curl -k "
284c284
<     registered_ip=$(echo $(nslookup "$domain" 2>/dev/null) |  grep -o "Name:.*" | grep -o "$ip_regex")
---
>     registered_ip=$(echo $(resolveip "$domain" 2>/dev/null))
299a300
>         logger -t dynamic_dns_updater.sh "$service_id update necessary: current IP = $current_ip, registered IP = $registered_ip"
330a332
>         logger -t dynamic_dns_updater.sh "$service_id update output: $update_output"
347a348
>        logger "$service_id no update required since last update = $human_time_since_update hours ago."

The above script...

  • Forces curl not to check SSL certificates for https posts
  • Uses resolveip instead of nslookup
  • Adds additional logging to keep me sane!

comment:19 Changed 3 years ago by mattbunce

Not sure why, but my message was posted twice...

Last edited 3 years ago by mattbunce (previous) (diff)

comment:20 Changed 3 years ago by anonymous

use this to support with or without space for ip address

registered_ip=$(echo $(nslookup "$domain" 2>/dev/null) |  grep -o "Name:.*" | grep -o "$ip_regex\($\| \)" | grep -o "$ip_regex")

comment:21 Changed 3 years ago by chris5560

Hi,
sorry I missed this ticket completely when monitoring TRAC.
To fix your problems, please provide me with the version of ddns-scripts you are talking about.
and possibly with a copy of the log-file via direct mail.
Thanks for support
Christian

comment:22 Changed 3 years ago by chris5560

Version 2.1.0-4 uses a modified "sed" to read ip from nslookup.
Tested with BusyBox nslookup and ISC nslookup.

Pull request created
Christian

P.S. if this fix your problems please close the ticket. Thanks.

comment:23 Changed 3 years ago by nbd

  • Resolution set to fixed
  • Status changed from new to closed

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.