nameserver 192.168.64.1 on OSX is not working correctly

[ailispaw]

nameserver 192.168.64.1 on OSX is not working correctly

[ailispaw]

Hi @bpinto ,

It's strange that it sill looks up index.docker.io on 192.168.64.1:53 after your modification.
Did you restart the VM after the modification of resolve.conf?
It's not persistent in boot2docker except Docker storage.

Your original problem is with nameserver 192.168.64.1 which is running on your Mac OSX.
Normally it works fine, but it seems something wrong with your host processes.
Please provide your OSX version and boot2docker version.
And also could you look at /etc/bootpd.plist while the VM is running?

Here is mine;

$ cat /etc/bootpd.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Subnets</key>
    <array>
        <dict>
            <key>_creator</key>
            <string>com.apple.NetworkSharing</string>
            <key>allocate</key>
            <true/>
            <key>dhcp_domain_name_server</key>
            <array>
                <string>192.168.64.1</string>
            </array>
            <key>dhcp_router</key>
            <string>192.168.64.1</string>
            <key>interface</key>
            <string>bridge100</string>
            <key>lease_max</key>
            <integer>86400</integer>
            <key>lease_min</key>
            <integer>86400</integer>
            <key>name</key>
            <string>192.168.64/24</string>
            <key>net_address</key>
            <string>192.168.64.0</string>
            <key>net_mask</key>
            <string>255.255.255.0</string>
            <key>net_range</key>
            <array>
                <string>192.168.64.2</string>
                <string>192.168.64.254</string>
            </array>
        </dict>
    </array>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_enabled</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>ignore_allow_deny</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>

[bpinto]

It's strange that it sill looks up index.docker.io on 192.168.64.1:53 after your modification.
Did you restart the VM after the modification of resolve.conf?
It's not persistent in boot2docker except Docker storage.

I didn't restart.

---

OSX: 10.11 Beta (15A278b)

Git: * 0940c82 - (HEAD, origin/master, origin/HEAD, master) Upgrade boot2docker to v1.9.0 (2 days ago) <A.I>

Boot2Docker version 1.9.0, build master : 16e4a2a - Tue Nov  3 19:49:22 UTC 2015
Docker version 1.9.0, build 76d6bc9

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Subnets</key>
    <array>
        <dict>
            <key>_creator</key>
            <string>com.apple.NetworkSharing</string>
            <key>allocate</key>
            <true/>
            <key>dhcp_domain_name_server</key>
            <array>
                <string>192.168.64.1</string>
            </array>
            <key>dhcp_router</key>
            <string>192.168.64.1</string>
            <key>interface</key>
            <string>bridge100</string>
            <key>lease_max</key>
            <integer>86400</integer>
            <key>lease_min</key>
            <integer>86400</integer>
            <key>name</key>
            <string>192.168.64/24</string>
            <key>net_address</key>
            <string>192.168.64.0</string>
            <key>net_mask</key>
            <string>255.255.255.0</string>
            <key>net_range</key>
            <array>
                <string>192.168.64.2</string>
                <string>192.168.64.254</string>
            </array>
        </dict>
    </array>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_enabled</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>ignore_allow_deny</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>

In case you would like to see the same file before I started the server (it doesn't seem to have changed): http://dpaste.com/14XB24S

[ailispaw]

Thanks for the info.
Here is in my case before starting and after shutting down the VM.

$ cat /etc/bootpd.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <false/>
    <key>dhcp_enabled</key>
    <false/>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>

It seems something else which uses NetworkSharing is running in your system.
But the file itself looks good, so DHCP server 192.168.64.1 should be fine.
You may be able to execute nslookup against 192.168.64.1 on OSX, right? Or not?

And I guess there is something wrong in the VM.
Anyway once you could use curl google.com with nameserver 8.8.8.8, the network connection is fine.
My biggest concern is why it still access to 192.168.64.1 with docker search ubuntu.

When you execute nslookup index.docker.io on the VM with nameserver 8.8.8.8,
what is the output?

[ailispaw]

One more thing, could you try https://github.com/ailispaw/docker-root-xhyve as well?
I would like to know if this one is fine or not on your system.

[bpinto]

I tried it with success:

[docker@docker-root ~]$ docker search ubuntu
NAME                           DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
ubuntu                         Ubuntu is a Debian-based Linux operating s...   2564      [OK]
ubuntu-upstart                 Upstart is an event-based replacement for ...   41        [OK]
torusware/speedus-ubuntu       Always updated official Ubuntu docker imag...   25                   [OK]
...

---

FYI, this happens on both repositories:

⋊> docker-root-xhyve on master ◦ make up                                                                                                                                  01:42:54
Password:
Booting up...
160:168: syntax error: Expected end of line but found identifier. (-2741)
make: *** [up] Error 1
⋊> docker-root-xhyve on master ◦ sudo ./xhyverun.sh                                                                                                                       01:42:59
"/Users/xxx" -network 192.168.64.0 -mask 255.255.255.0 -alldirs -mapall=501:20
Stopping the nfsd service (use 'disable' to make permanent)
Starting the nfsd service
Starting DockerRoot VM
....

[bpinto]

Sorry, I don't know much about network.

When you execute nslookup index.docker.io on the VM with nameserver 8.8.8.8,
what is the output?

docker@boot2docker:~$ nslookup index.docker.io
Server:    192.168.64.1
Address 1: 192.168.64.1

nslookup: can't resolve 'index.docker.io'

docker@boot2docker:~$ echo nameserver 8.8.8.8 > /etc/resolv.conf
docker@boot2docker:~$ nslookup index.docker.io
Server:    8.8.8.8
Address 1: 8.8.8.8 google-public-dns-a.google.com

Name:      index.docker.io
Address 1: 52.5.36.25 ec2-52-5-36-25.compute-1.amazonaws.com
Address 2: 52.1.242.236 ec2-52-1-242-236.compute-1.amazonaws.com
Address 3: 54.173.111.219 ec2-54-173-111-219.compute-1.amazonaws.com

---

You may be able to execute nslookup against 192.168.64.1 on OSX, right? Or not?

Is this what you meant?

⋊> boot2docker-xhyve on master ◦ nslookup 192.168.64.1                                                                                                                    01:52:36
Server:     8.8.8.8
Address:    8.8.8.8#53

** server can't find 1.64.168.192.in-addr.arpa.: NXDOMAIN

[bpinto]

By the way, I just got internet working on boot2docker:

docker@boot2docker:~$ echo nameserver 8.8.8.8 > /etc/resolv.conf
docker@boot2docker:~$ sudo /etc/init.d/docker restart
docker@boot2docker:~$ docker search ubuntu

Which explains why the nslookup worked but not the docker search. 😂

I don't know why I need to do it on boot2docker but not on docker-root...

[ailispaw]

Sorry for my poor English.

I mean;
On OSX,

$ nslookup
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
Server:     192.168.64.1
Address:    192.168.64.1#53

Non-authoritative answer:
index.docker.io canonical name = elb-io.us-east-1.aws.dckr.io.
elb-io.us-east-1.aws.dckr.io    canonical name = us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com.
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.1.242.236
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 54.173.111.219
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.5.36.25
>

[ailispaw]

docker@boot2docker:~$ sudo /etc/init.d/docker restart

Oh! I see.

[ailispaw]

160:168: syntax error: Expected end of line but found identifier. (-2741)
make: *** [up] Error 1

Hmm.. I will figure it out.

[ailispaw]

@bpinto
That syntax error is from AppleScript with osascript in bin/xhyveexec.sh.
Which terminal application do you use? Terminal or iTerm?

[ailispaw]

I don't know why I need to do it on boot2docker but not on docker-root...

/etc/resolve.conf in docker-root is as follows by default.

[docker@docker-root ~]$ cat /etc/resolv.conf
# Generated by dhcpcd from eth0.dhcp
# /etc/resolv.conf.head can replace this line
nameserver 192.168.64.1
nameserver 8.8.8.8
nameserver 8.8.4.4
options single-request-reopen

[bpinto]

Which terminal application do you use? Terminal or iTerm?

I'm using iTerm.

/etc/resolve.conf in docker-root is as follows by default.

👍 That explains it.

---

On OSX:

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:36:00
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

I didn't expect this! I have just replaced my router... Forgot to add Google's DNS to it!
Now I have configured it and nslookup has the expected output...

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:39:30
> index.docker.io
Server:     8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
index.docker.io canonical name = elb-io.us-east-1.aws.dckr.io.
elb-io.us-east-1.aws.dckr.io    canonical name = us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com.
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.5.36.25
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.1.242.236
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 54.173.111.219

But, same problem inside the VM:

docker@boot2docker:~$ docker search ubuntu
Error response from daemon: Get https://index.docker.io/v1/search?q=ubuntu: dial tcp: lookup index.docker.io on 192.168.64.1:53: read udp 192.168.64.1:53: connection refused

[ailispaw]

The test on OSX was whether the DNS server 192.168.64.1 is working.

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:36:00
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

The result means that the DNS server on your OSX is dead or not running correctly.
That's why you can not reach the server from the VM, either.

In conclusion, the VM itself is working fine, but your local DNS server in your host OSX is not somehow.
You have to fix the issue in OSX side, not the VM.
It may be caused by 10.11 BETA?

However, until the fix, you can use the VM by setting 8.8.8.8 as your DNS server instead of the local, as you have already found.
Another issue is that the modification for resolve.conf is not persistent across reboot and you have to restart the docker daemon as you showed me.
To solve this, you need to add the following line into the /var/lib/boot2docker/bootsync.sh which runs as a bootstrap before the Docker daemon starts.

echo "nameserver 8.8.8.8" > /etc/resolv.conf

And then reboot the VM.

[bpinto]

Thanks!

[girvo]

Hey @ailispaw I have a similar error

➜  code git:(master) ✗ nslookup
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

That's on my Mac. I wonder why that is?

If I ping 192.168.64.1:

  code git:(master) ✗ ping 192.168.64.1
PING 192.168.64.1 (192.168.64.1): 56 data bytes
64 bytes from 192.168.64.1: icmp_seq=0 ttl=64 time=0.054 ms
64 bytes from 192.168.64.1: icmp_seq=1 ttl=64 time=0.054 ms
64 bytes from 192.168.64.1: icmp_seq=2 ttl=64 time=0.137 ms
^C
--- 192.168.64.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.054/0.082/0.137/0.039 ms

[ailispaw]

Hmm.
I don't have a time now. I will check it out again later.
Thanks.

[okket]

FWIW: I ran in the same problem, but in my case I guess my local unbound DNS resolver listening only on 127.0.0.1 / ::1 was the problem. It was solved once I enabled unbound to listen on 0.0.0/0 and ::0/0 (don't forget to adjust allow also) .

(Related unbound setup gist: https://gist.github.com/denji/82dad8dee994ea197ea3)

[ailispaw]

This workaround may work for it.
nlf/dlite#151

[ailispaw]

#22 (comment)

[kim0]

In my case I was using dnsCrypt and disabling that made minishift work well again!

[ailispaw]

For me, in addition to the above, I needed the following command to get IP address correctly.

$ sudo /bin/launchctl load -w /System/Library/LaunchDaemons/bootps.plist
$ sudo /bin/launchctl unload -w /System/Library/LaunchDaemons/bootps.plist

It will activate bootpd to create /var/db/dhcpd_leases somehow.