[geek] initializing DST: openssl failure / upgrading ubuntu from lucid to precise broke DNS
I upgraded my linode server from ubuntu distro lucid to precise today. I didn't have any specific needs, but lucid is getting EOLed in a couple months.
It broke my DNS server. This is what ended up fixing it:
mkdir -p /usr/lib/i386-linux-gnu/openssl-1.0.0/engines
cp -a /usr/lib/i386-linux-gnu/openssl-1.0.0/engines /var/lib/named/usr/lib/i386-linux-gnu/openssl-1.0.0/
Where /var/lib/named was where I had bind chrooted to.
I noticed DNS didn't seem to be working. I ran:
# /etc/init.d/bind9 start
* Starting domain name service... bind9
...fail!
Then I checked /var/log/daemon.log, and noticed:
Feb 3 13:35:36 panic named[23991]: initializing DST: openssl failure
Feb 3 13:35:36 panic named[23991]: exiting (due to fatal error)
The useful way to find out what the problem was from there was:
strace -o named -ff named -u bind -t /var/lib/named
The "strace" command tells you, among other things, what files a program tries to read. "-o named" says to write to file "named", and "-ff named" handles forking into multiple threads, creating a file for each. "-u bind -t /var/lib/named" came from my /etc/default/bind9 :
OPTIONS="-u bind -t /var/lib/named"
"-u bind" means to run as user bind, and "-t /var/lib/named" means to run via chroot in /var/lib/named.
panic:~/tmp# grep -i ssl *
named.23906:open("/usr/lib/ssl/openssl.cnf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
named.23906:open("/usr/lib/i386-linux-gnu/openssl-1.0.0/engines/libgost.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
So there were some ssl related files it was trying to load and not finding. openssl.cnf was apparently not required.
I tried doing this via mount -bind in /etc/fstab, but it turns out you can't do that readonly. I added this to /etc/fstab:
/usr/lib/i386-linux-gnu/openssl-1.0.0/engines /var/lib/named/usr/lib/i386-linux-gnu/openssl-1.0.0/engines none bind,ro
But that commented that it got mounted writeable. Because... bind mounting doesn't have a way to do readonly.
And so I copied it. And will, apparently, need to forever maintain that manually. There's probably a better way. Haven't figured that out yet.
I also had some linode specific problems (login prompts went away), which were fixed by enabling "xenify distro".
It broke my DNS server. This is what ended up fixing it:
mkdir -p /usr/lib/i386-linux-gnu/openssl-1.0.0/engines
cp -a /usr/lib/i386-linux-gnu/openssl-1.0.0/engines /var/lib/named/usr/lib/i386-linux-gnu/openssl-1.0.0/
Where /var/lib/named was where I had bind chrooted to.
I noticed DNS didn't seem to be working. I ran:
# /etc/init.d/bind9 start
* Starting domain name service... bind9
...fail!
Then I checked /var/log/daemon.log, and noticed:
Feb 3 13:35:36 panic named[23991]: initializing DST: openssl failure
Feb 3 13:35:36 panic named[23991]: exiting (due to fatal error)
The useful way to find out what the problem was from there was:
strace -o named -ff named -u bind -t /var/lib/named
The "strace" command tells you, among other things, what files a program tries to read. "-o named" says to write to file "named", and "-ff named" handles forking into multiple threads, creating a file for each. "-u bind -t /var/lib/named" came from my /etc/default/bind9 :
OPTIONS="-u bind -t /var/lib/named"
"-u bind" means to run as user bind, and "-t /var/lib/named" means to run via chroot in /var/lib/named.
panic:~/tmp# grep -i ssl *
named.23906:open("/usr/lib/ssl/openssl.cnf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
named.23906:open("/usr/lib/i386-linux-gnu/openssl-1.0.0/engines/libgost.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
So there were some ssl related files it was trying to load and not finding. openssl.cnf was apparently not required.
I tried doing this via mount -bind in /etc/fstab, but it turns out you can't do that readonly. I added this to /etc/fstab:
/usr/lib/i386-linux-gnu/openssl-1.0.0/engines /var/lib/named/usr/lib/i386-linux-gnu/openssl-1.0.0/engines none bind,ro
But that commented that it got mounted writeable. Because... bind mounting doesn't have a way to do readonly.
And so I copied it. And will, apparently, need to forever maintain that manually. There's probably a better way. Haven't figured that out yet.
I also had some linode specific problems (login prompts went away), which were fixed by enabling "xenify distro".