I was successfully using a small (20,000 entries) zone file with bind9 server, but today my data provider sent an update which caused the zone file to become 300,000+ entries large (30Mb+).
The problem is the server would not start with this zone file. The named-checkconf would not report any errors. No log messages are available (or I could not log them properly).
I would like to know if bind9 is capable of handling large configuration files and if yes how do I fix it. If no I would like to know if there are any workarounds for this issue. Maybe it’s possible to store entries in a database?
The zone file I’m trying to use can be downloaded from here.
Update:
service bind9 status showed some information which may be relevant:
adjusted limit on open files from 4096 to 1048576 found 1 CPU, using 1 worker thread using 1 UDP listener per interface using up to 4096 sockets loading configuration from '/etc/bind/named.conf'
I’m not quite sure how to interpret or use this information… Any ideas?
Also I was not able to find where bind9 logs are located: /var/log/ has no bind9 entries. Can anybody tell me where they are located on Debian Jessie?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
I have seen your zone file: it appears to be a list of more than 350k domains, at the moment, where it is defined the local BIND server as the master. The domains are with the following format:
zone "xxxx.com" { type master; notify no; file "null.zone.file"; };
As per memory requirements, I would say as a ballpark figure you might need around 40MB-80MB of free RAM for that as domain tables are loaded in memory. (albeit I would feel more comfortable with 200MB at least)
Unless the server is severely constrained in RAM, it seems a bit improbable, but it could happen.
I also have noticed there are underscores (“_”) in the name of several domains. Having underscores in DNS RR breaks a couple of RFCs (RFC 952 and RFC 1123), and you need to add to the BIND options section the directive:
check-names master ignore;
As for the format and method being used for blacklisting domains. From version 9.8 onwards, BIND supports what is known as Response Policy Zones (RPZ), that were created specifically for blacklisting domains.
Several (commercial) blacklist providers follow nowadays that format. (I myself use RPZ both at work and at home).
Using RPZ should make more sense and also means a lighter load, and as such, if you are paying the service, I would advise you to contact your supplier to know how to use it. The RPZ format also supports to some extent wildcards, which would mean a much smaller blacklist file.
An alternative is also to process the file with a script to alter it to RPZ format.
I will leave here relevant links about RPZ and official RPZ providers:
and a tutorial how to configure RPZ:
http://www.zytrax.com/books/dns/ch9/rpz.html
As you may have noted, with the current configuration, you will also have a lot of open files; hence I recommend again using RPZ.
For dealing with more open files, in large email, DNS or HTTP servers, the limits have often to be raised.
The situation is not so bad as it used to be with older kernels, but nonetheless I do recommend raising the limits.
Edit /etc/sysctl.conf and modify/add the directive fs.file-max for the global limit of open files:
fs.file-max=500000
For applying the new file limit without rebooting, you need to run:
sudo sysctl -p
And for the files limits per process, edit, /etc/security/limits.conf:
* - nofile 400000
To apply the file limits per process, either logout and login, or run:
sudo ulimit -n 400000
After raising these two limits, you need to restart BIND:
sudo service bind9 restart
To convert your file to RPZ format, you run:
cat bind | tr -d " | awk ' { print $2" CNAME ." } ' > /etc/bind/rpz.db
The script will convert the entries to the following format:
zeus.developershed.com CNAME . zeusclicks.com CNAME . zintext.com CNAME .
Add in the options section of named:
response-policy { zone "rpz"; };
Create the declaration of the RPZ zone:
zone "rpz" {
type master;
file "/etc/bind/rpz.db";
};
Add to the top of /etc/bind/rpz.db file:
$TTL 604800
@ IN SOA localhost. root.localhost. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
@ IN NS your_dns_fqdn.
Deconfigure that DNS file of yours and restart your BIND server. Evidently the RPZ file can be optimised with wildcards and made much shorter, however even without that optimisation now you won’t need so much open files.
As for consulting BIND/DNS logs, they are together with the system logs in /var/log/syslog with the tag named. You can use the command:
sudo grep named /var/log/syslog
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0