typo3bloke.net

In this article I am going to discuss advanced guestbook spam blocking. "Advanced" means that techniques will require not TYPO3 configuration but compiling, configuring and installing additional server components. This article is intended to web server administrators who are not afraid to protect their customers by installing non-traditional software.

I have to put standard disclaimer here: this technique works well for me. However it may not work for you. It may block some of your customers (though I provide ways to unblock most of them). If you use this technique, you must acknowledge that you do it with full understanding and at your own risk.

After such serious and dangerous warning. let's move to real work.

How did I come to the idea

I have mod_security installed on this web server. It can log many bad actions and even protect from typical spam. However it requires certain configuration and tuning. Logs may become really huge (for example, today I have several hundreds spam attempts blocked and logged, many kilobytes each). It would be great just to block spammers by IP address. The same way as PostFix can block spammers using DNS blocking lists (DNSBLs). This will free lots of resources on popular servers.

Can it be done? Yes. Can it be done safely? Yes. Can it be done effectively? Yes.

The idea

The idea consists from several parts:

1. Block well-known spam networks using iptables (or another method)
2. Use cbl.abuseat.org to block infected computers that spammers use to send spam.
3. Use our own local DNSBL to dynamically block spammers, who is not yet in cbl.abuseat.org.
4. Block some countries using GeoIP information.
5. Unblock potential customers from your own country using GeoIP information.

Prerequistes

You need the following software on your server:

• Apache with apache-devel package installed
• gcc, make, ldd and other C building tools
• Command line version of PHP
• root account or sudo access

Block spam networks

Spammers are not chaotic. Spam business is criminal and gives huge amount of money. Therefore spammers are very-well organized. They have whole networks that server their needs. Fortunately, there is an easy way to block access from these networks. SpamHaus project provides a list of identified criminal networks. This is called DROP (Don't Route Or Peer). By blocking these networks you loose nothing but gain a lot: spam decrease dramatically.

A list of networks is contained in drop.lasso downloadable file. To use this file you need a small shell script that will fetch the file once a day and block listed networks using iptables. If you use ipchains or another firewalling method, adjust process accordingly.

Grab DROP.sh file from the end of this article and put it somewhere (this is a modified version of SpamHaus file with proper cleanup for iptables). I assume /root/cron-jobs/DROP.sh. Now make it executable:

chmod u+x /root/cron-jobs/DROP.sh

and add it to cron using

crontab -e

This will open cron jobs for editing (typically in vi or vim editor). Add the following line (press i in vi/vim to add characters):

0 3 * * * /root/cron-jobs/DROP.sh &>/dev/null

Save and quit (press :wq in vi/vim for this). You can also execute DRIOP.sh manually for the first time to load data immediately:

/root/cron-jobs/DROP.sh

Preparing to use DNSBL

To use DNSBLs in Apache we need to compile, install and configure additional module. Module is called mod_setenvdnsbl. It can query DNSBLs and set environment variable, which we will use to block spammers. There are other modules that can just block by DNSBL but using variable is better because you can unblock some clients manually even if they are in DNSBL.

Unfortunately original source file for mod_setenvdnsbl has several problems. It makes DNSBL requests for each HTTP request (even for GET, which is inefficient) and pollutes Apache logs for each requests. It also have a bug that prevents us from using more than one DNSBL. Therefore you should get modified mod_setenvdnsbl.c file from the end of the article.

Put the file somewhere in the file system (/tmp/mod_setenvdnsbl is a good place). Now go to that directory and execute:

apxs2 -cia mod_setenvdnsbl

apxs2 is a module compiling tool from Apache. "2" in the end means that it is for Apache 2. If you still use old Apache 1.x, proper name will be "apxs". "-cia" means "compile, install and activate". On most systems this is enough to get module enabled but some systems use different configuration. For example, SuSE requires manual changes in /etc/sysconfig/apache2. There is a directive in that file with list of active modules. If you use SuSE, you have to add mod_setenvdnsbl there. See /etc/sysconfig/apache2 for details.

Configuring mod_setenvdnsbl

Now we are going to configure mod_setenvdnsbl. We will create a separate file. Go to the Apache configuration directory (typically /etc/apache2) and create a new file named dnsbl_block.conf with your favorite text editor. Put the following inside:

<IfModule mod_setenvdnsbl.c>
SetEnvIfDNSBL remote_addr cbl.abuseat.org dnsbl_spam
<Limit POST>
Order Deny,Allow
Deny from env=dnsbl_spam
</Limit>
</IfModule>

Save the file.

The first line checks if we have mod_setenvdnsbl loaded. If you ever decide to stop using it, you can just remove module from the list of loaded modules and do not have to comment configuration.

Next line will instruct mod_setenvdnsbl to check client's IP address over cbl.abuseat.org and, if found, set dnsbl_spam environment variable.

Lines 3 to 6 say that POST request should be allowed unless dnsbl_spam variable is set. Notice usage of <Limit>. You may ask why did I put SetEnvIfDNSBL into <Limit> statement. The answer is simple: <Limit> applies only to access directives (Order, Deny and Allow) and I could not find how to apply it to other directives. If anyone knows - drop me a message.

Add the file to Apache's main configuration file (usually named httpd.conf):

include /etc/apache2/dnsbl_block.conf

It is best to put it inside <Directory> block for the host that you are trying to protect. Certain directives in this file will require it to be inside <Directory> block.

Now restart Apache:

/etc/init.d/apache2 restart

Command may be different on your system (for example, httpd2 instead of apache).

Installing GeoIP

For further blocking and unblocking we need another Apache module, named mod_geoip. If it comes with your distribution, just install it and skip till the rest of of this section. If you cannot find it, read in.

Firsts, you need to download and compile GeoIP C library. Instructions are here. You do not have to follow "Usage" section on that page. But watch the output and notice where GeoIP.dat is located. You will need to put that path to Apache configuration file. Normally path is /usr/share/GeoIP.dat.

Next grab mod_geoip and compile it:

apxs2 -cia -lGeoIP mod_geoip.c

Note the use of "-l" (this is lower case L, not 1) option. If you accidentally skip it, module will compile but will not load. If you use SuSE, edit /etc/sysconfig/apache2 again and add mod_geoip there.

Open dnsbl_block.conf and add the following in the beginning of the file. It is important that you add to the beginning.

<IfModule mod_geoip.c>
GeoIPEnable On
GeoIPDBFile /usr/share/GeoIP.dat
</IfModule>

Reload Apache.

Blocking some countries using GeoIP

Lets block some countries. This is optional because too broad. But you may find it useful. If you want, you can

Open dnsbl_block.conf and add the following to the end of <IfModule mod_geoip.c> section.

SetEnvIf GEOIP_COUNTRY_CODE BR BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE CN BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE RU BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE TW BlockCountry

In this example we listed Brazil, China, Russia and Taiwan for blocking. Now find "Deny" instruction below in the file and add a line after it saying:

Deny from env=BlockCountry

You are just blocked several most spamming countries. Notice: visitors from these countries will still see your site. They only will not be able to post comments.

Unblocking your country using GeoIP

cbl.abuseat.org does a good job by blocking infected or spamy computers. However computers may change their IP addresses and sometimes innocent people get blocked. Generally you should unblock your country to let your own visitors post messages. This is unfortunate for USA because most of the spam comes from USA these days.

To unlock a country add this to the end of <IfModule mod_geoip.c> section.

SetEnvIf GEOIP_COUNTRY_CODE LV AllowCountry

Here we lsited Latvia for unlocking (country code LV). Now add another line below in the file where you have "Allow from" statement:

Allow from env=AllowCountry

Restart Apache.

Making your own blocking list

There are spammers, who specialize on the guestbook spamming. They are not included in cbl.abuseat.org because that list intercepts e-mail connections. Therefore you may want to catch and block guestbook spammers.

To block spammers you need to setup a small DNS server named rbldnsd. Why not to use BIND? Well, BIND is pretty slow when it comes to lots of hosts. rbldnsd is small and highly efficient exactly for handling DNSBLs. But you can also use rbldnsd with BIND, I'll explain later how.

Installing and configuring rbldnsd

Download rbldnsd package. Compile and install it:

./configure
make
make install

Now create a directory where you will have your own DNSBL zone:

mkdir /var/lib/rbldnsd

We will call this zone dnsbl.zone. Add the file named dnsbl.zone.soa.txt into that directory with the following content:

$SOA 3600 127.0.0.1 my.email.addr. 0 1h 5m 5d 600
$NS 3600 127.0.0.1

Notice my.email.addr there. This is your e-mail address with "@" replaced by ".". SInce this zone is not public, you can put whatever garbage there, just be careful to put all "." symbols exactly as in the example above.

Create a file named dnsbl.zone.txt and add the following content there:

:127.0.0.2:My DNSBL block
127.0.0.2 ; test value

There must be a new line after this line!

Now you have to create a script in /etc/init.d to start rbldnsd when your system starts. SuSE users can grab rbldnsd.tar.bz2 from the end of this article and see the script and configuration file that it uses. Other users have to create script according to their system's way. They also have to provide the following command line to rbldnsd:

-u nobody -r /var/lib/rbldnsd -b 127.0.0.1/53 -p /var/run/rbldnsd.pid dnsbl.zone:ip4tset:dnsbl.zone.soa.txt,dnsbl.zone.txt

This forces rbldnsd to listen for connections on port 53. If you have other DNS server on this host, you will need to use another port (530, for example) and setup forwarding from main DNS server. See later in this article.

Now start rbldnsd you can test that rbldnsd works:

/etc/init.d/rbldnsd start
nslookup 2.0.0.127.dnsbl.zone 127.0.0.1

This should tell you that 2.0.0.127.dnsbl.zone has IP address 127.0.0.2. If it does not, you have to see what went wrong.

Now enable rbldnsd to run at system start. SuSE users can do this:

chkconfig -a rbldnsd
chkconfig rbldnsd 35

This enables rbldnsd to run at runlevels 3 and 5.

Add your DNSBL to Apache

Open dnsbl_block.conf and add the following on a new line after the first SetEnfIfDNSBL directive:

SetEnvIfDNSBL remote_addr dnsbl.zone dnsbl_spam

Now you must configure your host to use your DNS server in addition to other name servers. If you do not do it, dnsbl.zone will not be seen. Open /etc/resolve.conf and add

nameserver 127.0.0.1

as the first entry. Now you can restart Apache again. If you find spam message, you can add it to /var/lib/rbldnsd/dnsbl.zone.txt. rbldnsd will detect file change and pick it up automatically. But you can go futher, you can catch and add spammers automatically! Read on.

Blocking spammers automatically

To block spammers automatically we will use mod_security. If it does not come with your distribution, download and install it using standard apxs2 way as you did already at least two times.

Create mod_security.conf and make sure it is mentioned in the Apache main configuration file (you did similar thing for dnsbl_block.conf already). Content of the file:

<IfModule mod_security.c>
SecFilterEngine On
SecFilterDefaultAction "deny,log,status:403"
SecFilterScanPOST On
SecAuditLog /var/log/apache2/audit_log
SecFilterSelective POST_PAYLOAD "phentermine|tramadol|viagra" "exec:/usr/bin/add2dnsbl.php"
</IfModule>

I will not explain this configuration, except for one line. If you are interested, read about mod_security configuration on its web site.

Important! SecFilterSelective and "exec..." is on one line! Wrapping happens only on this page.

The line that does magic is second from the end. It will check POST data for typical spam words and call our script to add IP address of the spammer to our DNSBL. The script is attached to the end of the article. You must place it to /usr/bin and make sure it is executable by issuing this command:

chmod +x /usr/bin/add2dnsbl.php

Next you must let Apache user to write to DNSBL file:

chown apache /var/lib/rbldnsd/dnsbl.zone.txt

If your apache runs under different user, adjust second word in the line above correspondingly.

Now restart Apache.

You are done. Now your protection is almost perfect. (Why "almost"? Because there is nothing absolutely perfect in the word).

What else can I do?

For some time you can watch for spam on your web site and if it appears, you can add more entries to mod_security to filter that spam. This will ban corresponding IP addresses.

You can clean up /var/lib/rbldnsd/dnsbl.zone.txt from time to time to exclude IP addresses from it.

Using BIND together with rbldnsd

If you run bind on the same host, you must forward your DNSBL zone from BIND to rbldnsd. Add the following to your /etc/named.conf:

zone "dnsbl.zone" IN {
 type forward;
 forward first;
 forwarders {
  127.0.0.1 port 530;
 }
}

This instructs BIND to query requests for dnsbl.zone from DNS server on port 530. Do not forget to configure rbldnsd to listen on that port (see above for details).

Conclusion

I know that this method is complicated. It took me several hours to set it all up properly. But it is definitely worth it. I get much less spam now. Try it and let me know what you think. If there are mistakes, I will correct them.

DROP.sh

add2dnsbl.php.bz2

mod_setenvdnsbl.c

rbldnsd.tar.bz2