Linux goesZen

Running your own server is challenging. One of the first steps you should take though, are measures to make your system more secure and keep unwanted guests or intruders outside. In this simple how-to on securing a linux server, I'd like to mention a few basic steps I would recommend in order to harden your server.

Let's begin: Most ISPs or hosting companies sell dedicated servers, this means they have the hardware sitting in their racks located at a datacenter somewhere in the world. The machine is your own server, leased, rented or bought, ready for complete reconfiguration matching your needs. Although you might visit them and physically work on the machine, in most cases you will remotely administer it via remote shell. The weapon of choice in this case is Secure Shell (SSH).

1. Securing SSH

There are many SSH clients out there, a good bet is the preinstalled ssh on linux and putty for the DOS/Windows world. See this comparison of clients at Wikipedia.

The first rule: always use the SSH-2 protocol. You can configure your client to do so, others use this newer and more secure protocol by default. Also force the SSH daemon to only allow protocol version 2:

$ nano /etc/ssh/sshd_config
change "Protocol xx" to "Protocol 2"

On newer systems,Protocol 2 is the default, so just check to make sure.

1.1 Prevent intruders from guessing your login/password

By default, SSH operates on IP port 22. As this is a well known fact, many spammers and scriptkiddies will try to find a random machine with this port open by automatically running attacks against this port. Depending on your IP, subnet, ISP etc. this can happen quite often (although there are measures to prevent others hammering your server, most ISPs expose your machine unprotected to the outside). So a good idea is to make SSH as secure as possible. A simple measure is to choose a good secret. A cryptic username in combination with a good password (longer than 12 letters, as most password brute force software currently tries a maximum of 12 positions) that contains letters, numbers and special chars.

Also: use the last command to see who was last logged in and from what IP.The most extreme form of an unguessable password is a password that is so long, you'll need a file to store it in: this method uses an RSA key as password. You can configure SSH to only accept this kind of passwords, thus effectively locking others out. If you are interested, here and here is how to do it.

An additional measure is to permanently scan the logs and see if someone is systematically trying to get in. There are tools that do so in form of a daemon. Fail2ban is such a tool. It looks for failed-login patterns and then bans the calling IP for a while. This is far more effective than manually collecting a list of IPs to ban.

See how to install and configure fail2ban on Debian Squeeze and other *nix distros here.

1.2 Change the SSH port

If you alone or a closed group use SSH on the machine, you can change the port of the SSH daemon, preferably to a number higher than 1024. Do so like this

$ nano /etc/ssh/sshd_config
change "Port 22" to "Port 14444"

1.3 Allow only certain users in

Granting direct access for "root" via SSH is a bad idea. So create a user for an initial SSH login. Only this user is then allowed in (the username is the first thing a potential intruder would have to guess, so chose a complex username!). Configure SSH to do so:

$ nano /etc/ssh/sshd_config

change "PermitRootLogin yes" to "PermitRootLogin no"
insert "AllowUsers userxyz"

or even a user + IP combination (here: local net only):

AllowUsers userxyz at 192.168.1.0/254

You can further restrict access to certain groups with:

          AllowGroups wheel admin 

(Don't forget to restart ssh after changes: /etc/init.d/ssh restart

If you have SSH configured like this, first login with userxxxxxxx. Then you are free to su to root with the additional root password at any time.

1.4 Restrict access to certain IPs

Using the standard linux tool iptables you can very effectively block IP ranges, and if you do something wrong - yourself! So be careful when using iptables. Read about that here.

One more hint: after you have spend hours on configuring iptables, its a good idea to backup your config. Backup and restore with these commands:

iptables-save > /var/iptables_regeln
iptables-restore -c /var/iptables_regeln

A somewhat more comfortable approach would be to allow requests only from a certain country. So you, for example in the Netherlands, can get in while everyone else, including the script kiddies from China are left outside. This aproach requires support for country resolution within your system, and be sure that your ISP/your domain (from which you are accessing the internet) ends in a country code like .nl (some ISPs use the .net or .com domain, check this beforehand!). Country based blocking requires that ssh was compiled with the "--with-tcp-wrappers" flag! Here is how to find that out. If so, edit your /etc/hosts.allow and add:

sshd : .nl : allow
sshd : ALL : deny

Further reading here, here, here.

1.5 Restrict access to a specific IP

In case you have multiple networks adapters installed, it might be a good idea to restrict ssh to listen only on one IP. Then add:

ListenAddress 192.168.0.1, for example.

1.6 Port knocking

A newer and interesting approach is port knocking. This keeps the SSH port closed until you do a series of predefined "knocks" on other ports, a technique resembling that of someone knocking on a door with a secret knocking-code . You can read about this here and here, but I currently do not use it as I think it is a burden.

2. Turn off unused services

After a standard install, some distros have a lot of unused services running. You should decide which of these services are need and then deactivate the rest. One of your mantras should be: every aspect that makes the system more complicated might open another attack vector!

Use the command netstat to get an overview of open ports and running services. On a standard and well secured system, the only port that should carry a "LISTEN" is port 22, our SSH port.

netstat -l

An additional tool to have a look at the open-port situation on your system is nmap. It's not always installed by default so run "apt-get install nmap". Then, executing nmap <IP of your server> should give you an overview of open ports.

netstat -tulpen

will give you a detailed listing, also showing the related process. If a process is only listening on 127.0.0.1, you don't have to worry about it, only processes listening to IP 0.0.0.0 (all local IP adresses) and the assigned ip, 192.168.x.x or similar (assigned IP) are relaevant.
A good way to get rid of an open port and its daemon is to completely uninstall the related packet. Remember our mantra! To do so: apt-get remove --purge <paketname>.

A good alternative to nmap is lsof. Install if needed and use like this: lsof -i. Another tool to dig further is tcpdump, for example to get a look at broadcast traffic: tcpdump -n arp. Some advanced measures if you are fighting with ICMP echo requests are described here, but: know what you do before you do it!

Some daemons use the internet super server inetd. If a service is running on this vehicle, you can deactivate it this way: update-inetd -remove <service>. Remember to restart the inetd after that with /etc/init.d/inetd restart.

In order to track down what service is responsible for a specific port, dpkg -l , which lists installed packages, might be helpful. A first example is to do this:

apt-get --purge remove pidentd (port 113 auth/ident daemon)
apt-get --purge remove portmap

apt-get --purge remove exim4-daemon-light (ua port smtp, which debian etch has by default, remove when not running email)

Further reading here.

Please note: You will have a hard time removing all remains of exim4, as a UNIX system needs some form of MTA (mail transport agent) to work, some processes like cron use it to communicate with each other. So either wait until you get another MTA like Postfix on your system or use an offline, non-daemon, MTA like esmtp.

3. Remove shell for low-priority users

A potential intruder might use a common but unused user to gain access to the system. To add another hurdle for this attack, we can delete the shell on certain users, users that normally do not need to log in, like nobody, cron etc:

$ nano /etc/passwd
nobody:x:65534:65534:nobody:/nonexistent:/dev/null

This sets the default shell for user nobody to /dev/null, effectively blocking him.

4. Keep your system up to date

You should regularly update your system and all installed packets to close discovered security holes. Do so with

apt-get update
apt-get upgrade

You could automate this task (there are tools to do so, or script your own), but there is an ongoing debate about wheather this itself imposes a security leak or not. So revert to doing it manually.

5. See if Apache is okay and secure (when running a webserver)

To find out how to install Apache (Apache2) on your syste, refer to Installing Apache2 on Debian Sqzeeze.
But keep in mind that the Apache as well can be a security leak: To manually scan the logs for intruders/attackers do this:

$ cat /var/log/apache/access.log | grep wget
or
$ cat /var/log/apache2/access.log | grep wget

it looks for someone using wget against your server. This scan can be also perromed for other suspicious proggies and URLs. Examples are someone trying to exploit nonexisting or commonly known weak .php pages, holes in pgpmyadmin are common (so move it to a non-standard directory), also someone might try to sneak in code snippets via parameters on URLs or via CGI-Scripts you are running (so be sure to read about potential problems with your scripting language of choice).

5.1 Chroot apache

Common practice is to change the user apache operates under or at least change the user scripts or php is executed under. Read about that here and here.

6. Bugging binaries on your system

I have never done it, but among administrators of firewall systems, the bugging of certain apps is common practice. The technique is similar to a honeypot: in case someone gains access to your system, it is quite probable that the intruder will use certain tools. You expected that and compiledyour own bugged version of this tool with a built-in trap. So, if the intruder uses the bugged app, you are alarmed via a certain mechanism and know that someone sneaked in.

6.1 Monitoring file alterations done to critical binaries

Using Gamin allows you to track if someone tampers with your files, directories and/or binaries.

7. Running rootkit detection tools

A rootkit is a set of tools a hacker sneaks in in order to gain root priviledges. Not all hackers have the brains or time to build these rootkits from scratch, so they tend to rely on readily available ones. Similar to a virus scanner, there are tools to help us detect these known rootkits on our system, bot are available via apt-get:

chkrootkit is a quite sensitive tool but it often leads to false-positives, particularly when DHCP is running.

rkhunter is another tool. Run it with rkhunter --checkall --skip-keypress

8. Securing the MySQL server

First run the security wizard script shipped with MySQL. It will remove unneeded users, test tables, etc. It is recommended that you answer "yes" to all questions. If you are prompted to reload the privilege tables, select "yes.":

mysql_secure_installation

Then go into the MySQL conf file to check some things

nano /etc/mysql/my.cnf

Make sure that your MySQL daemon is only accessible from the local machine (assuming that this server is acting as the database server for local applications). The bind address should be 127.0.0.1 (commonly localhost).

9. Further topics

which we declare as W.I.P. here are: remove sudo/root, file system permission rules, etc. You are encouraged to do further research on these topics on your own.

Disclaimer

Security gained from using the above information cannot be guaranteed and steps are never complete. If you use the above information for any purpose, you do so at your own risk.