Home » tutorials

How to Stop DDOS on Server

6 February 2009 24 views No Comment

Because a null-route to an ip is not a solution, it is a kludge.

1) it is based on real attacks.
2) there is not anything of theory, single part practice.

A—>Detecting the attack

1) using the command netstat

netstat -an | grep :80 | sort

netstat -n -p | grep SYN_REC | awk '{print $5}' | awk -F: '{print $1}'

netstat -n -p|grep SYN_REC | wc -l

netstat -lpn|grep :80 |awk '{print $5}'|sort

netstat -an | grep :80 | awk '{ print $5 }' | awk -F: '{ print $1 }' | sort | uniq -c | sort -n

tcp 0 0 192.168.0.3:80 192.168.0.5:60808 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60761 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60876 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60946 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60763 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60955 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60765 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60961 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60923 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61336 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61011 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60911 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60758 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60828 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61114 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61074 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60826 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60959 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60900 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60940 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60920 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60825 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60945 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60913 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61009 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60755 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60904 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61583 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60910 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60915 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60827 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61458 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60908 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61007 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60927 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60951 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60942 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61113 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60909 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60822 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60894 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60952 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60928 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60936 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60906 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61466 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60919 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60914 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60926 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60939 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60931 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60831 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60823 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60954 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60916 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60963 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60947 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61006 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60933 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60950 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60895 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60917 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61480 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60935 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60960 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60767 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60918 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60821 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61077 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60905 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61517 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60893 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60953 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60903 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61439 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61337 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61545 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61299 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61010 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60930 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60744 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60929 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60754 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61008 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61116 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60811 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60807 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60938 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60764 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60873 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60817 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61550 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60748 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60956 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60753 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61115 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60741 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61075 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60948 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60829 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60943 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61338 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60762 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60824 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60830 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61535 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60898 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60815 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60962 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60957 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60944 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60921 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60759 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60897 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61518 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60958 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60922 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60937 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60875 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60766 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60751 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60768 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60743 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:61076 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60912 SYN_RECV
tcp 0 0 192.168.0.3:80 192.168.0.5:60816 SYN_RECV

example of SYN Attack to the Apache.

2) looking at the server-status of the Apache

If we look at the server-status of the Apache we will see connections in state” Reading” (” R” Reading Request).

The problem is that when the number of connections” Reading” full the” MaxClients” of the Apache it doesn’t accept new petitions, for that that the new clients, although they are legitimate, they won’t be accepted.

We can increase the value of the” MaxClients” so that is not filled the line of petitions and accept all the clients, be attackers or no.

Another good measure is to lower the value of the” Timeout” of the Apache so that the petitions” Reading” they are” killed” quickly, before the MaxClients can be filled to its end.

3) looking at the logs of the mod_evasive

If it is at once a very distributed DDoS we will notice that many different ip’s DoSean the Apache.

4) looking at the logs of the syslog (of Kernel).

Lines to look:

possible SYN flooding on port 80. Sending cookies.

“Sending Cookies” if we have it activated in the /etc/sysctl.conf

# Enable TCP SYN Cookie Protection

net.ipv4.tcp_syncookies = 1

It is sometimes better to disable it:

net.ipv4.tcp_syncookies = 0

This way we can see the ip’s of the attack:

May 17 14:13:24 lan kernel: ip_conntrack: table full, dropping packet.

Full Table. We have a problem because we won’t admit more connections although they are legitimate.

We can increase the value of this chart if our net gives for more.

Directly:

echo "65535" > /proc/sys/net/ipv4/ip_conntrack_max

So that the value is saved and don’t get lost when restarting, we should add it in the sysctl.conf

net.ipv4.ip_conntrack_max = 65535

remembers to restart the net to apply the changes in the / proc (service network restart).

Martian packages:

Aug 31 12:41:29 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 12:45:07 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 12:52:57 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 12:58:55 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:08:12 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:12:03 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:34:38 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:37:38 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:52:42 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:56:18 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 13:59:54 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:13:32 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:38:08 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:43:42 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:50:05 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:51:05 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 14:57:58 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 15:05:27 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 15:06:14 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0
Aug 31 15:09:08 lan kernel: martian source 192.168.0.10 from 0.0.0.0, on dev eth0

They are unexpected packages that arrive for a road for which you/they cannot arrive it indicates some problem of audacity (cracker).

Using packages like these can be attacked remote vulnerabilities in stacks TCP/IP

5) looking at the graphs of the MRTG, RRDtool

If you see that the traffic inbound ascends up to the 100mbps it is that they are wish you hehehe

B) to Try to stop the attack

1) - mod_evasive
Web Oficial:
http://www.nuclearelephant.com/projects/mod_evasive/

We consider that 50 connections per second to 2 pages it is enough I motivate like to block that ip:
The same as the previous one but with 50 petitions in one second to 1 single page:
If we want to block the ips that floodean, we can use the iptables:

DOSSystemCommand "sudo -u root -c '/sbin/iptables -A INPUT -s %s -j DROP"

To remember to look at the syslog for if there are possible positive reinforcements (ip’s that didn’t make flood).

To avoid false positive:
Important:
So that the mod_evasive works correctly you will modify the one:

MaxRequestsPerChild 0

To put a high but never limitless value (0).

MaxRequestsPerChild 10000

Config example:
http://www.eth0.us/mod_evasive

2 - mod_security
The only problem of the mod_security is that we need an argument at least to detect the attack.

In the example we use in http_referer and the User Agent to detect the DDoS:

3- tcplimit, ipdrop, ipblock
Using dynamic firewalls

4 - optimizing and assuring the net with the sysctl.conf

cat /proc/sys/net/ipv4/tcp_syncookies

# Enable IP spoofing protection, turn on Source Address Verification

net.ipv4.conf.all.rp_filter = 1

# Enable TCP SYN Cookie Protection

net.ipv4.tcp_syncookies = 1

# Enable ignoring broadcasts request

net.ipv4.icmp_echo_ignore_broadcasts = 1

1). Activate SynCookies protection

It works by sending out ’syncookies’ when the
syn backlog queue of a socket overflows.
2). Disable source routing
3). Reverse Path Filtering

Reject incoming packets if their source address doesn’t match
the network interface that they’re arriving on
4). Log RP filter dropped packets (martians)
5). Maximal number of remembered connection requests

=> /sbin/sysctl -w net.ipv4.tcp_max_syn_backlog=256

). How may times to retry before killing TCP connection

(default 7 on most systems)

=> /sbin/sysctl -w net.ipv4.tcp_orphan_retries=4

7). Number of SYN packets the kernel will send before giving up

=> /sbin/sysctl -w net.ipv4.tcp_syn_retries=5

8). Disable broadcast icmp reply

=> /sbin/sysctl -w net.ipv4.icmp_echo_ignore_broadcasts=1

9). Ignore Bogus icmp packets

=> /sbin/sysctl -w net.ipv4.icmp_ignore_bogus_error_responses=1

10). Disable ICMP redirect
11). Disable timestamps
12). Reduce DOS ability by reducing timeouts
It lists of all the variables of the TCP:
http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html

lists of Variables of the / proc/sys/net/ipv4 / * (with default varlores and explanations)
http://ipsysctl-tutorial.frozentux.net/other/ip-sysctl.txt

More examples of complete configuration of the sysctl.conf in the references of the document

5 - APF Firewall with the module anti-ddos

wget http://www.rfxnetworks.com/downloads/apf-current.tar.gz
tar xvzf apf-current.tar.gz
cd apf-0.9.6-1/
./install.sh

service apf start
/usr/local/sbin/apf -s

Configuration file

/etc/apf/conf.apf

After making the tests leave:

DEVEL_MODE="0"

If it leaves us a similar error to this:

apf(9413): unable to load iptables module (ip_tables), aborting.

We change this:

SET_MONOKERN="1"

Ports that we want to open up (inbound)

IG_TCP_CPORTS="21,22,25,53,80,110"

If you want to block the whole exit traffic puts it in 1 (outbound)

EGF="0"

If we want to use the module antddos to put at 1:

USE_AD="0"

Log:
/var/log/apf_log

To see the packages:

LOG_DROP="1"

will keep it in the syslog, example:
remembers that to use the antidos you should add the cron job:

*/8 * * * * root /etc/apf/ad/antidos -a >> /dev/null 2>&1

http://www.r-fx.org/apf/README.antidos

KISS My Firewall is an alternative.

Script PHP
http://www.prism-hosting.com/AntiDoS

6 - to stop the botnet
ZmbScap - Zombie Scapper - Stoopt DDoS Programs
http://www.metaeye.org/projects/zmbscap/

Tracking Botnets - Bot-Commands
http://www.honeynet.org/papers/bots/botnet-commands.html

Tracking Botnets
http://www.honeynet.org/papers/bots/

Tracking Botnets - DDoS-attacks
http://www.honeynet.org/papers/bots/botnet-ddos.html

Phatbot Trojan Analysis
http://www.lurhq.com/phatbot.html

F-Bot by f-secure - it Eliminates the Agobot and all their variants
http://www.f-secure.com/tools/f-bot.zip

Nepenthes - Autoinfecarse without danger to analyze
http://nepenthes.mwcollect.org/

honeytrap – trap attacks against tcp services
http://honeytrap.sourceforge.net/

7 - using rules of the iptables
8 - using the mod_throttle
http://www.snert.com/Software/mod_throttle/

Others:
Mod_Throttle, mod_bandwidth, mod_iplimit, mod_tsunami, mod_limitipconn.c

ForApache 2:
mod_cband

<IfModule mod_throttle.c>
ThrottlePolicy Volume 10G 30d
</IfModule>
<Location /throttle-me>
SetHandler throttle-me
</Location>

http://www.webhostgear.com/160.html

C) References
- Syctl.conf Hardening
http://www.eth0.us/sysctl

- Ipsysctl tutorial 1.0.4
http://ipsysctl-tutorial.frozentux.net/ipsysctl-tutorial.html

- Hardening the TCP/IP stack to SYN attacks
http://www.securityfocus.com/infocus/1729

- DDOS and SYN_Recv Attacks And some SOlutions
http://www.vbulletin.com/forum/showthread.php?t=126699

- Distributed Reflection Denial of Service
http://www.grc.com/dos/drdos.htm

- Dynamic iptables firewalls
http://www-128.ibm.com/developerworks/library/l-fw/

- Preventing DDoS Attacks
http://www.linuxsecurity.com/content/view/121960/49/

- Distributed Denial of Service (DDoS) Attacks/tools
http://staff.washington.edu/dittrich/misc/ddos/

- Linux firewall rulesets and snippets of rule sets
http://www.gotroot.com/tiki-index.php?page=Linux%20Firewall%20rules

Share and Enjoy:
(No Ratings Yet)
 Loading ...

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.