latest DoS strategy

hi all - over the past six months the server attacks kept getting worse and worse, even shutting down my VPS.

the two best approaches have been either to rename the wordpress login, and/or simply put up an .htaccess line that makes people log in.

this protects wordpress just fine (along with captcha and limit-login-lockdown) but apache was left unprotected.

previously my strategy for apache was to look every two minutes for cgi-php processes, and restart apache when i see more than 20 of them. that worked fine, but it seemed like a bit of overkill. so now i scan for the cgi-php processes every minute, and specifically only kill those processes if there are too many for a given owner.

my bash script is below for anybody who might want it. i am using virtualmin (of course!). it even emails me when it had to kill off a group of processes and gives me the offending IP numbers, complete with the URL where i can rat them out.

Code:
#!  /usr/bin/bash  -w

#       processLevelCgiTest.bsh

MAXALLOWED=${1};
DATE=$(date);

# ps -ef  | grep  php-cgi   example:
#511      17669 12623  0 08:59 ?        00:00:19 /usr/bin/php-cgi
#551      30882 12623  0 10:29 ?        00:00:00 /usr/bin/php-cgi
#551      30883 12623  0 10:29 ?        00:00:00 /usr/bin/php-cgi
#547      31910 12623  0 10:36 ?        00:00:00 /usr/bin/php-cgi


mailString='';

        # extract the first three digits, do the uniq count, and replace the spaces with dashes [otherwise the 'for' loop treats spaces like line breaks ]
for record in $( ps -ef | grep  php-cgi  | sed -e 's/ * / /g;' | cut -d' ' -f1 | uniq --count | sed -e 's/ * /-/g;s/^-//;' ); do
                ## example 2-511
        count=$(echo $record | sed -e 's/-.*$//;' );            ## extract out the count (the first part)
        etcPasswdIdNbr=$(echo $record | sed -e 's/^.*-//;' );   ## extract out the etcPasswdIdNbr (the last part)
        if  [[ $count -ge $MAXALLOWED ]]
        then
                        ## go back to the ps-ef and extract out the process numbers for this etcPasswdIdNbr
                for procNbrKill in $( ps -ef | grep  php-cgi  |  grep  "^$etcPasswdIdNbr"  | sed -e 's/ * / /;' | cut -d' ' -f2  ; ); do
                        echo killing process: $procNbrKill ;
                done
                resultingString='';
                for etcPasswdName in $(cat /etc/passwd | cut -d':' -f1,3 | grep ":$etcPasswdIdNbr$" | sed 's/:.*$//;' ); do
                    LOGFILE=/var/log/virtualmin/${etcPasswdName}_access_log             ;
                    echo 'testing for ' $LOGFILE        ;
                    if  [ -e $LOGFILE ]
                    then
                        echo "killing for : $etcPasswdName";
                        resultingString+=" -- $etcPasswdName" ;
                        for countAndIpNbr in                                                            \
                                $(/usr/bin/tail  -500  $LOGFILE                                         \
                                |  grep   '"GET /wp-login.php'                                          \
                                |  egrep "$(date -d '-0 min' +"%d/%b/%Y:%H:%M")|$(date -d '-1 min' +"%d/%b/%Y:%H:%M")|$(date -d '-2 min' +"%d/%b/%Y:%H:%M")|$(date -d '-3 min' +"%d/%b/%Y:%H:%M")|$(date -d '-4 min' +"%d/%b/%Y:%H:%M")" \
                                |  cut -f1 -d' '                                                        \
                                |  uniq -c                                                              \
                                |  sed -e 's/^ * //;s/ * /-/g'                                          \
                                ); do
                                echo "Found: " $countAndIpNbr;
                                        # format 1-88.235.227.12
                                $hackCount=$(echo $countAndIpNbr | /bin/sed -e 's/-.*$//';)             ;
                                echo '$hackCount: ' $hackCount
                                $ipNbr=$(echo $countAndIpNbr | /bin/sed -e 's/^.*-//';)                 ;
                                echo "Writing IP number " $ipNbr " to csf-deny file!";
                                csfCommand=$(echo $ipNbr '# domain: ' $domain 'attempts:' $hackCount $(date) $(basename "$0") '!!!'     );
                                echo $csfCommand                                                                        ;
                                echo $csfCommand  >>/etc/csf/csf.deny                                                   ;
                                echo mailString;
                                mailString+= $csfCommand " http://whatismyipaddress.com/ip/${ipNbr}"                    ;
                                echo mailString;
                        done
                    else
                        echo $LOGFILE 'does not exist.....' ;
                    fi
                done
                mailString+="\n\rKilling processes for $resultingString - we found $count and only $MAXALLOWED are allowed."    ;
                #tail  -500  /var/log/virtualmin/${resultingString}_access_log  |  grep   '"GET /wp-login.php'  |  egrep "$(date  --date '-34 min' +"%d/%b/%Y:%H:%M")"  |  cut -f1 -d' '


        #else
                #echo $count  ' --  count is low....';
        fi
done

if  [ -n "$mailString" ]
then
                echo $mailString | mail -r info@edwardsmarkf.info -s 'killing processes ! ' mark@edwardsmark.com        ;
fi

#
i welcome any opinions or comments, so long as they are positive.
 

hansenlaw

New Member
I'm just curious, have you made any enemies? I've been running 2 blogs for over a year and have been quite successful. I've never had an issue with them, do you write controversial articles or are the attacks just random?
 
Lee Atwater, the former head of the Republican National Committee once explained the attacks from his democrat rivals by saying "I wear it like a badge of honor".

What blog are you using? the attacks against the wordpress wp-login.php file are very well documented. assuming you are running WP, have you looked in the apache logs for hits against your wp-login file? you could do something like:

grep 'wp-login.php' /name/of/your/logfile.log | wc ;

i have about a dozen WP sites, but one of them is multi-user WP with about 50 accounts. maybe thats why i attract the attention. either that, or its my good looks.
 

Jonesy

New Member
Mark, like hansenlaw, I have never had any of these attacks. I have over 10 WP sites. What happens in these 'server attacks'?
 
my situation may be different than yours because i am using the virtualmin control panel and each process spawns off a cgi call. so i may end up with many processes in memory.

i have this virtualmin setting: "CGI wrapper (run as virtual server owner)"

when i begin to have serious problems is when i have lots of php-cgi processes in memory, and my server overloads. here is an example of what i see:

518 17816 8509 0 May13 ? 00:00:00 /usr/bin/php-cgi
512 19950 23854 0 May12 ? 00:00:30 /usr/bin/php-cgi
514 20956 8509 0 09:02 ? 00:00:02 /usr/bin/php-cgi

i still suspect that if the WP people look in their access logs, they will see many attempts to reach the wp-login.php file.

maybe WP and CGI dont play well together ?
 

Jonesy

New Member
That's very interesting. I will try to take a look at some of my log files tonight and see if I have this as well. Could any of these hits be from spiders? I remember having a plugin once that logged all my hits and was amazed at how many of them were spiders (or at least what the plug-in was labeling as spiders).
 
not sure what spiders mean in this context, but i have seen upwards of 20k hits in just a few minutes.

the most simple strategy to deal with this for protecting wordpress is probably to edit dot-access and password-protect the wp-login.php file -- but that didnt stop all the cgi processes overloading memory. thats why i wanted a script that monitors how many php-cgi processes i had at any given time.

of course, the problem is i cant really debug the script until the attack happens ! about all i can do is to fill the bash script with echo commands and carefully review the log after the next attack.
 

Jonesy

New Member
Well I guess I consider myself lucky. I've checked several log files from various sites and I have not had this attack - yet. Guess it's time to get a little proactive. Thanks for the heads up.
 
here is a list of my favorite wordpress security plugins:

si-captcha -- the easiest one to use i have found
math-captcha -- redundant, but at least it keeps your math skills current
limit-login-lockdown (either of these two)
limit-login-attempts
login-delay -- adds a one second delay to logging in
rename-wp-login -- new one, lets you rename the wp-login.php file

to consider:
login-security-solution -- have not tried it yet but it gets high marks

some have suggested renaming the 'admin' username. there are quite a few cases where you cant do this with third-party paid plugins i have found.

and of course my original trick of just entering something to block the wp-login.php file in the dot-htaccess file.

keeping the hackers out of WP is pretty easy. but dealing with all the extra spawned processes has been my headache!
 

coltonaron

New Member
The information which you shared was worthy. I am searching for this kind of updated info about the disk operating system strategy. I want to implement this one to some of my files. I hope, I get positive results.

DDoS Protected VPS
 
Top