Category Archives: Programming

Material related to computer programming.

Watching the web-logs on all of my servers in real time

Posted on by
Reply

I have a computer sitting on my desk that is always on (it’s my file server) and it has a monitor attached which is almost never in use (I ssh to that server if I want to do things so it’s hardly ever logged in).

I thought it would be cool if on that monitor the web-logs from all of the systems I manage were shown so I could keep an eye on things and maybe learn a thing or two about my web-sites and how people are using them.

So the first thing I did was write a script to grab any given web log:

root@orac:~# cat /root/get-web-log.sh
#!/bin/bash
echo Starting download of $3...
while : ; do
  su -c "ssh $1 tail -f /var/log/apache2/$2 < /dev/null" jj5 \
    | tee -a /var/log/web.log \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Googlebot.2.1. .http...www.google.com.bot.html." \
    | grep --line-buffered -v "Baiduspider...http...www.baidu.com.search.spider.htm." \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Baiduspider.2.0. .http...www.baidu.com.search.spider.html." \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Exabot.3.0. .http...www.exabot.com.go.robot." \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. YandexBot.3.0. .http...yandex.com.bots." \
    > /var/log/web/$3
  sleep 60
  echo; echo; echo Restarting download of $3...; echo; echo;
done

Then I wrote a series of scripts which call the get-web-log.sh script for specific web-sites on specific servers, e.g.:

root@orac:~# cat /root/web-log/get-jsphp.co
#!/bin/bash
/root/get-web-log.sh honesty www.jsphp.co-access.log jsphp.co
exit

Then I wrote a main script, rather unoriginally called info.sh, that kicks off the web logs downloads and then monitors their progress as they come through:

root@orac:~# cat /root/info.sh
#!/bin/bash

# disable the screensaver
setterm -blank 0 -powersave off -powerdown 0

# start downloading the web-logs
cd /root/web-log
./get-jsphp.co &
sleep 1
#...all the other downloaders, one for each site

# watch the web-logs
cd /var/log/web
tail -f *

# stop downloading the web-logs
kill %1
#...all the other kills, one for each downloader

exit

Then I edited /etc/init/tty1.conf so that on tty1, instead of having a login console, I automatically ran my info.sh script:

root@orac:~# cat /etc/init/tty1.conf
# tty1 - getty
#
# This service maintains a getty on tty1 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
#exec /sbin/getty -8 38400 tty1
exec /root/info.sh < /dev/tty1 > /dev/tty1 2>&1

And that was it. The only trick was that I needed to disable the screen saver (as shown in the info.sh script) so that the screen didn’t constantly blank.

And now I can watch the web activity on all of my sites in real time.

Slowing down fail2ban

Posted on by
39

I had some messages from my fail2ban log like this:

 2012-02-03 00:59:33,810 fail2ban.actions.action: ERROR  iptables -N fail2ban-apache
 2012-02-03 00:59:33,838 fail2ban.actions.action: ERROR  iptables -N fail2ban-apache-overflows
 2012-02-03 03:42:49,355 fail2ban.actions.action: ERROR  iptables -D INPUT -p tcp -m multiport --dports http,https -j fail2ban-apache-overflows
 2012-02-03 03:43:04,998 fail2ban.actions.action: ERROR  iptables -N fail2ban-ssh-ddos
 2012-02-03 03:43:05,035 fail2ban.actions.action: ERROR  iptables -N fail2ban-apache-overflows
 2012-02-03 07:13:04,720 fail2ban.actions.action: ERROR  iptables -D INPUT -p tcp -m multiport --dports http,https -j fail2ban-apache-overflows
 2012-02-03 07:13:20,154 fail2ban.actions.action: ERROR  iptables -N fail2ban-ssh-ddos

I read over here about a workaround that goes like this:

Edit /usr/bin/fail2ban-client and add a call to time.sleep to mitigate a race condition:

def __processCmd(self, cmd, showRet = True):
    beautifier = Beautifier()
    for c in cmd:
        time.sleep(0.1)
        beautifier.setInputCmd(c)

Ignoring emails with old dates

Posted on by
38

I was reading about Postfix’s cleanup facility which supports header_checks which can be specified in a regexp: table. And it inspired me to come up with this header_filter_map file:

/^Date: .* [JFMASOND][aepuco][nbrynlgptvc] 1/ REJECT
/^Date: .* [JFMASOND][aepuco][nbrynlgptvc] 200/ REJECT
/^Date: .* [JFMASOND][aepuco][nbrynlgptvc] 2010/ REJECT
/^Date: .* [JFMASOND][aepuco][nbrynlgptvc] 2011/ REJECT
/^Date: .* Jan 2012/ REJECT
/^Date: .* Feb 2011/ REJECT
/^Date: .* Mar 2011/ REJECT
/^Date: .* Apr 2011/ REJECT
/^Date: .* May 2011/ REJECT
/^Date: .* Jun 2011/ REJECT
/^Date: .* Jul 2011/ REJECT
/^Date: .* Aug 2011/ REJECT
/^Date: .* Sep 2011/ REJECT
/^Date: .* Oct 2011/ REJECT
/^Date: .* Nov 2011/ REJECT
/^Date: .* Dec 2011/ REJECT

Which I applied in Postfix by adding the following line to /etc/postfix/main.cf:

header_checks = regexp:/etc/postfix/header_filter_map

It remains to be seen if what I’ve done will work, and at the moment this is a bit of a pain because I have to manually update the header_filter_map file every month, but the general idea is that if the regexp matches a date too far in the past then the message is rejected. Hopefully then those spammers who have messages turning up in my history will be gone.

Making directories executable and files read and writeable

Posted on by
Reply

I had a problem with my rsync backups. The problem was that the first time I ran it everything worked fine. The second time it ran (and all subsequent times) I got back the phone book of error messages, because the first time I’d run rsync it had copied in a whole heap of read-only files, and then when I ran it again it wasn’t able to overwrite those read-only files. At least I think that was what was happening. So I added the following to my backup script:

  find . -type d -exec chmod u+x {} \;
  if [ "$?" -ne "0" ]; then
    echo "Cannot chmod directories in '$PWD'.";
    exit 1;
  fi
  find . -type f -exec chmod u+rw {} \;
  if [ "$?" -ne "0" ]; then
    echo "Cannot chmod files in '$PWD'.";
    exit 1;
  fi

This code runs after rsync and processes the files and directories that have been synchronised. That is, it processes the copy of the data, not the data I copied from.

For the copy of the data I want to make sure that the owner of the files can read and write them and that the owner of the directories can execute them. So that’s what the above code does.