Peter's Blog

After a couple of weeks of neglect I ran an update on the packages on my debian server. It gave me a few surprises:

• it upgraded mysql from 4.1 to version 5
• it uninstalled the webmin mysql module
• it uninstalled my vim-python version 6.3 and installed plain vim version 6.4

The new version of mysql seems to be running smoothly (or you wouldn't be reading this). Must see if it supports cascade delete, my all-time favourite sybase sqlanywhere feature.

I manually installed vim-python using dselect and was happy. Installer glitch. I haven't addressed webmin-mysql yet, it wasn't obvious from what dselect showed me whether what it had would work with mysql 5. Maybe it doesn't say because it doesn't matter? I find webmin-mysql can do all the mysql database administration I need (create database, add users, set permissions, view table etc) with a much simpler interface than phpmyadmin, which is slightly ott.

I think vim 6.4 is a bug fix release. Upgrading on windows involves recompiling it with python support, not a big deal but it takes time that could be spent doing other things. Sometimes I contemplate hosting vim-python for windows downloads as a public service. I'll contemplate it some more.

Was contemplating setting up a vpn between my debian dedicated server and my home windows pc, but how to set it up? Thought about openvpn and found this lengthy article which looked like far too much hastle.

Then I came across mention of hamachi, an easy to set up vpn system. It is closed source but still free. It is a unique system that uses a special hack to get through firewalls in the same way as voice-over-ip.

I installed it on the server first using these instructions which are pretty straightforward. I then installed the windows client which was even easier to install, it starts a wizard up automatically.

Once connected it assigns both ends of the network static ip addresses and the windows client displays the ip addresses of both ends. From windows, just ping the ip address of the server and it worked. Add the windows ip address to the servers webmin access list and I could access webmin from the pc. Hibernate the laptop and unhibernate and it reconnects automatically.

Conclusion: like it says on the box, easy vpn. Now do I trust a closed source system that is begging to be abused by hackers?

Update: should mention that this worked despite the firewalls in my di624 router, Windows XP noddy firewall and the iptables firewall on the server. I don't think it will work if the firewall blocks outgoing UDP packets.

I have realised that I have a full peer-peer tcp/ip network: no more fiddling with ssh tunnels. The server can even push stuff to the client, I'm not tied down to sftp'ing from the server. Next step is to set up samba on the server: I wouldn't want this open to the internet and it can only be tunnelled through ssh if you disable file and printer sharing in windows.

The server could send a WOL packet to my laptop to turn it on and an xml-rpc server on the laptop can do just about anything: record tv, stream webcam, turn the lights on... This was possible before but now it can all be done in an even cooler way.

Update 2: next day after writing this the Hamachi servers went down, taking my vpn, and however many other hamachi vpn's, down with it. Looking at their forum, their servers do seem a slight liability, being subject to DOS attacks and whatnot.

The linux tools don't give much in the line of diagnostic information: if it does ever time out it just says 'Failed', no clue why. Maybe good for security to give no clues but not good for debugging. Had troubles getting three computers on the same network, getting three connected happily at the same time: one or other would be unable to ping it's peers.

In conclusion, I've given up on this, when it works it is nice but I want something that is more reliable and has proven security.

Playing with trac I had to set up apache login authentication to set up access permissions. This is good, I now know how to password protect personal areas of the site (not that personal).

I've used auth-digest as it's supposed to be more secure than basic authentication. It may have problems with some versions of internet explorer: no, lets rephrase that, it is better at keeping the proles out. Here is how I did it for my debian system:

• Enable the Digest Authentication module in apache2:

  sudo a2enmod
  auth_digest<cr>
  apache2ctl restart
  

• Create a digest file:

  mkdir /somewhere/to/keep/it
  htdigest -c /somewhere/to/keep/it/auth.htdigest Area51 me
  

  where me is my user id. You will be prompted for password for the user.
• Edit site configuration file: in the case of my trac url, I've protected it thusly:

  ScriptAlias /trac /usr/share/trac/cgi-bin/trac.cgi
  <Location "/trac">
      AuthType Digest
      AuthName "Area51"
      AuthDigestDomain /var/www/Trac http://www.somewhere.org/Trac
      AuthDigestFile /somewhere/to/keep/it/auth.htdigest
      Require valid-user
      SetEnv TRAC_ENV "/var/www/Trac"
  </Location>
  

Now I have to log in to get into www.somewhere.com/Trac.

Trying to run subversion on ubuntu, I kept getting the error:

svn: error: cannot set LC_ALL locale
svn: error: environment variable LANG is en_GB.UTF-8
svn: error: please check that your locale name is correct
svn: Connection closed unexpectedly

Googling seems to imply that this one is a bit of a mystery, svn doesn't like the LANG variable and is happier if it is not set. I found that LANG was being set in /etc/environment on my ubuntu box and that this file didn't exist on my debian server where LANG was not defined.

I commented it out and reconnected and joy ensued.

Running

sudo dpkg-reconfigure locales

does not break it again.

I did a google for LANG and found k.d.lang's website.

Moral: hack it out and see what breaks.

I bought a copy of Linux Format magazine and found at least one useful thing in it: the screen command. This allows me to set up multiple consoles within a single ssh terminal connection: no need to open multiple terminals, I can switch between bash sessions within one terminal screen. It is even possible to split the screen into two halves and have, say, vim in the top and midnight commander in the bottom:

I think I used to use this 17 years ago on a Vax cluster through a VT100 type terminal (lanpar?). I had forgotten all about it.

For my ssh server I have disabled root login and I have chosen a slightly less common username and reasonably tough password but still I get people testing the locks. In the log files it is typically shown as a burst of login failures due to unknown name/incorrect password. This is no more than an annoyance, log files full of rubbish, but I'd like to prevent it and who knows, some day someone may hit the jackpot.

Linux iptables has a neat trick to limit the rate of connection attempts: three failed connection attempts and you can ban whatever is trying to connect for a couple of minutes. This is cool as it will still allow you yourself in as long as you get the password right in the first attempt or two.

I use the firewall module in webmin to manage iptables and I figured out how to implement this feature. It can be edited through webmin but I found it easier to edit the /etc/webmin/firewall/iptables.save file directly and to use webmin to apply it. This is the important bit:

# Allow connections to our SSH server from my IP address
-A INPUT -p tcp -m tcp -s 12.34.56.78 --dport ssh -j ACCEPT
# Allow connections to our SSH server from my other IP address
-A INPUT -p tcp -m tcp -s 65.66.67.68 --dport ssh -j ACCEPT
# Allow connections to our SSH server from localhost
-A INPUT -p tcp -m tcp -s 127.0.0.1 --dport ssh -j ACCEPT
# Allow three connection attempts in 60 seconds for anyone else
-A INPUT -p tcp -m tcp -m state -m recent --dport ssh \
   --state NEW  --set
-A INPUT -p tcp -m tcp -m state -m recent --dport ssh \
   --state NEW -j DROP  --update --seconds 60 --hitcount 3

One objection I have read to this technique is that it can allow someone to lock you out of your own server by continually hitting it with spoofed IP packets of your own originating server. For this reason I added rules to let connections from my normal haunts straight in: if someone bothers to IP spoof these addresses then they are talking directly to sshd and cannot do much more than a DOS attack, no different to if the firewall was not there. This also means I can get the password wrong or open and close connections as often as I like from the computers I normally use.

I could simply deny access from any alien IP except that the IP address of my home PC is not technically static (although it doesn't change very often) and I don't want to lock myself out if it suddenly changes. Besides, who knows, I may be out and about and want to log in (putty and a USB memory key, the world is mine).

This appears to work in both ubuntu and debian.

I bought another book, the Linux Pocket Guide (ISBN 0596006284). It is an interesting read as it is a summary of the most useful commands and utilities available in a typical linux distribution (the book is aimed at fedora but most of it, apart from the section on upgrading packages, applies to debian and ubuntu as well).

By summary of commands, I mean that it will describe a typical command like 'find' in a reasonable amount of detail and will describe the most useful options. It is not a reference book where you have to struggle to find the salient information: if you want to be bewildered there are always man pages to read. It is written in a clear and informative way: it is less than 200 pages and I skim-read it in an evening.

Things I have learnt from it:

• I use 'dig' but 'host' is a simpler alternative (no having to pick out the answer).
• by using

  set -o vi
  

  you can enable 'vi' editing on the bash command line
• why you need to use 'export' in bash: the variable is defined locally until you export it!

It is a user level guide, there is nothing in there about system configuration, daemons, /etc/init.d etc. I would be tempted to buy other pocket references for that kind of stuff. The book is only £5.56 on amazon.

This is the script I am using to back up my debian dedicated server to my ubuntu desktop. It uses ssh and rsync. It uses the cool rsync link-dest trick so that instead of creating multiple copies of the same file, it creates only one copy of the file with multiple hard links to it. I have my ssh keys set up so I don't need to give a password to log in via ssh.

This uses a 'pull' technique: the desktop reads the files from the server using this script.

This is not entirely efficent in that it will create a new set of backup files even if nothing changes: if you run the script ten times in a row then you will end up with ten identical sets of files. However, it backs up a web site that changes every day so running it once a day is valid.

Next job is to put selected files within the backup set into subversion. I decided against using subversion for everything, I can't see a way to automatically delete files bit I'd like to put the main sql dump into subversion.

   1  #!/bin/bash
   2  
   3  #
   4  # Rotate old backups:
   5  #   $1 = remote directory to backup
   6  #   $2 = local backup directory
   7  #
   8  function rotate {
   9      # Ripple old backups
  10      rm -rf $2/Backup9
  11      mv $2/Backup8 $2/Backup9
  12      mv $2/Backup7 $2/Backup8
  13      mv $2/Backup6 $2/Backup7
  14      mv $2/Backup5 $2/Backup6
  15      mv $2/Backup4 $2/Backup5
  16      mv $2/Backup3 $2/Backup4
  17      mv $2/Backup2 $2/Backup3
  18      mv $2/Backup1 $2/Backup2
  19      mv $2/latest $2/Backup1
  20  
  21      # Copy current version to latest, creating hard links where files have not changed.
  22      #
  23      rsync -avz --delete --exclude=.svn --link-dest=$2/Backup1 -e ssh $1/  $2/latest/
  24  
  25      #
  26      # Put a date stamp in the backup directory.
  27      #
  28      echo >`date +$2/latest/Backup-%Y-%m-%d` "Hello Peter"
  29  }
  30  
  31  rotate sshusername@ssh.server.address:/var/www/petersblog.org /home/peter/Backup/petersblog.org
  32  
Toggle Line Numbers

I was missing Midnight Commander in my debian server setup. Using dselect to try to install it and searching for 'midnight' or 'commander' did not find it. Woe. Tried downloading the source but running ./configure complained about missing glibc. Googling revealed some religious wars about gnome bloat and glibc dependancies.

Out of desperation went back to dselect and searched for 'mc'. This matched umpteen things but eventually I found midnight commander! Installed instantly! The description clearly called it 'midnight commander', no weird spellings, k's in odd places, l33t speak etc.

Searches for 'midnight' and 'commander' still fail, even after installing it, so what does the search option in dselect search through? There are 17,000 packages to choose from, a decent search facility is pretty much essential.

I set up awstats on my dedicated server. Awstats is a very comprehensive apache log file analyser that lets me see what has been going on at my site. I mainly use statcounter for visitor analysis is it allows me to see precisely what they have been doing, where they came from, which pages they looked at etc. Awstats is more statistics based, giving overall averages and summaries. Also, Awstats tells me about bots and crawlers which statcounter filters out.

Setting it up amounted to:

• install awstats package using dselect
• edit /etc/awstats/conf.local to customise, using settings from /etc/awstats/awstats.conf
  • point it at my log file
  • give it site name
  • set log format 1 which appears to be bog standard apache
  • exclude me/my ip addresses from stats
  • enable reverse DNS to see who is accessing me, not just ip addresses
• edit /etc/logrotate.d/apache2 and add:

  # pcw: from awstats faq: run awstats before log file is lost
  prerotate
  /usr/lib/cgi-bin/awstats.pl -update -config=petersblog.org
  endscript
  

  so log files get processed before logrotate renames/deletes them
• set up cron job to update stats every three hours. This is to keep awstats database updated and spread out the time it takes

  10 0,3,6,9,12,15,18,21 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -config=petersblog.org -update >/dev/null
  

• set up apache to deny access to awstats from anyone but me. This is for two reasons:
  1. privacy
  2. awstats has had at least one bad vulnerability in the past that allowed sites to be hacked

This gives me a better awstats setup than site5 gave me as I have enabled the reverse DNS lookup, meaning I see originating site names rather than IP addresses.

Now I have debian and webmin running on my oneandone server I must admit to being happy. It is much more to my taste than the plesk and fedora setup preinstalled on the server. I've lost the web reselling tools but I've got a powerful server that I feel I have some control over.

Installed webmin firewall module on oneandone debian server to make ipchains (or is it iptables? confused) setup a little easier. Enabled only ssh and http as incoming and can still connect to the box on ssh but I cannot telnet into the mail server.

I wasn't keen on making the webmin server open to all so I set up webmin to only allow local access on 127.0.0.1 and I wrote a batch file to create an ssh tunnel to get to it:

ssh -L 10000:127.0.0.1:10000  me@myserver.org

Opening localhost:10000 and I'm webminning.

In old dos days I used to use the fc command to do a binary compare on two files. I tried it today at a debian bash shell and something very odd happened. It opened up vim showing me the last command line I had typed. When I exited vim it then executed that command.

I had stubled upon something that might be useful, if I can remember it is there next time I want to edit a long command line.

I have a few problems with my oneandone dedicated server:

• I don't know my way around Fedora Core and cannot find anything
• Plesk seems to hide most of the configuration deep in it's own directories, making finding anything even harder.
• It is Fedora Core 2 which is a year or two old. The packages available are a little long-in-the-tooth (Mysql 3.2, Subversion 1.0.9) and the versions are becoming deprecated. Plesk 7.5.3 only supports Fedora Core 2 so ultimately again Plesk is an annoyance.

So I decided to install debian. Ubuntu, my distribution of choice is derived from debian so I know my way around it. Also there is a complete debian mirror on oneandone's servers that I can access without impact on my bandwidth allocation.

I was inspired by this article about installing debian on a oneandone server which gave me the confidence to give it a try. However I didn't follow these instructions for a few reasons:

1. It requires building a kernel on a running debian system. I don't have one of those and didn't feel like making one. The instructions erroneously say that the network interface has to be built into the kernel but mine is ok as a module.
2. It sets up static ip addresses instead of using dhcp. The oneandone faqs seem very insistant on using dhcp. Getting this going was the most difficult part of this setup but I am glad I have done it this way: my server will not suddenly break if oneandone reconfigure their network.
3. I didn't want to clear out my entire hard disk, I wanted to keep the fedora/plesk system in case I wanted to return to it. The fedora installation allocates 27G to a /home partition. I decided to copy the little of this that is used to the root partition and use the home partition (/dev/hda7) for my new installation. I can always re-use the fedora partitions later, if 27G is not enough for me.

I am still grateful to the author of that article for making me believe it was possible.

These are essentially the steps I took:

Start of logged into the fedora system, copy the old home directory to the root partition:

mkdir /newhome
cp -a /home /newhome
umount /home

If the umount fails because files in /home are open then run

/usr/sbin/lsof | grep home

to see what has the files open and stop or kill the processes (lsof, list open files, damn useful).

Create new file system in partition where /home was:

/sbin/mkfs.ext3 /dev/hda7

Mount the new installation:

mkdir /mnt/new
mount /dev/hda7 /mnt/new

Download debootstrap from debian mirror on oneandone servers. This mirror is only visible to oneandone customers. Untar and try to run

tar xvfz debootstrap-0.3.1.5.tar.gz
cd debootstrap-0.3.1.5
export DEBOOTSTRAP_DIR=`pwd`
./debootstrap --arch i386 sarge /mnt/new http://update.pureserver.info/debian sarge

gives error:
E: no /home/peter/debootstrap-0.3.1.5/devices.tar.gz. cannot create devices

Hum, supposed to be in debootstrap archive from what I can tell. Installed debootstrap under ubuntu and copied the file up.

If you get:

W: Failure trying to run: chroot /mnt/new mount -t proc proc /proc

then log in properly as root to get path set up correctly:

su -l root

rather than

su

Looking good:

I: Base system installed successfully.

Do base configuration. This is essentially the debian setup program, it installs loads of packages, asks you what time zone you are in etc.

chroot /mnt/new
mount /proc
base-config

Mostly obvious stuff but:

• apt sources set to

  deb http://update.pureserver.info/debian unstable main contrib non-free
  

• Install the following
  • Web Server
  • Mail Server
  • SQL database
• Use readline interface for configuring packages: I find the dialog interface invasive, it pops up and stops you seeing anything else on screen which you sometimes need to understand what is going on.

For me the nfs package failed to install so I ran dselect and removed it. Installed openssh server and, most importantly, vim. sshd just works, needs no configuration.

Setup /etc/fstab:

/proc           /proc   proc    defaults                0       0
/dev/hda7       /       ext3    defaults,noatime        0       0
/dev/hda2       none    swap    defaults                0       0

Set up /etc/network/interfaces:

auto lo eth0
iface lo inet loopback
iface eth0 inet dhcp

Use dselect to install Linux kernel image for PPro/Celeron/PII/PIII/P4 machines.

Ensure network card is in /etc/modules (without this we are lost):

via-rhine

My box has via-rhine, you can check in /var/log/messages to see what yours might be and look in /lib/modules/*/kernel/drivers/net to see if there is a module for it.

Install lilo:

apt-get install lilo

Copy /etc/lilo.conf from fedora and modify it:

boot=/dev/hda
# for fedora
#root=/dev/hda1
root=/dev/hda7

install=/boot/boot.b
vga=normal
timeout=10
prompt
lba32

read-only

default=lx
append="console=tty0 panic=30"

#
# Debian install.
#
image=/vmlinuz
    initrd=/initrd.img
    label=lx

Run lilo to set this up:

lilo

Quit out of the chroot and copy the /etc/hosts from fedora to the new install.

exit
cp /etc/hosts /mnt/new/etc/hosts

Now go to oneandone control panel and reboot the system into the recovery system. This will have you box running an emergeny debian install. It's handy because we are installing debian. Once installed, mount your new install so that you can copy some more files to it:

mount /dev/hda7 /mnt
cp /etc/dhcp3/dhclient.conf /mnt/etc/dhcp3
cp /etc/dhcp3/dhclient-exit-hooks.d/local /mnt/etc/dhcp3/dhclient-exit-hooks
chmod 755 /mnt/etc/dhcp3/dbclient-exit-hooks/local

This last step took me hours to figure out, I think the dhcp3/dhclient-exit-hooks.d/local is the trick as it does not exist in a standard debian install. While I was messing around I managed to reset the execute bits on the files which broke dhcp completely, hence the chmod to make sure.

From the oneandone control panel, reboot server in normal mode and enjoy.

What is so damn cool about linux is that doing something like this is possible at all. Reninstall a different operating system on a pc in a different country (Germany) through a command line terminal? Try doing that in windows.

I needed to set up another drupal site on my ubuntu linode. I had a domain name, I wanted to make it an independent site. I decided to keep it seperate from my existing site by putting in a fresh Drupal 4.6.1 installation and not to use Drupals virtual server facility.

I knew Apache2 supported virtual hosting and I decided to use that. I tried creating a new virtual host by creating a file in /etc/apache2/sites-available as follows:

<VirtualHost *>
        ServerName www.site2.com
        ServerAlias site2.com
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/site2
        <Directory /var/www/site2/>
                Options Indexes FollowSymLinks MultiViews
                # pcw AllowOverride None
                AllowOverride All
                Order allow,deny
                allow from all
                # This directive allows us to have apache2's default start page
                # in /apache2-default/, but still have / go to the right place
                # Commented out for Ubuntu
                #RedirectMatch ^/$ /apache2-default/
        </Directory>

        ErrorLog /var/log/apache2/site2/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/site2/access.log combined
        ServerSignature On

</VirtualHost>

where site2 is the name of the new site. Note that I created /var/log/apache2/site2 so that the site would get it's own access logs.

I used the command

a2ensite site2

to enable the site. I restarted apache2 and, bang, both this site and the new site showed the new site, I had broken this site.

After faffing around and googling, I tried a simple experiment. I removed the symbolic link to site2 from /etc/apache2/sites-enabled created by a2ensite and I just appended the above file to /etc/apache2/sites-available/default. I restarted apache2 and this worked, I had two sites. This is probably not the right way to do it but it works and any time I spend fixing it will bring this site down which bothers me so I'll leave it as it is unless I come across the correct way to do it.

Update: On my oneandone server running debian this is working fine as a seperate file, enabled with a2ensite:

<VirtualHost *>
    ServerName petersblog.org
    ServerAlias petersblog.org *.petersblog.org
    ServerAdmin webmaster@localhost

    DocumentRoot /var/www/petersblog.org
    <Directory />
        Options FollowSymLinks
        AllowOverride All
    </Directory>
    <Directory /var/www/petersblog.org>
        # pcw No directory listsings
        # Options Indexes FollowSymLinks MultiViews
        Options -Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
    </Directory>

    ErrorLog /var/log/apache2/petersblog.org/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog /var/log/apache2/petersblog.org/access.log combined
    ServerSignature On

</VirtualHost>

I have four sites set up like this and all are working.