Peter's Blog

I think I have found out what was mysteriously caching pages while I was editing drupal entries which I blamed on site5. It appears that Microsoft ISA firewall is caching pages, even if your proxy settings are not set up to use it. It also appears that it does not cache very well, if you look at a page, go to edit it, then return to the page, ISA serves up the old version from the cache and you have to refresh the page to see any changes. It's cache settings have a crude option marked 'Cache dynamic content (objects with question marks in the URL)' but I have nice url's set up so that doesn't do me any good.

I have set it to 'frequently expire' cache entries.

Moral: use squid which does not appear to have the same problem.

Had problems with Windows Update not working in Windows XP SP2 on my pc at work. Doing a manual update (running Windows Update from the Start Menu) all 18 pending updates failed after a time. The error code for all failures was 80072EFE. I found the errors by going to the windows update site and looking into my download history. Clicking on the 'failed' link gives a small window with the error code and bugger all explanation about what it means.

Googling for the error code gives vague rumours about proxy server settings so I checked mine. It turns out that by accident I was routing https requests through our proxy server. This is Windows ISA server on Windows 2000. I fixed this as follows:

• Go into IE/Tools/Options/Connection/LAN Settings/Advanced
• Turn off 'Use the same proxy server for all protocols
• Clear the entries for 'Secure'

After this things were fine.

Came in monday morning (when else) to find hard disk on Ubuntu server had gone down. It was giving repeated journal errors from the ext3 file system. I went to the IT dept to ask for a new hard disk and they gave me a brand new 40G Maxtor 6E040L0, a 40G 7200rpm beast. As this was better than the 16G IBM disk in my main PC which is suffering from about 5 years of accumulated crud I decided I would rather put the new disk in the main PC.

My old hard disk had Windows 2000 installed and years ago I was nieve enough to make it a 'dynamic disk' rather than a 'basic disk'. The advantages of a 'dynamic disk' are vague and only applicable to RAID arrays and suchlike. For a workstation there is no point and it becomes a liability as no tools like Norton Ghost, Partition Magic, Maxtor EZ-Drive or whatever work properly, they all run into limitations. After messing around trying to work out how to clone the Win2k partition to the new disk I decided upgrade to Windows XP on a clean install as I had a spare licence going unused on the old server PC. This PC had 96M of ram and XP is unusably slow on it.

I installed XP and it is wonderful, the system is so much faster and quieter than the cruddy old IBM disk.

XP salient points:

• would not recognise old non-plug and play DE220 ISA network card. Had to use a PCI DE528.
• Recognised Matrox 440 dual-head graphics card and ATI Rage IIC and offerred my three monitors. The graphic card setup dates back to Windows NT where the dual-head was a poor imitation of multi-head. I stuck with dual head using one of the Matrox channels and the ATI. I could plug another monitor in, for the hell of it, but:
  • I don't want to be irradiated.
  • Don't want to load the system to much as it only has 256M ram.
• although I was transferring the installation from a different PC I activated it online with no problems. I did wonder if I would have to ring Microsoft to justify it but no.

I am not sure that the old server hard disk is totally dead, I can read the partition table in my main PC but not in the server PC. I may be able to rescue the old installation, none of which is backed up. Thank god for this blog, where I noted down everything I did to set it up.

Installed Ubuntu linux. Posting this from Firefox running on Ubuntu.

The installer created a version that worked pretty much automatically. The only tweeks I had to do are:

• the installer hardware detection could not find my old dlink de220 ne2000 compatible ISA network card. I had to switch terminals and run:

  modprobe ne io=0x300 irq=9
  

  switching back to the installer, it found the network and was happy. It downloaded a load of updated modules from somewhere via the nerwork. It configured everything via DHCP, even found the name servers.
• Once Ubuntu was installed it ran ok but network was dead again. I had to edit /etc/modules to add the line

  ne io=0x300 irq=9
  

  and restart (although a modprobe would probably have done the trick).
• I used the network manager to change from the DHCP setup to a manual one so server has a static IP address.
• I edited /etc/apt/sources.list to add a 'universe' repository so I could use the Synaptic package manager to install packages unapproved by Ubuntu. This allowed me to install xawtv. I also installed 83 module updates.
• Ran xawtv from the command line and it Just Worked: the hardware detection had found my Pinnacle PCTV Rave and loaded all the bttv modules. Ubuntu seems to use a 2.6 kernel, hence my old problems were resolved.

Ubuntu observations:

• Don't like the brown colour scheme.
• root account is disabled! Everything has to be done via sudo. Apparently Max OS/X works this way. I was able to edit the config files above using 'sudo vim '. Sudo keeps prompting for passwords.
• Synaptic package manager looks nice, nicer than dselect (old text based apt tool I used to use on debian with weird key assignments).
• Samba network client Just Worked: was able to find Windows XP pc and poke around.
• Not bowled over by the documentation on the Ubunto web site. Mostly Wiki notes, often directing me to Room 101, a catchall.
• Modern X has much nicer fonts than Suse 6.4 vintage.

I've written a python script to handle email archives. This script scans my Maildir directories and moves messages that are older than a directory-dependant age to a compressed archive. I will run this from cron.

#!/usr/bin/python
#
# Walks through Maildir store, compressing and deleting old email messages.
#
import email
import os
import time
import bz2

strRootMailDir = '/home/pcw/Maildir'

#
# Folders that can be pruned
# Defined as a dictionary where keys are directory names and values 
# are maximum age in weeks
#
oPrunable = { '.IT.Auto.ISA Server Alert': 2,
        '.IT.Auto.Message Deleted': 2,
        '.IT.Auto.Sophos Alert': 2,
        '.IT.Auto.Sweep Report': 2,
        '.IT.Auto.Virus': 2,
        '.INBOX': 12,       # can sit in inbox for 3 months
        '.Sent': 4}

#
# Callback from os.walk to prune a particular directory.
#
def ProcessDir( strArg, strDir, strNames):
  #
  # Split tail part of directory name.
  # This should be cur, new, tmp etc.
  #
  strPartDir, strTailDir = os.path.split( strDir)

  if strTailDir != 'cur':
    return

  #
  # Get maildir mail folder name.
  #
  strPartDir, strMailDir = os.path.split( strPartDir)

  #
  # Make sure folder is in list that are subject to pruning.
  #
  if not oPrunable.has_key( strMailDir):
    return

  #
  # Go through the files.
  #
  for strFile in strNames:
    strPath = strDir + '/' + strFile

    #
    # Get file creation time. Don't look in message itself as sent time 
    # is untrustworthy. Process old files
    # 
    oStat = os.stat( strPath)
    nFileTime = oStat.st_ctime
    if time.time() - nFileTime > (3600 * 24 * 7 * oPrunable[strMailDir]):
      #
      # Attempt to parse message. Make sure it is a mail message.
      #
      oMessage = email.message_from_file( open(strPath))
      strFrom = oMessage['from']

      #
      # Archive old email comressed.
      # Shove in existing BZ2 archive in mbox format.
      # The mbox is readable by the python mailbox module.
      #
      oBZ2 = bz2.BZ2Compressor()
      oBZ2.compress( 'From %s  %s\n' % (strFrom,
            time.strftime( "%a %b %d %H:%M:%S %Y", time.localtime( nFileTime))))

      oBZ2.compress( open( strPath).read())
      oBZ2.compress( '\n')

      #
      # Create Archive directory, per maildir, per month.
      #
      strArchiveName = strRootMailDir + '/Archive.%s.%s.bz2' % (
                      strMailDir[1:],
                      time.strftime( '%b%Y', time.localtime( nFileTime)))
      open( strArchiveName, 'a').write( oBZ2.flush())

      #
      # Delete the file.
      #
      os.unlink( strPath)

os.path.walk( strRootMailDir, ProcessDir, '')

Dixons in town were having a closing down sale so I had a browse. They had a Packard Bell system without monitor for just under £130. It is a couple of year old Athlon 1.7GHz thing with DVD/CDR, 256M and 40G hard disk. It was cheap because it had some kind of traumatic accident and the case was severly bent. They told me it worked and gave me a 6 month guarantee so I bought it.

I have put it's guts into the case of my old desktop pc so it is now more presentable.

Pros:

• It feels much faster than my old system, and in fact any system i have used.

• Hopefully it won't lock up randomly like the old one: this is what made me lose confidence in the old system and not want to use it.

• It has freed up extra 128M memory and a CD/RW for my Gentoo box.

• I can build a cheap system with the bits left over for my mum. I can set it to PIO mode so it won't crash and it'll still be faster than what she's got.

• I have a spare Windows XP Home licence

• I have a spare 80G hard disk to fix my sky plus box with

• It's got a keyboard with lots of extra functions. I just found a function button where ctrl should be: function+left takes me to line start.

• Nice mouse

• Lots of software. Microsoft Zoo Tycoon!

Cons:

• Not the most high tech motherboard I could have bought (MS 6511). Still, I found a reference for it on the internet easy enough.

• Only 2 PCI slots: 1 after network card is put in. No ISA (rip).

• Probably USB 1.1 from it's vintage. I have a USB 2 card to put in but then all the PCI slots are gone.

• It starts up quite rapidly but it takes 30 seconds to shut down: very annoying when rebooting.

In a nutshell: I'm happy.

A footnote on why I use Windows and Gentoo. I think Windows is a good GUI and Linux is a good server. X apps are just not as slick as windows apps and setting up Windows server software is a nightmare of tree's and property pages (I'm talking Active Directory, Microsoft Exchange, ISS and ISA, I use them at work and I hate them all).

I used to get very little spam at work since the company email was protected by SurfControl spam filter. It's a cruddy program with limited MTA features but it does a valiant job in fighting off the spam, most of which fails a DNS PTR record lookup.

Recently I was added to an email group which started 'it@' and since then I have started to get spam since this is fairly easily guessable compared to my initials. Also I started to get cron alerts, ISA server port attack alerts, Sophos sweep virus scan alerts (a 1Mb list of encrypted files that cannot be scanned, no viruses) and assorted other stuff, making it hard to find real email amongst the noise.

So I had a reason to setup SpamBayes. It's cool and it already doing a good job of getting rid of this stuff. I set it up as a pop3 proxy running on a debian box that uses fetchmail to read my email accounts, exim MTA and qpopper for pop3. I have Big Plans for the box to archive my project related email so I can hyperlink to it from my project notes. I like the power of using an exim .forward file to run a python script on my email: limitless power.

I have 270MBytes of email archives trapped in outlook proprietary format. I have dabbled in writing outlook macros but they are incredibly flaky: it randomly stops running then from mail rules for no apparent reason (silently unchecking the box that enables them). At least the .forward file activities get logged somewhere.

Been looking at the ISA firewall/proxy thing at work and studying the logs. It's amazing what people get up to when they are supposed to be working. This has filled me with paranoia (not that I look at naked ladies while I am supposed to be working).

I installed squid on my server at home. I can connect to that through ssh. I don't know how fast it will be because of the uplink speed but it does means I can do banking and stuff from work.