Peter's Blog

Last time I looked at the Palm T2 the battery was flat so I put it on charge. Turned it on this morning to find out time of an appointment and the thing was totally reset, back to initial settings.

It was my first opportunity to try out BackupBuddyVFS Restore and it has worked perfectly. Thumbs up for that. It may be that the auto-backups at 5pm everyday are what flattened the battery but it has made up for it.

The digitiser on my palm is definitely flaky. The initial calibrate thing asks me to tap on the centre of three crosses and I have to do this many times before it will accept the calibration. It is very fussy about precisely where I tap on the screen. For examplem if I want to click on a button, sometimes I have to tap just above the button to get a hit.

Spamassassin at work is detecting spam nicely, no false positives so far although some false negatives are getting through. I don't have a big enough spam or ham corpus to get the bayesian spam filter running but when I do this could make a difference.

I have created three IMAP folders for spam:

I then have a cron job running each night to run sa-learn --spam on the 'categorically' folder and sa-learn --ham on my inbox. It runs on the 'cur' subfolder but not the 'new' subfolder so it should not pick up spam that arrives when I am not around to approve it.

The Definitely spams are not added to the bayesian filter. I think this is best, it avoids reinforcing prejudices which is not a good thing.

The spam I receive has already been through the companys surfcontrol spam filter. Draw your own conclusions.

Something I must do sometime is extract 250M of old emails from my outlook archive and put them on the IMAP server. I could do this by hooking Outlook up to the IMAP server and just moving the messages but I have a gut feeling that this will take hours and involve a lot of pain.

At home I must figure out a way to get my old email out of the clutches of Intellimail so that I can uninstall it.

Google know how to make me happy. They've given me 6 gmail invites. I'll be giving them to friends/family. I won't sell them on ebay, not even to recoup my £1.30 investment.

I got a spam message today so I marked it as such and looked in the spam folder to find there were already 3 spams in there: there was no other indication that they were there. They were legitimate spams though, asking me if I wanted to advertise my web site to 28 million people by sending spam. Does the 28 million people include the ones with spam filters? If so I'd want a discount.

The gmail invite scheme is a great ploy. Slow adoption of the new system, spreading in a way that makes people feel privileged, starting with techy types who can invite and help their non-techy friends.

Update:

• I gave myself a gmail invite to use for work and managed to grab the coveted address that has my own name (guess what it is, cursed spambot). My name is not uncommon (I appear on the third page of a google search which finds my Firl entry) so this is a win.

• I've given one other invite away and yet I still have 6 invites.

• I logged into my new account and that has 6 invites.

• I think the spam I received is spam I forwarded to myself from my home server.

So are google going crazy with invites today, is it a weird cookie bug, or am I just lucky?

Update^2:

• It was a cookie bug. I had two different accounts open in two firefox tabs and the first account got transformed into the second account. The new account did have 6 invites already. It seems you can only look at one account at a time

Latest discovery is this awkwardly named site. It's a stream of interesting links recently submitted by people. I find it interesting to scan through when I run out of RSS feeds. In fact it can generate an RSS feed but that would probably swamp my aggregator.

Some parasite posted a comment to an entry in this blog that was just a list of links to other websites, probably to boost google rankings. I'm fuming at these parasites. I've deleted the comment. It was only up for a day so hopefully the googlebot didn't spot it.

Python Desktop Server (or the version I am still using) seems to have a problem generating the archive calendar, some days do not get hyperlinks to archived entries. This makes it awkward to find them, I have to guess dates in the hyperlinks.

Also, if I have an archive page open and I type in a new posting, PyDS creates the new posting with the archived page date rather than today. It just happened, you may see two similar posts.

Welcome

Created a new hosting account with site5. Impressed so far, 1.5G of storage, PHP, unlimited MySQL databases, auto-install of drupal, fast server, all for $7 a month. Sorry if this sounds like an advert.

My previous blog was hosted on Python Community Server and written using Python Desktop Server. I have a couple of reasons for changing, especially to something I have to pay for (!)

• My Python Desktop Server install was not the 'official' version, it was a Gentoo package installation and it suffered from a number of little problems that I didn't have time to fix.
• I've used Drupal at work and on another site and I really like it. Even when I have to drop into the php code I am not totally lost, the code is easy to follow.
• Drupal is better documented.
• All my blog entries are stored in a standard database format, not something obscure (heard of metakit ?).
• I (think) I can set things up so that I can moderate comments. I was getting comment spam and I hate it, it makes me feel violated.
• Drupal data entry has a cool 'preview' button, with Python Desktop Server it just gets published, complete with formatting errors.
• Drupal search facilities just work. I never did work out the hack required to add a search to my Python Cumminity Server. The only way to find old posts was using the calendar thing, and that was buggy and did not link every day that had postings.

I could go on and on but the fact is, here I am.

ToDo list:

• Create nice custom theme
• Upload old blog

Getting Drupal to run on Site5 was not entirely straightforward. I used the fantastico script thing to install it but I got 403 errors whenever I tried to access the site. The error told me that I could not access /index.php. This was resolved by putting the following in my .htaccess file:

Options ExecCGI

Then I was still getting a 403 error on the directory /. The error log said:

[Thu Sep  9 06:07:36 2004] [error] [client 80.88.204.40] Options FollowSymLinks or
SymLinksIfOwnerMatch is off which implies that RewriteRule directive is forbidden:
/home/bisiand/public_html/403.shtml

So I changed .htaccess as follows:

# Set some options
Options -Indexes
Options +ExecCGI
Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

Drupal started working but I kept getting the following errors on each page:

warning: Cannot modify header information - headers already sent

This was because I had been editing the Drupal conf.php file using Site5's NetAdmin tool and whenever I saved the file a blank line was added to the end. php was treating this as content to be output. I had to download the conf.php file, edit it in Vim to delete the blank lines and upload it again.

Trying to modify the Drupal theme, I then got an error from marvin.theme about no base class to inherit from. To fix this I had to move the directories themes/marvin and themes/unconed to the /tmp directory to hide them. There may be a fix to get them going but I don't really care.

After this, everything was fine.

I've spent a good few hours trying to set up my email on my new Site5 account. I wanted to set it up as follows:

• Mails to my various target addresses (personal, public and wife) all arrive in a single account in different folders. This way I only have to poll a single account. This account is also my backup email archive.
• All mail (except maybe spam checked mail) is forwarded to gmail to use as my email client. Site5 gives access to three web based email packages (Squirrelmail, Neomail, Horde) but they are not as good as gmail.

This is how my work email is set up and also how my former home email was set up. Both use procmail to do filtering and forwarding and are working sweetly.

So how to do this in the more restricted world of Site5?

On the Site5 site this is the main clue on how to set things up. This describes a technique of editing a file called /etc/valiases/ to get the email system to forward emails to procmail. procmail is then used to filter mails to the appropriate target folders, run spamassassin etc.

I tried using the prescribed setup and encountered two problems:

• forwarding to gmail from the procmailrc did not work. According to the procmail log it did try to use sendmail to send the message but the messages never arrived.
• I ended up with two copies of each mail, one that had been through procmail and another that did not (it had no spamassassin headers).

I did a google on cpanels (which is a relative of netadmin, which Site5 use) emailing forwarding facility and worked out that it is not necessary to edit valiases directly, the file can be edited via the netadmin email forwarding section. For each of my required email addresses I added two forwarding entries:

1. A forward to my gmail account
2. A forward to procmail (e.g. "|/usr/bin/procmail /home//.procmailrc")

The .procmailrc is set up in a conventional enough way, e.g:

## basic .procmailrc
SHELL=/bin/sh
LOGFILE=/home/blah/procmail_log   ##replace with your actual domain!
VERBOSE=yes    ## you can set this to YES for debugging
MAILDIR=/home/blah/mail/blah.com
DEFAULT=messages/inbox

# isn't bigger than a few k and working with big messages can bring
# SpamAssassin to its knees.
#
# The lock file ensures that only 1 spamassassin invocation happens
# at 1 time, to keep the load down.
#
:0fw: spamassassin.lock
* < 100000
| spamassassin

# Mails with a score of 15 or higher are almost certainly spam (with 0.05%
# false positives according to rules/STATISTICS.txt). Let's put them in a
# different mbox. (This one is optional.)
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
messages/spamdefinitely

# All mail tagged as spam (eg. with a score higher than the set threshold)
# is moved to "probably-spam".
:0:
* ^X-Spam-Status: Yes
messages/spampossibly

#
# Sort into recipients
#
:0
* ^TO_someone@blah.com
messages/someone

:0
* ^TO_someoneelse@blah.com
messages/someoneelse

:0
DEFAULT

All emails go into an account called 'messages' which has folders for each recipient and also folders for messages that might be spam or are definitely spam.

This setup seems to be working. I was getting two copies of my emails before because I had set up email accounts with the same names as I was forwarding to procmail. If the Site5 email system receives a message to fred@somedomain.com it will look for email accounts called fred@somedomain.com and deliver the message there if it exists. It will then look for an email forwarding link for fred@somedomain.com and will forward it if that exists. I was getting two copies of email because the email account existed and got email delivered directly and also because my procmailrc was delivering another copy. After I deleted the extra email accounts I stopped getting duplicate copies. Everything arrives via procmail.

I'm not sure why procmail could not forward email to gmail but I suspect the Site5 account may be set up to restrict sending of emails to stop spammers using Site5 hosting.

I don't think it is necessary to have shell access to Site5 to do this. Editing the valiases file apparently would require shell access but that is not necessary if you use netadmin (aka cpanel) to set up the forwarding. The .procmailrc file can just be uploaded using ftp. I did ask Site5 for shell access and got it within an hour. So far I am very pleased with Site5: fast servers and fast support.

Been playing with Gmail some more, looking at it's features.

Labels

Instead of putting email in folders you assign it labels. You can then click on the label name to get a list of all emails that have that label. This is functionally the same as having folders. One advantage is that more than one label can be assigned to an email so it can be categorised in different ways. I don't see any way to AND labels together, i.e list emails with label1 AND label2. I tried using the search box but that did not work. I would say this would be a useful geeky feature. The labels then approach the power of the taxonomy system in Drupal (although in a single vocabulary with no hierarchy: hey I sound like a computer scientist!!).

Filters

Easily used to apply labels to new email.

Archive

I had ignored this before but I think the intention is that instead of leaving everything in the 'Inbox', once you have read your mail you press the Archive button to put it in an archive. This leaves the Inbox uncluttered. You can get at your archive by searching, clicking on labels, etc.

Spam

Automatically filtered, and generally very well. Spam messages to not appear in the 'All Mail' list or search results. There is no way of knowing new spam is there except by looking in the folder (folders, not labels, mixed paradigms).

Conclusion: I love Gmail, it follows googles fast, simple and powerful approach.