Peter's Blog

I've missed blogging while I've been working on PetersBlogger. My old drupal blog was limping along: while administering the site it kept logging me out. I can admit now that I was running an old version of drupal (4.7?) which I couldn't upgrade due to module dependencies (mainly awtags). The old code wasn't running that smoothly under php5 despite initial appearancies.

I'm happy with my new blog platform, it's running really nicely on my slicehost slice and I've learnt an awful lot about rails while developing it. I will try to do a braindump of rails experiencies in forthcoming posts (cross fingers).

Drupal is a fine CMS and I'm using Drupal 5 on my company intranet. My thoughts on a drupal vs ruby on rails debate would be:

• drupal/php/apache is easier to deploy than a rails/ruby/mongrel/nginx deploy: I moved my drupal site between about four different hosts and did it each time in less than an hour. A rails setup is more convoluted (especially mongrel_cluster). However, this site is running much faster under rails than it was under drupal. The difference is probably down to php and apache being more mainstream and slightly more refined and hence easy to set up. I'm not saying rails is a total pain, just that if I only had ten minutes in which to deploy a site I would be reaching for drupal. If I only wanted to install the source and a few modules and never touch any coding then I would be happy with drupal.
• I love ruby and rails development. There is no comparison, php seems to me as much a bastard language as visual basic 6. Ruby was cleanly developed as an object orientated language, php is having object orientation grafted on as an afterthough.
• If I look through my drupal source I see no unit testing. Ruby/Rails has unit testing built in and I'm completely sold on it. I don't think I will ever trust any code (especially code I write) if it hasn't been unit tested. Some interesting aspects of unit testing:
  • I understand now the 'test driven' approach to development where you design a module's api by first roughing out how the tests will work: the unit test is your first experience of using the new api. It's during testing that you learn how nice an api will be.
  • If the documentation of a new ruby/rails tool is dubious, look in the unit tests to see how the author intended it to be used. If there are no tests then run.
  • If I run into a tricky bug, it's better to first reproduce it in unit tests, fix it there and then be happy that the bug will never recur. It's easier to debug code in a unit test (essentially a command line application) than on a live website.
• all ruby on rails applications follow basically the same architecture. It is fairly easy to figure out how a new application works. Every php application is different. I know I'm comparing a web framework to a programming language but none of the php applications (drupal, phpmyadmin) or am familiar with (wordpress) share a common architecture. New php application to maintain? New learning curve.
• my new site is being hammered with attempts to post to /comments/reply, even though it doesn't use that url for comment submission. The comment spammers know drupal and know where to test the locks. If any of them ever bother to try to find my comment submission url they will only find the same captcha that protected my drupal site (which was much easier to implement in rails, simply as a validation on the model).

This blog was offline for a while yesterday. One of the perils of shared hosting is that you have to share the mysql database with other people who tend to use up all the available connections etc and an admin has to sort it all out.

The blog was left broken and visitors were advised that the drupal sessions table was corrupted and it needed repairing. How nice of drupal to blab secrets like the name of the database. Repairing the database through the site5 interface didn't work, it didn't even list the sessions table in the results. Fixing it turned out to be quite simple, log into mysql and go:

REPAIR TABLE SESSIONS;

The sessions table had 25,000 entries in it! Since this is old session data and not at all essential (I have the only active account) I zapped it:

DELETE FROM SESSIONS;

Fixed.

Apart from server maintenance, problems with mysql seem to be the main reason for downtime on this blog. Should I move it across to my new server where all the connections are mine and mine alone? If I did I would probably want to make sure I was backing the blog up: I wouldn't want to lose three years and 43 weeks, 1480 articles, a detailed history of my World of Warcraft adventures and the internets primary cuprin0l fence sprayer vitriol page. Site5 do automatic backups, backing up on the new server would be totally down to me. Fortunately I can back it up to... my site5 account!

There are three ways to blog from a pocketpc that I have found:

1. open your blog in a web browser and edit it as normal. This doesn't work so well in practise unless your web page has been designed with a css style sheet that works nicely on a pda. I tried modifying my drupal theme accordingly and made it presentable but it's not up to data entry.
2. try using an application for posting to blogs. There are a couple of these about but those that I tried were buggy or had tiny little edit boxes.
3. write the article and submit by email. This is how I am posting this, I use phatnotes, a notetaking application that can send the notes via email. I don't use pocket outlook for the simple reason that it doesn't seem to save sent mail so I cannot edit the article and send it again to modify it. I use my old mailbot script on the server to capture posts and poke them into drupal.

So far this has worked ok and I can compose posts offline and upload them at my leisure.

Disadvantages:

• No preview unless I go online, hence I keep posts simple.
• cutting and pasting urls is fiddly and I cannot be bothered with it.
• my script adds tags by scanning the database for existing tags, searching the post for the tag words and adding the tags that it finds. This can give irrelevant tags and I cannot define new tags. I need a neat way to specify tags in the post.

Advantages:

• the convenience of whipping my pda out and having a quick blog. I don't have to go upstairs and get my laptop.
• can lay on settee and blog in comfort.

What's the single most tedious thing about blogging: having to press the 'Add Blog Item ' button. I think I've hit on why taking notes with EverNote is fast: there is a blank note at the bottom of the screen that you can just start typing in.

I've realised that this could be done in a web app, simply by having a box on every page ready to start typing in. Google mail does something like this already: there is a little text box at the bottom of each message that you can select and start typing a reply: when selected it resizes itself and formatting toolbars and stuff appear. It's all very AJAXy and slick. Most of the time you can ignore it because it's not too big, once you explicitly start using it all the associated tools appear.

Compare to posting in drupal where there's all kinds of stuff filling the page: Input format, date edit boxes, categories, tags, I have to scroll down two pages to find the 'preview' and 'submit' buttons.

Here's a rough outline of the quick blogging features:

• Regular blog page appears with a textarea box ready for typing in
• Title is first line (if preceeded with a - or something), then a list of tags line, each tag preceeded with a dot or something similarly lightweight.
• Rest of post is in wilki format.
• Big button that uses AJAX to generate a preview which appears under the text box
• After preview a post button appears.

Everything goes in the one textarea box, no need to tab or click between controls. Now I know what I want, how to implement it?

I am growing disillusioned with EverNote, mainly because of the buggy handling of formatting: if you mark something bold, for example, it has an annoying habit of not turning the bold off, you have to fiddle around selecting a big block and turning it all off explicitly (very much like Microsoft Word). I'm happy to use markup to make things bold, it's simple and understandable.

I want all my notes on a server where I can get to them from anywhere.

I knocked up a quick python script to scan my drupal watchdog list for comment spammers. The log covers the last week. In total there were 1250 spam attempts from 448 distinct ip addresses.

All these comment spams pretend to come from Windows XP, IE 6 so they cannot be filtered out by user agent.p

My hack to the comment module to prevent urls being submitted generates watchdog messages and this script looks for these.

Here is the script:

   1  import MySQLdb
   2  
   3  o = MySQLdb.connect( '127.0.0.1', 'me', 'secret')
   4  
   5  o.select_db( 'drupal_db')
   6  
   7  c = o.cursor()
   8  
   9  c.execute( """select message, hostname from watchdog
  10                where message like 'Comment:%'""")
  11  
  12  oBadGuys = {}
  13  oGoodGuys = {}
  14  
  15  while 1:
  16      oRow = c.fetchone()
  17      if not oRow:
  18          break
  19  
  20      strMessage, strSender = oRow
  21  
  22      if strMessage.startswith( 'Comment: attempted'):
  23          oBadGuys[strSender] = oBadGuys.get( strSender, 0) + 1
  24  
  25      if strMessage.startswith( 'Comment: added'):
  26          oGoodGuys[strSender] = oGoodGuys.get( strSender, 0) + 1
  27  
  28  #
  29  # Good guys manage to submit comments without problems.
  30  # Remove them from the bad guy list.
  31  #
  32  for strKey in oGoodGuys.keys():
  33      if strKey in oBadGuys:
  34          print strKey + ' is not so bad'
  35          del oBadGuys[strKey]
  36  
  37  nTotal = 0
  38  
  39  for strKey, nCount in oBadGuys.items():
  40      print strKey, nCount
  41      nTotal += nCount
  42  
  43  print "%d spams from %d bad guys" % (nTotal, len(oBadGuys))

I must get on with my turbogears based blog so I can do more about this. Drupal logging is a bit lame: doesn't log referrer or user agent which might be useful, have to cross reference with apache logs. There is more that I can do to make it harder to suck my bandwidth but my php is not strong enough and it's more fun to do in python (won't wear my $ key out).

My site is being really hammerred by comment spammers today but not one has got through thanks to my policy of refusing to allow comments containing urls to be submitted (not even for moderation: I moderate all comments, I found deleting comment spam to be tedious as well as annoying).

It is a simple hack to the drupal comment module but it is very effective. Ok, I could get spam without url's in but what's the point, apart from vandalism? They still go into the moderation queue and get deleted.

And when people do want to post url's they soon figure out how to get around the block. If they cannot do that then their comments are probably not worth consideration anyway.

I'd give the details of the hack here but it gives the spammers a clue. If you are interested then email me.

Update: following on from the vast surge in comment spam attempts (three or four a minute, 24/7), statcounter tells me people are searching for drupal captchas. I have given up on these, something about drupal states, redirects, session management or whatever stops them working reliably. The spam comment check is just part of the comment validation, there is nothing much that can go wrong with it, it is just straight if/then/else code.

In a way the spam check is a captcha (Completely Automated Public Turing Test to Tell Computers and Humans Apart), you can still get through if you show some smarts. It doesn't use graphics so it doesn't look cool and it doesn't shut out blind people.

The comment spam is coming from a range of ip addresses, maybe an array of compromised pc's (thanks Microsoft). Each 'failure' page is using some of my 10G/month bandwidth. I'll have to keep an eye out and see what kind of impact this is having. It could be even worse than inktomi slurps bots doing 100M of crawling a month and not directing anyone here through their search results.

Had some users complain about drupal generating narrow little comment boxes in version 4.6.4. drupal was generating 'cols="70"' instead of 'cols="70"' in the html.

Traced it to the function include/common.inc/form_textarea which handles cols in a different way to rows and messes it up. Worse still, it looks as if someone has done this on purpose with of course no mention why. I changed the code so cols are handled in the same way as rows and I'll wait to see what has broken.

Here is my version, reformatted to avoid the last line being 413 columns long (what's that smell?).

   1  function form_textarea($title, $name, $value,
   2                         $cols, $rows, $description = NULL,
   3                         $attributes = NULL, $required = FALSE) {
   4  // pcw  $cols = $cols ? ' cols="'. $cols .'"' : '';
   5    $pre = '';
   6    $post = '';
   7  
   8    // optionally plug in a WYSIWYG editor
   9    foreach (module_list() as $module_name) {
  10      if (module_hook($module_name, 'textarea')) {
  11        $pre  .= module_invoke($module_name, 'textarea', 'pre', $name);
  12        $post .= module_invoke($module_name, 'textarea', 'post', $name);
  13      }
  14    }
  15  
  16    return theme('form_element',
  17                 $title,
  18                 $pre .'<textarea wrap="virtual"'
  19                     .' cols="' . check_plain($cols).'"'
  20                     .' rows="'. check_plain($rows)
  21                     .'" name="edit['. $name .']" id="edit-'
  22                     . $name
  23                     .'" class="'
  24                     . _form_get_class('textarea',
  25                                   $required,
  26                                   _form_get_error($name))
  27                     .'"'
  28                     . drupal_attributes($attributes)
  29                     .'>'
  30                     . check_plain($value)
  31                     .'</textarea>'
  32                     . $post,
  33                 $description,
  34                 'edit-' . $name,
  35                 $required,
  36                 _form_get_error($name));
  37  }

Since I started with statcounter very nearly a year ago, it tells me my site has served up 100,214 pages to 72,521 visitors.

I have quickly become bored with Google Analytics: I think it's appeal is more for marketing departments and powerpoint/pie chart enthusiasts. I like the raw detail that statcounter gives me in an easily accessable way. I can see what people are searching for and sometimes it even inspires me to update articles to be more useful.

I still think their counting is buggy: returning visitors seems unreliable, as if page refreshing counts as a return visit, so I take it with a pinch of salt and follow the trends. The numbers are roughly on a par with google analytics.

Statcounter is easier to install, it can go anywhere on the page, Google Analytics has to go in the header. For drupal this means editing the page template rather than sticking it in a block.

Been looking at Mochikit, a javascript utility library, and it's good enough to remove my cynicism about javascript.

I had to play with the AJAX stuff, the ability for javascript in a browser to send a request to a server and the server to respond with data for the javascript to poke into the existing page. The big advantage of this is that the server does not have to send back a whole page for display, so the overall effect is smoother and faster.

Mochikit makes this easier and hides some of the problems with different browsers (i.e. bugs in IE). It won't work on old broken browsers but there are enough modern browsers going for free that I don't really care.

I've put my example online here but I may remove it if it gets abused or I may rearrange things and break it. In this example you type something in the box and click the link. Whatever you type is sent to the server, processed and sent back. The browser then puts the new version at the bottom of the page. Not a big deal but it's not what it does, it's the way that it does it.

Here is the code.

First, the web page:

   1  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   2  <html>
   3      <head>
   4          <title>Peter plays with Mochikit</title>
   5          <script type="text/javascript" src="MochiKit.js"></script>
   6          <script type="text/javascript" src="Peter.js"></script>
   7      </head>
   8      <body>
   9          <h1>
  10              Peter plays with Mochikit
  11          </h1>
  12          <form name="Enter Stuff">
  13              <label>Enter something:</label>
  14              <input id="blah" name="user" value="fred"/>
  15          </form>
  16          <a href="javascript:void(0)" onclick="onDoit()">Click Me</a>
  17          <div id="putithere"/>
  18      </body>

Then the javascript (Peter.js):

   1  
   2  var gotMetadata = function (oData) {
   3      var payload = evalJSONRequest( oData);
   4      replaceChildNodes( "putithere", P( null, payload.what));
   5  };
   6  
   7  var metadataFetchFailed = function (err) {
   8    alert( "The metadata for MochiKit.Async could not be fetched");
   9  };
  10  
  11  function onDoit() {
  12      var xmlHttpReq = getXMLHttpRequest()
  13  
  14      xmlHttpReq.open( "POST", "Peter.py", true);
  15      xmlHttpReq.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  16      var d = sendXMLHttpRequest( xmlHttpReq, "blah=" + escape(getElement( "blah").value));
  17      d.addCallbacks(gotMetadata, metadataFetchFailed);
  18  }

Then the python cgi that runs on the server (Peter.py):

   1  #!/usr/bin/python
   2  
   3  import cgi
   4  
   5  form = cgi.FieldStorage()
   6  strBlah = form.getfirst( "blah", "")
   7  
   8  print """
   9  { "what": "It says %s" }
  10  """ % strBlah.replace( '"', '\\"')

This also requires the packed Mochikit library which is available in the Mochikit distribution (/packed/Mochikit/Mochikit.js). All the files can go in the same directory on the server. Obviously you have to get cgi working with python.

My main motivation for using this is to see if I can optimise the preview facility in this blog: I would like more-or-less instant preview generation. This is definitely achievable, the only problem being getting mochikit to work with php rather than python, thats where I am in unknown territory. I might cheat and use xml-rpc from python to drupal.

The only downside of Mochikit that I can see if that it is a 96k download which makes dial-up users suffer even more. You have my sympathy.

If my rss subscribers are getting a flood of seemingly duplicate postings it is because I decided to reformat my posts a tad to emphasise the awtags links below the node bodies. I edited the awtags source to change the word 'tags:' to the more informative 'Related Topics:' and I edited the awTags_TagLinks css style to delineate the links from the node body:

   1  .awTags_TagLinks {
   2      padding: 5px;
   3      margin: 20px 10px 10px 10px;
   4      border-top: 1px solid black;
   5  }
   6  
   7  .sticky .awTags_TagLinks {
   8      visibility: hidden;
   9      display: inline;
  10  }

This also hides awtags from sticky nodes which I use at the top of tag descriptions.

I had 190 pictures to import into a Drupal based system. I had been avoiding using the image module as it is hard wired to upload one image at a time, give it a title and description etc and the whole thing would take hours.

I had been using Gallery2 till now because of the bulk picture import. Gallery2 is very sophisticated, has many features and is still beta software. I ran an update on my debian server and this managed to break Gallery2: any attempt to access the gallery gave a meaningless error and referred me to the gallery home page. Since the installation is quite complex, a mix of virtual directories, symbolic links and any number of other tricks to hide itself I couldn't be bothered to try to fix it. It is still beta so I'm not surprised it broke and it may well break again. Also it says in the blurb that it had not been through a security review and that did bother me.

I hadn't been impressed with any other gallery software I found so I decided to return to Drupal and the image module, if only because it is an integrated solution. Searching the drupal site, I came across the image_import module which allows what I wanted: bulk import of a directory full of pictures. I only had to install the image_import.module file, it seems I already have the 'walkah' version of the image module that is one of the requirements. I have only installed standard modules so the walkah one may be vanilla drupal 4.6 image module (it does say walkah at the top of the file).

It works as it said on the box, it imported all my files in minutes. It gave titles and descriptions based on the file names, e.g. DSC_123123.jpg but I don't really care about that. It has a nice 'trial run' feature that tests to see if the import is likely to work by checking file existence, permissions etc. It also has a timer facility to stop it from breaking any apache/php security rules that prevent long-running processes. My system limits a php run to 30 seconds so I had to set this time limit to 20 seconds and import my files in two or three chunks. This was not a big problem, the form settings were still filled in I just had to keep pressing the go button.

Two final problems, both with the image module:

• it would be nice to have 'prev' and 'next' links to walk through the pictures. Can this be rigged with the book module?
• drupal tries to theme the original images, although they are way too big to fit in the theme and look broken. It really only needs a link to the original jpg file. It is not important to me to show the title and description on the full image, it is mainly there for downloading. I can hack this one in myself.

I think I have found out what was mysteriously caching pages while I was editing drupal entries which I blamed on site5. It appears that Microsoft ISA firewall is caching pages, even if your proxy settings are not set up to use it. It also appears that it does not cache very well, if you look at a page, go to edit it, then return to the page, ISA serves up the old version from the cache and you have to refresh the page to see any changes. It's cache settings have a crude option marked 'Cache dynamic content (objects with question marks in the URL)' but I have nice url's set up so that doesn't do me any good.

I have set it to 'frequently expire' cache entries.

Moral: use squid which does not appear to have the same problem.

I have developed my strategy for putting drupal database dumps into subversion every day. This was slightly more complicated than I though it would be. In principle I could just use mysqldump to dump the sql and put that in subversion but the problem is that mysqldump by default will output the insert statements to reconstruct a table in a single line in the file, even of that line grows to 600k long. If subversion does a line-by-line diff on the files it stores it will end up writing all 600k to it's transaction log.

To minimise the sizes of each diff I dump the mysql in two parts, the data and the structure. For the data dump I do the following:

• use --skip-opt to ensure that each row is inserted on a seperate line: this limits the line length. The --opt option is the default for my version of mysql (4.1.13) so I have to turn it off.
• remove comments
• strip out data for the 'accesslog', 'cache', 'search_index' and 'sessions' tables. I don't think these need backing up.
• sort the remaining lines into alphabetical order as I am not sure the sql dump is guaranteed to always dump the rows in the same order

The structure is dumped straight, I certainly don't sort the lines in the file!

This creates sql data files that should diff pretty optimally.

Here is the bash script that does this for me:

   1  #!/bin/bash
   2  
   3  function SqlDumpData {
   4      mysqldump -u secret -psecret --no-create-info --skip-opt --comments=0 $1 | \
   5        egrep -v "INSERT INTO \`(accesslog|cache|search_index|sessions)\`" | sort >$2
   6  }
   7  
   8  function SqlDumpStructure {
   9      mysqldump -u secret -psecret --no-data $1 >$2
  10  }
  11  
  12  cd /home/peterc/DatabaseDumps
  13  
  14  SqlDumpData petersblog petersblog_data.sql
  15  
  16  SqlDumpStructure petersblog petersblog_structure.sql
  17  
  18  svn commit -m "daily backup"

This is all done on the server. Following this I can use my standard strategy to backup the subversion repository.

I have tested that the sql dumps can be reimported into mysql and give a functioning website.

I have put my drupal stuff under subversion for some source control so I can unify the source of three separate installations. In theory I could use drupal's virtual hosting features and have a single set of drupal files but:

• I want these installations to have seperate files and images directories and drupals virtual hosting does not support this
• I want apache2 to control all my virtual hosting.
• I want my own drupal hacks under source control

Some notes on how I did this:

• Create temporary copies of files I want submitted to subversion:

  mkdir ~/drupal
  mkdir ~/drupal/trunk
  cp -r /var/www/petersblog.org/* /var/www/petersblog.org/.htaccess ~/drupal/trunk
  

• remove stuff I don't want shared with other drupal installations:

  rm -r ~/drupal/trunk/files
  rm -r ~/drupal/trunk/images
  rm -r ~/drupal/trunk/sites
  rm -r ~/drupal/trunk/favicon.ico
  

• Put into subversion:

  svn import ~/drupal file:///repository_name/drupal -m "First Import"
  

• Get out of subversion again:

  mkdir /var/www/petersblog2.org
  cd /var/www/petersblog2.org
  svn checkout file:///repository_name/trunk/drupal .
  

• Since that worked ok, the copies of the files I created to put into subversion can now be zapped:

  rm -r ~/drupal
  

  If there is one thing I do not like about subversion it is this 'trunk' directory thing. I know it is not a requirement but just a convention but still.
• restore non-archived files

  cp -r /var/www/petersblog.org/files /var/www/petersblog2.org/files
  cp -r /var/www/petersblog.org/images /var/www/petersblog2.org/images
  cp -r /var/www/petersblog.org/sites /var/www/petersblog2.org/sites
  cp -r /var/www/petersblog.org/favicon.ico /var/www/petersblog2.org
  

• set up subversion to ignore these files that I don't want under svn control. After this running svn status will not list these files and subversion will not attempt to add them to the archive:

  svn propedit svn:ignore .
  files
  images
  sites
  favicon.ico
  

• commit changes, namely the changes to the svn:ignore properties

  svn commit -m "Ignore files I do not want shared"
  

• make sure my new copy has everything:

  diff -r --exclude=.svn /var/www/petersblog.org /var/www/petersblog2.org
  

• change to the new site archive

  cd ..
  mv petersblog.org petersblog_presvn.org
  mv petersblog2.org petersblog.org
  

• list anything that has changed:

  cd /var/www/petersblog.org
  svn status
  

• Get the code out on a new site:

  mkdir /var/www/newsite
  cd /var/www/newsite
  svn checkout file:///repository_name/trunk/drupal .
  

• After modifying the archive, get latest code out of subversion:

  svn update
  

• Put changes to code into subversion:

  svn commit -m "did some hacking"
  

this cheat sheet was useful.

They've found another hole in drupal's xmlrpc library and they have released a new version to fix it.

My advice: rename the xmlrpc.php file to something hard to guess and make sure noone can list your directorys. You can also use mod_rewrite to limit access to the file to known ip's:

#
# Don't allow remote xmlrpc
#
RewriteCond %{REMOTE_ADDR}       !^(1.2.3.4|127.0.0.1)$
RewriteRule ^/?xmlrpc.php         -   [L,F]

I fell victim to the last security hole and didn't update in time. Once bitten.

Something is starting to really irritate me. Whenever I make some change on this site through Drupal such as editing a page, I press the submit button and the browser shows me the page as it was before the edit. I have to press refresh to see the changes. This happens all over, if I look at the logs, add a page, look again at the logs, nothing has changed until I refresh the page.

Clues to the cause:

• I have caching disabled in Drupal and it should not be caching updated pages anyway.
• My browser is connected directly, no caching proxy.
• If I run ethereal I can see that the browser is sending a GET for the updated page and receiving the old version
• If I look at the site apache access logs I do not see this GET reaching the server

Something in between is caching the pages. J'accuse Site5. I've looked all through their site and I see nothing about this but the forums tend to be full of noobs (like most forums). I tried editing the .htaccess file to disable mod_cache if it was loaded but that just caused an internal server error. It is highly likely that they are running some form of cache the other side of apache (squid?), I'd probably do the same to save the load on a shared server for other peoples sites. However, I find it intrusive when I'm the victim.

Hosting contract is coming to an end, I still have time to evaluate a new hosting solution. I'm highly tempted by a cheap dedicated server. It's expensive but:

• I've plenty of power: I can run X and use a vnc terminal.
• I can do what I like in python. I have no interest in doing php at all and Site5's python support is weak (python 2.2.3, no generators ). They now support ruby on rails but I'm more interested in php than ruby (weak python clone).
• I learn to secure a linux box properly. I think my linode server was hacked through an xmlrpc weakness in drupal which has been fixed now.
• It's all mine. With VPS's and shared hosting, you are having to share disk and cpu with other people. I would only have to share network bandwidth with them.
• I can sell CPU time/hosting/web sites should I feel the desire

One thing about dedicated servers: I cannot believe that to get the server rebooted you have to email some guy who has to run to the box and press the reset button. How primitive is that? If I screw things up to the extent that shutdown -r now does not work I will either learn to prototype hacks on a local box or give up and become an estate agent.

UPDATE: er, it wasn't site5's fault, it was Microsofts.

My Microblogging tool is coming together. This is my vision for a blogging tool that is:

• Fast to post with so posting is as lightweight as possible
• Supports tagging to organise the posts. I have decided that tagging is preferable to putting articles in a rigid tree hierarchy.

I decided on the following architecture:

• VIM as an editor. For me this is by far the quickest way to do data entry
• I create however many posts in an single page and post them in one go
• I use Vim's python scripting to send the file full of posts to the server via an xmlrpc call
• The server implements a python cgi script to process the file full of posts. It splits the files into seperate posts and submits them to Drupal one by one. Drupal is the backend database and is also a reporting tool.

Most of the processing is done by the cgi script. I could post directly from VIM to Drupal, post-by-post but the Drupal blogapi does not support awtags and I'd rather write some python to do it that struggle modifying the php code. The cgi script also supports reading back a days-worth of postings so I can also edit existing posts.

Using VIM I was able to create a nice syntax-highlighing definition to make the posts easy to edit. Here is an example of how it looks in VIM:

This is the title
    23 August 2005 12:34
    [some stuff]
    This is a posting in Vim. I can create stuff in VIM much faster than I
    can type it in textarea in a web browser.

    As posts support basic html markup, html tags are <i>syntax highlighted</i>.

    It will syntax-highlight python:
    <python>
    for i in range( 10):
        print "Python rox"
    </python>

    It shows blocks with a green mark in the first column.

    It will syntax-highlight php:
    <php>
    $strStr = "What is the dollar for?";
    </php>

    It marks out my verbatim blocks:
    <verbatim>
    This is preformatted.
    </verbatim>

The format of the posts is a title line followed by date/timestamp and a list of tags, followed by the post itself, indented to delineate it. For me the syntax highlighting is the killer feature, I am almost word-blind without syntax highlighting. I stole the idea fromwas inspired by The Vim Outliner.

I won't post all the code here as it's too big to post inline and it is highly likely that nobody else is interested anyway. It's not quite finished, I still have a few issues to resolve:

• VIM macros to help in posting, such as filling in the date.
• Support for multiple blogs: work blog, personal note blog and this blog. How to cross post?
• Any way to handle images/attachments?

I call it the Microblog, micro to emphasise the fine-grained nature: not big laborious articles (like this one) but short pithy notes.

Here is an endorsement for the Drupal Admin Block Module.

This simply adds a block to the page to tell me if there are comments ready to be moderated. This saves me from going to the administration menu to scan the log or going to administration/comments/approval queue. This isn't the biggest chore in the world but the Admin Block module has removed it for me anyway. The block only shows itself to me (as administrator) if there are comments to approve.

I have hacked my comment module to refuse any comment that attempts to create a link: no http, www, whatever. By refuse I mean it fails to validate, it doesn't get as far as the approval queue. This has so far proved quite effective at eliminating comment spam. Real users can read the error message and be reassured that I will edit a link to make it work when I approve the post. Any captcha haters reading, this should even work for blind people.

I have some ideas for the note-taking and organising tool I want. My thoughts have been inspired by Dave Winer's OPML editor.

I started blogging as a way of organising notes but that has gone in an awkward direction: I write mini-articles and this takes time. What I am thinking of is the ability to just chuck down a sentence or two, maybe a web link, and just store it. Associated with these jottings would be a few tags that would enable me to organise them. The jottings would not necessarily have titles, I don't want to waste time thinking of a title. The intention of all this is to be light-footed, nothing to slow down the process of using it: the main reason I don't take down as many notes as I should is that I can't be bothered. The OPML editor is close in terms of being able to bang down notes with no titles but I have yet to explore it's categorising/tagging facilities.

One way I could use the OPML editor would be to compose an OPML file full of each days notes. I don't want to mess with a different file for each thing I work on, I'd rather use categories/tags to do that. At the end of the day the OPML could be consolidated into Drupal or whatever for archiving/viewing/editing as ultimately OPML is not a database. Posting articles directly into Drupal is too laborious, especially over the net: I don't want this to take more than a second or so. Another advantage of storing the articles in Drupal is that I can use my wilki syntax to add syntax highlighting.

I have to think about this some more.

Update: from the OPML Editor notes:

The core purpose of this program is to create outlines and share them with other people, in various forms.

I played with Mambo some more and decided that I didn't like it:

• the architecture is just plain weird. There are components (variations on pages, like a page to show poll results etc), modules (essentially simpler these just generate content for blocks) and mambots (filters). The distinctions just seemed arbitrary to me, a sign of bad design. I prefer Drupal's system of just modules that can do any or all of these things (although, when I think about it, why can't I redirect the output of a node to a block?).
• The module/component thing got to me because by the default Mambo shows an opinion poll block on the right of the page. I decided to try repositioning it to the left. For the life of me I could not work out how to do this: because poll is a compunent and not a module there is not a simple option in the admin menus to say where you wanted it to go. After I while I got fed up and decided I didn't want an opinion poll anyway (who cares what other people think) so I deleted the poll component: and I still got the opinion poll block on the left! I couldn't get rid of the damn thing. I had caching turned off, the cache directory was empty, argh.
• Mambo has an administration option to install themes from a tgz file or directory on it's server. The installation from directory did not work for me, it kept getting the install path wrong and could not find the index.php file. The template I was trying to install came in a zip file so I tried converting it to a .tgz and installing that way but it moaned that the tgz was corrupt (which may well be true).
• Drupal allows pages and blocks containing php to execute. This is a nice quick and easy way of customising it. I didn't see how this could be done in Mambo (except maybe by installing modules/components).
• Drupal has a set of modules available on the drupal site, all cleary listed on a single long page with descriptions and download links. Mambo points you to http://www.mamboforge.net a site that gives immediate information overload and you can't find anything.

I may come across as a bit of a noob being unable to do what I wanted in Mambo but please bear in mind that I have installed and configured Drupal many times without such frustrations.

I was mainly interested in Mambo because of the excellent themestemplates available. I came up with a better idea: port the themes I wanted to Drupal. The Mambo template mechanism is very similar to Drupals phptemplate so I got the best of both worlds, the nice look of mambo and the nice design of Drupal. It just required replacing the Mambo inline php with Drupals.

Was moved to look at Mambo Open Source for use on some non-blog websites. It is actually quite appealing compared to Drupal because of the quality of the templates available for it. The sites look really profesional compared to Drupal sites (not that this site is a good example). You can even buy good-looking sites for $25 or so, a fraction of the price of DreamWeaver ($400).

Mambo has a really nice feature to allow many boxes to be defined at various places around the page template and then you can set (to a limited extent) what goes in each box. Compare to Drupals choice of putting stuff in left or right sidebars.

Mambo cannot do blogs out-of-the-box as there is no support for comments on articles. There may be an add-on for it (Component? Module? Mambot? mambo has all three) but it would have to be good to tempt this blog away from Drupal.

Update: Drupal has Flexiblock which allows you to spread your blocks far and wide.

Got my hands on a new server box for work. It was an anonymous beige box but it turned out to have an AMD Duron 1G with 256M ram. Compared to the Pentium MMX 233 with 96M I was using it flies. The old box took about 4 seconds to bring up a drupal page, the new box takes about a second. My desktop pc at work is a pentium 2 450M with 256M ram: just about bearable running Windows XP.

I tried cloning the disk from my old server but the new server would not boot so I just did a clean ubuntu hoary install. Since it was a decent box I did a desktop install rather than a headless server. I copied over the drupal installation of the intranet easily enough: moving drupal is really easy, dump the database, copy /var/www, load the database, that's about it.

VNC server running on the new server is just brilliant. It really is good enough to wean me off ssh. Even running the client here at home on my laptop through an ssh tunnel it is fast enough to play tetris on.

I was doing some stuff on the Drupal blogapi and noticed that they have changed the interfaces slightly in a way that broke an old favorite posting. The newPost and editPost functions now take a node type as a first argument instead of a blank string.

Here is updated code:

   1  def PostBlog():
   2      import xmlrpclib
   3      import re
   4  
   5      #
   6      # If first line contains a blog entry ID then edit existing post,
   7      # otherwise write a new one.
   8      #
   9      nFirstLine = 0
  10      strID = vim.current.buffer[0]
  11      if not re.match( '^\d+$', strID):
  12          strID = ''
  13      else:
  14          nFirstLine = 1
  15  
  16      strTitle = vim.current.buffer[nFirstLine]
  17      strText = "\n".join( vim.current.buffer[nFirstLine+1:])
  18  
  19      oDrupal = xmlrpclib.ServerProxy( strDrupal + '/xmlrpc.php')
  20  
  21      oPost = { 'title': strTitle,
  22                  'description': strText}
  23  
  24      if strID == '':
  25          strID = oDrupal.metaWeblog.newPost( 'blog', strUserName, strPassword, oPost, 1)
  26      else:
  27          bSuccess = oDrupal.metaWeblog.editPost( strID, strUserName, strPassword, oPost, 1)
  28  
  29      print "Posted entry %s" % strID
  30  
  31      #
  32      # Don't intend to write posts to disk so unmodify the buffer and
  33      # allow easy quit from VIM.
  34      #
  35      vim.command( 'set nomodified')
  36  
  37  def ReadBlog( strID = None):
  38      import xmlrpclib
  39  
  40      oDrupal = xmlrpclib.ServerProxy( strDrupal + '/xmlrpc.php')
  41      if strID:
  42          oBlog = oDrupal.metaWeblog.getPost( strID, strUserName, strPassword)
  43      else:
  44          oBlogs = oDrupal.metaWeblog.getRecentPosts( 'blog', strUserName, strPassword, 1)
  45          oBlog = oBlogs[0]
  46          strID = oBlog['postid']
  47  
  48      vim.current.buffer[:] = []
  49      vim.current.buffer[0] = strID
  50      vim.current.buffer.append( oBlog['title'])
  51      vim.current.buffer.append( '')
  52      for strLine in oBlog['description'].split('\n'):
  53          vim.current.buffer.append( strLine)

Instructions here.

If you can read this then it is working again.

I realised that a lot of my old postings were not tagged by awtags because I hadn't been through them to categorise them and, worst still, I couldn't be bothered. This meant that the postings weren't indexed unless someone went way back through the blog history.

I decided to create a new tag for them called untagged. I wrote a python script to look for awtags with no tags assigned and to assign them to this new tag. I used python because I am far more confident in it than I am in php. There is not much point in making this a module or anything because it only needs doing once if I am methodical about giving tags to new postings. I could also have done this in raw SQL but the version of MySQL on Site5 does not support nested selects.

Once the 'untagged' tag is in place the only chore is to remove this tag from postings that I generate new tags for using my search facility. This can be done through the awtags administration interface (e.g. search for tag 'whatever' and remove tag 'untagged'). Also, now I can easily list the untagged articles, it is much easier to see what tags need adding.

   1  #
   2  # Assign a tag to nodes with no awtags
   3  #
   4  import MySQLdb
   5  import DBTable
   6  
   7  o = MySQLdb.Connect( 'localhost', '<mysql user name', '<password>', '<mysql database name>')
   8  
   9  oAwNodeDB = DBTable.DBTable( o, 'awtags_node')
  10  
  11  oAwNodeDB.Select()
  12  
  13  oTaggedNodes = {}
  14  
  15  while 1:
  16      oRow = oAwNodeDB.FetchOne()
  17      if not oRow:
  18          break
  19      oTaggedNodes[oRow['nid']] = 1
  20  
  21  oNodeDB = DBTable.DBTable( o, 'node')
  22  
  23  oNodeDB.Select( "SELECT * FROM node WHERE node.type = 'blog'")
  24  
  25  oNodes = {}
  26  
  27  while 1:
  28      oRow = oNodeDB.FetchOne()
  29      if not oRow:
  30          break
  31      oNodes[oRow['nid']] = 1
  32  
  33  for nNid in oNodes.keys():
  34      if not oTaggedNodes.has_key( nNid):
  35          oDict = { 'nid': nNid,
  36                          'tid': 84}
  37          oAwNodeDB.Insert( oDict)

This assumes the 'untagged' tag has a tid of 84: you should create your own tag and see what number it is.

This uses the DBTable module I wrote a while back. I discovered to my delight that the python odbc and MySQLdb modules had virtually identical interfaces so this module worked largely unchanged. I had to tweek it a bit because the field types were recorded as numbers instead of strings. Here is the modified version. It should work with odbc as well.

   1  #
   2  # Database wrapper class.
   3  #
   4  class DBTable:
   5      """
   6      Wrapper for database table
   7      """
   8      FIELD_TYPE = 0
   9  
  10      def __init__( self, oConnection, strTable):
  11          self.oConnection = oConnection
  12          self.strTable = strTable
  13  
  14          oCursor = oConnection.cursor()
  15          oCursor.execute( "SELECT * FROM %s" % strTable)
  16  
  17          self.oFields = [ oField[0] for oField in oCursor.description]
  18          self.oFieldDescription = dict( [ (oField[0],
  19                                   oField[1:]) for oField in oCursor.description])
  20  
  21      def Select( self, strQuery = None):
  22          """
  23          Select records from query
  24  
  25          Takes either SQL of select statement or a dictionary containing
  26          field names and values to find.
  27          """
  28          self.oCursor = self.oConnection.cursor()
  29          if strQuery == None:
  30              self.oCursor.execute( "SELECT * FROM %s WHERE 1" % (self.strTable))
  31          elif type( strQuery) == type(""):
  32              self.oCursor.execute( strQuery)
  33          else:
  34              #
  35              # assume query is a dict
  36              #
  37              self.oCursor.execute( "SELECT * FROM %s WHERE %s" % (self.strTable,
  38                                                       self.DictToWhere( strQuery)))
  39  
  40      def FetchOne( self):
  41          """
  42          Get next row of results
  43          Returns a dictionary holding field names and values.
  44          """
  45          oRow = self.oCursor.fetchone()
  46          if oRow:
  47              """
  48              Build a dictionary to map field name->value
  49              """
  50              return dict([(self.oFields[i], oRow[i]) for i in range(len(oRow))])
  51          else:
  52              return None
  53  
  54      def Insert( self, oDict):
  55          """
  56          Insert a row in the database
  57          Takes a dictionary holding field names and values.
  58          """
  59          strFields = oDict.keys()
  60          strValues = []
  61          for strField in strFields:
  62              strValue = oDict[strField]
  63              strType = self.oFieldDescription[strField][DBTable.FIELD_TYPE]
  64  
  65              strValues.append( self.FormatField( strField, strValue))
  66  
  67          strSQL = "INSERT INTO %s ( %s) VALUES(%s);" % ( self.strTable,
  68                                                          ", ".join( strFields),
  69                                                          ", ".join( strValues))
  70          print strSQL
  71          oCursor = self.oConnection.cursor()
  72          oCursor.execute( strSQL)
  73          self.oConnection.commit()
  74  
  75      def Update( self, oDictWhere, oDictNew):
  76          """
  77          Update a row in the database
  78          Takes a dictionary holding field names and values to find
  79          and dictionary to replace it with.
  80          """
  81          strFields = oDictNew.keys()
  82          strValues = []
  83          for strField in strFields:
  84              strValue = oDictNew[strField]
  85              strType = self.oFieldDescription[strField][DBTable.FIELD_TYPE]
  86  
  87              strValues.append( "%s = %s" % (strField, self.FormatField( strField, strValue)))
  88  
  89          strSQL = "UPDATE %s SET %s WHERE %s;" % ( self.strTable,
  90                                                    ", ".join( strValues),
  91                                                    self.DictToWhere( oDictWhere))
  92          print strSQL
  93          oCursor = self.oConnection.cursor()
  94          oCursor.execute( strSQL)
  95          self.oConnection.commit()
  96  
  97      def InsertOrUpdate( self, oDictWhere, oDictNew):
  98          """
  99          Seek record in database, add it if not found, update it if found.
 100          """
 101          self.Select( oDictWhere)
 102          if self.FetchOne():
 103              self.Update( oDictWhere, oDictNew)
 104          else:
 105              self.Insert( oDictNew)
 106  
 107      def Delete( self, oDict):
 108          """
 109          Delete row based on dictionary contents
 110          Takes a dictionary holding field names and values.
 111          """
 112          strSQL = "DELETE FROM %s WHERE %s;" % ( self.strTable, self.DictToWhere( oDict))
 113  #        print strSQL
 114          oCursor = self.oConnection.cursor()
 115          oCursor.execute( strSQL)
 116          self.oConnection.commit()
 117  
 118      def DictToWhere( self, oDict):
 119          """
 120          Convert dictionary to WHERE clause.
 121          """
 122          strFields = oDict.keys()
 123          strExpressions = []
 124  
 125          for strField in strFields:
 126              strValue = oDict[strField]
 127              strType = self.oFieldDescription[strField][DBTable.FIELD_TYPE]
 128  
 129              strValue = self.FormatField( strField, strValue)
 130  
 131              strExpressions.append( '%s = %s' % (strField, strValue))
 132  
 133          return " AND ".join( strExpressions)
 134  
 135      def FormatField( self, strField, strValue):
 136          """
 137          Format a field for an sql statement.
 138          """
 139          strType = self.oFieldDescription[strField][DBTable.FIELD_TYPE]
 140          if strType == 'STRING':
 141              return "'%s'" % str(strValue).replace( "'", "''")
 142          elif strType == 'NUMBER' or strType == 3:
 143              return '%d' % int( strValue)
 144          else:
 145              #
 146              # Treat as a string.
 147              #
 148              return "'%s'" % str(strValue).replace( "'", "''")
 149  

I noticed I broke the syntax highlighting in my wilki module when I moved the site to Linode. I forgot to change a hard-coded link to the geshi syntax highlighting library. Fixed now and I took the opportunity to upgrade to the latest version of Geshi and, I'm pleased to say, it Just Worked, didn't need any code changes from me.

Geshi is a very nice library, for me it has only two problems:

1. it is php, not python
2. it does not appear to have a mode for highlighting generic config files as does vim.

Silly things like fixing syntax highlighting can cause duplicate postings to appear in RSS aggregators as they nievely think the articles have been edited. Sorry for this, there is not much I can do about it.

I needed to set up another drupal site on my ubuntu linode. I had a domain name, I wanted to make it an independent site. I decided to keep it seperate from my existing site by putting in a fresh Drupal 4.6.1 installation and not to use Drupals virtual server facility.

I knew Apache2 supported virtual hosting and I decided to use that. I tried creating a new virtual host by creating a file in /etc/apache2/sites-available as follows:

<VirtualHost *>
        ServerName www.site2.com
        ServerAlias site2.com
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/site2
        <Directory /var/www/site2/>
                Options Indexes FollowSymLinks MultiViews
                # pcw AllowOverride None
                AllowOverride All
                Order allow,deny
                allow from all
                # This directive allows us to have apache2's default start page
                # in /apache2-default/, but still have / go to the right place
                # Commented out for Ubuntu
                #RedirectMatch ^/$ /apache2-default/
        </Directory>

        ErrorLog /var/log/apache2/site2/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/site2/access.log combined
        ServerSignature On

</VirtualHost>

where site2 is the name of the new site. Note that I created /var/log/apache2/site2 so that the site would get it's own access logs.

I used the command

a2ensite site2

to enable the site. I restarted apache2 and, bang, both this site and the new site showed the new site, I had broken this site.

After faffing around and googling, I tried a simple experiment. I removed the symbolic link to site2 from /etc/apache2/sites-enabled created by a2ensite and I just appended the above file to /etc/apache2/sites-available/default. I restarted apache2 and this worked, I had two sites. This is probably not the right way to do it but it works and any time I spend fixing it will bring this site down which bothers me so I'll leave it as it is unless I come across the correct way to do it.

Update: On my oneandone server running debian this is working fine as a seperate file, enabled with a2ensite:

<VirtualHost *>
    ServerName petersblog.org
    ServerAlias petersblog.org *.petersblog.org
    ServerAdmin webmaster@localhost

    DocumentRoot /var/www/petersblog.org
    <Directory />
        Options FollowSymLinks
        AllowOverride All
    </Directory>
    <Directory /var/www/petersblog.org>
        # pcw No directory listsings
        # Options Indexes FollowSymLinks MultiViews
        Options -Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
    </Directory>

    ErrorLog /var/log/apache2/petersblog.org/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog /var/log/apache2/petersblog.org/access.log combined
    ServerSignature On

</VirtualHost>

I have four sites set up like this and all are working.

I've moved this site to my Linode. I have a couple more months before my Site5 account runs out so I have time to move back if necessary. I moved the site as follows:

• Dump the sql:

  cd ~/www
  mysqldump -u <user> -p drupal > pb.sql
  

• Zip the useful stuff:

  zip -r pb cron.php database/ fail.html favicon.ico files/ images/ includes/
  index.php misc/ modules/ pb.sql robots.txt scripts/ sites/ themes/ tmp/ xmlrpc.php
  

  Will I ever learn to love tar?
• copy the zip to new site by running sftp on new site.
• unzip the zip file in /var/www
• load the database into mysql:

  mysqladmin create drupal -u <user> -p
  mysql -u <user> -p drupal < pb.sql
  

• go to domain registrar and change DNS address of site from site5 to linode. It only took three hours or so for this to propogate enough for me to access the linode via the petersblog.org name.

And if you are reading this then you are reading the site from the Linode as I haven't posted it on the old site. I can leave the old site floating for a while until the old address is flushed from all the DNS caches out there. I'll leave bisiand.me.uk pointing there until the account expires: unless of couse something happens to convince me to stick with Site5.

I have no problems with Site5, I would recommend them to anybody, it's just that Linode is more technically challenging and hence more interesting and fun.

I updated my linode to Ubuntu hoary hedgehog. I followed the upgrade steps here except I didn't install the ubuntu-desktop package as its a GUI-less server (although I may regret that one day).

Notes:

• it only took a few minutes to upgrade 250 packages.
• installing one of the packages failed with:

  udev requires a kernel >= 2.6.8, upgrade aborted.
  dpkg: error processing /var/cache/apt/archives/udev_0.050-3ubuntu7_i386.deb (--u
  npack):
   subprocess pre-installation script returned error exit status 1
  Errors were encountered while processing:
   /var/cache/apt/archives/udev_0.050-3ubuntu7_i386.deb
  E: Sub-process /usr/bin/dpkg returned an error code (1)
  

  which appeared to be because I am running a 2.4 kernel. I am not sure how free I am to change kernels on a Linode. I ignored the error and life went on.
• libphp4 module installation failed because it could not find a php.ini file. I create one for it to fiddle with and this time it was happy.
• I did shutdown -r now to reboot but this only powered down the linode. I had to go into the Linode control panel to boot it again.
• It came up and I was happy. If it hadn't I would have to do a clean install. I MUST get around to adding a rescue partition on the virtual hard disk.
• It had lost the hostname and I had to use the hostname command to reset it.
• It had decided to upgrade me to apache2 and install the generic index.html which overrode my drupal index.php so I deleted it.
• I had to enable php in /etc/apache2/apache2.conf by uncommenting:

  AddType application/x-httpd-php .php
  

• I had to enable mysql and gd in /etc/php4/apache2/php.ini by uncommenting:

  extension=mysql.so
  extension=gd.so
  

• It has python 2.4
• It still has subversion 1.06 I wanted to try out 1.2 which is why I upgraded Ubuntu.

awTags was adding an entry to the navigation menu called 'My Tags'. This was irritating me because it was presented to anonymous users and was the only reason for the navigation menu to appear. Looking in 'awtags.module', there are no options to control it so I changed the source so it will only appear for logged-in users:

   1  /*
   2   * Implementation of hook_menu
   3   */
   4  function awTags_menu($may_cache) {
   5    global $user;
   6  
   7    $items = array();
   8  
   9    if ($may_cache) {
  10  
  11      // pcw: only logged in users can have 'my tags'
  12      if( $user->uid) {
  13         // /usertags/tags (my tags)
  14         $items[] = array(
  15         'path' => "usertags/$user->uid",
  16         'title' => t('my tags'),
  17         'access' => user_access('access tags'),
  18         'callback' => '_awtags_page',
  19         'callback arguments' => $user->uid,
  20         'type' => MENU_DYNAMIC_ITEM);
  21     }
  22  
  23     ... rest of function unchanged.
  24  }

I tested this in IE where I am anonymous (like all IE users) and no change. Forgot to flush the damn Drupal cache for the umpteenth time: the menu's are cached. I took the time to knock up a php script to flush the cache for me so I don't have to fiddle with the mysql command line:

<?php
include_once 'includes/bootstrap.inc';
include_once 'includes/common.inc' ;

db_query('DELETE FROM {cache}');

echo( "Done");

?>

Save the above in a file called FlushCache.php on your server and just open it in a browser to flush the cache. It may be advisable to set up your .htaccess so that only you can access the file:

<Files "FlushCache.php">
  order deny,allow
  deny from all
  allow from [my ip address]
</Files>

Found a problem searching Drupal sites that use 'clean urls'. It even happens on the main Drupal site. Just do a search for 'die/die' and you get the following error:

Not Found
The requested URL /search/node/die/die was not found on this server.

Apache/1.3.33 Server at drupal.org Port 80

Clean urls requires a mod_rewrite hack to turn

http://www.drupal.org/search/node/die/die

into

http://www.drupal.org/index.php?q=search/node/die/die

but this is not working properly. If you try the second form above in a browser then it works, the search is done but mod_rewrite does not seem to correctly munge it.