Peter's Blog

This and this aren't about me. There are a surprising number of illustrious figures with that name though.

Gave qumana a try. It is a blog editing application that posts entries using whatever xmlrpc interface may be available. Here is the bullet list:

• very wysiwyg, with all the advantages and disadvantages that follow: easy to do simple/pretty stuff but interferes horribly with wilki formatting.
• easy to add adverts to your posts, should you be that concerned with making money.
• supports posting with tags and adds it's own tag by default, which upsets some purists who haven't figured out how to disable it.
• supports image uploading which had a silly bug when I tried it: if you don't select an image file it still tries to upload it and locks up.
• it doesn't honour the ancient ctrl-ins, shift-ins copy and paste keys I prefer. This may be a consequence of being written in java which also makes it boot sluggishly.
• comes with a 'drop pad' onto which you can drag and drop stuff for posting: very nice, except you cannot drag the drop pad onto a second monitor (so time for a 30inch wide screen).

For me it interfered with my wilki formatting too much to be useful on this blog. If I was happy to let it take care of the formatting then it would be just fine.

There are three ways to blog from a pocketpc that I have found:

1. open your blog in a web browser and edit it as normal. This doesn't work so well in practise unless your web page has been designed with a css style sheet that works nicely on a pda. I tried modifying my drupal theme accordingly and made it presentable but it's not up to data entry.
2. try using an application for posting to blogs. There are a couple of these about but those that I tried were buggy or had tiny little edit boxes.
3. write the article and submit by email. This is how I am posting this, I use phatnotes, a notetaking application that can send the notes via email. I don't use pocket outlook for the simple reason that it doesn't seem to save sent mail so I cannot edit the article and send it again to modify it. I use my old mailbot script on the server to capture posts and poke them into drupal.

So far this has worked ok and I can compose posts offline and upload them at my leisure.

Disadvantages:

• No preview unless I go online, hence I keep posts simple.
• cutting and pasting urls is fiddly and I cannot be bothered with it.
• my script adds tags by scanning the database for existing tags, searching the post for the tag words and adding the tags that it finds. This can give irrelevant tags and I cannot define new tags. I need a neat way to specify tags in the post.

Advantages:

• the convenience of whipping my pda out and having a quick blog. I don't have to go upstairs and get my laptop.
• can lay on settee and blog in comfort.

I've moved this site to my Linode. I have a couple more months before my Site5 account runs out so I have time to move back if necessary. I moved the site as follows:

• Dump the sql:

  cd ~/www
  mysqldump -u <user> -p drupal > pb.sql
  

• Zip the useful stuff:

  zip -r pb cron.php database/ fail.html favicon.ico files/ images/ includes/
  index.php misc/ modules/ pb.sql robots.txt scripts/ sites/ themes/ tmp/ xmlrpc.php
  

  Will I ever learn to love tar?
• copy the zip to new site by running sftp on new site.
• unzip the zip file in /var/www
• load the database into mysql:

  mysqladmin create drupal -u <user> -p
  mysql -u <user> -p drupal < pb.sql
  

• go to domain registrar and change DNS address of site from site5 to linode. It only took three hours or so for this to propogate enough for me to access the linode via the petersblog.org name.

And if you are reading this then you are reading the site from the Linode as I haven't posted it on the old site. I can leave the old site floating for a while until the old address is flushed from all the DNS caches out there. I'll leave bisiand.me.uk pointing there until the account expires: unless of couse something happens to convince me to stick with Site5.

I have no problems with Site5, I would recommend them to anybody, it's just that Linode is more technically challenging and hence more interesting and fun.

blog, a short for of 'web log', a kind of diary that 100 million people can have access to, but maybe a dozen actually read.

Two concepts I have been contemplating recently are starting to blur together: outliners and tags.

I have read of people who have moved from Outliners to Wikis as a means of organising their notes. I have found Wiki's rather simplistic and unattractive, especially as web-based editors are sluggish compared to desktop applications.

However, now I have changed my site to use awTags I see how tags can be used to implement a wiki. But this is better than a wiki: a link leads to a list of related articles rather than just a single page. In the manner of an outliner, an article leads on to a list of child articles according to which tag you follow. The article itself can be tagged with a number of different tags representing different concepts that the article itself can be filed under and these links can be regarded as parent relationships. This is just what TheBrain is trying to do: Mind Mapping but without the fancy graphics.

I have altered my wilki module accordingly and have used it right there. By typing in something as simple as

[wilki|tags/wilki]

I created a link to all my wilki module related postings filed under the wilki tag. Alternatively, I could have linked to the wilki introductory article and from there a reader, once they know what the wilki module is, can follow the wilki tag if they so desire.

This is all pretty abstract and I will have to see if it is actually of any use in the real world. On a practical level it will be easier for me to reference other articles by linking to the tag name, rather than trying to find the drupal node number of a specific article I am thinking of. The downside is that going to the introductory article is probably a better pattern to follow, especially as articles are listed in reverse chronological order, putting the most informative introduction at the very end.

I've been contemplating some python web development. This is mainly for the following reasons:

• I'd like my blog to have features for threading subjects based on keywords, a kind of related articles list. This would be dynamically updated, old articles could link to newer articles. This is possible in Drupal but the taxonomy system is laborious to configure through the web interface. There is a new keyword feature (folksonomies) but it appears to be a library for other modules to use. There are some modules that promise this kind of thing but they are not what I am looking for. The most important thing is that old articles are displayed with links to new articles, which involves messing around with themeing to add links to a cached page or figuring out how to flush the cache and put articles through a filter that would add the links.
• I'd rather spend my limited programming time being creative with technologies I am already familiar with. I don't like php enough to want to learn it any more. I know and like python.
• I don't want to spend my time trying to figure out how the Drupal code base works. Ok, I'm lazy and I have a short attention span and I'm not very interested in how it works.
• If I do this as a spare time project I want to enjoy it, I don't want it to feel like a maintenance exercise. I can do that at work.
• I have found that using librarys etc written by third partys can give you 95% of what you want, then you spend weeks trying to hack them to get the extra 5%. The only exception to this rule I can think of is the python standard library which is way cool.

So I looked at a few web application frameworks. This is a little lightweight and short on detail, I did most of it a few weeks ago and I didn't take notes.

cherrypy: Requires python 2.3 and my hosting service will only give me python 2.2 with cgi. I think generator functions are way cool and I'd love to use them. Do I stand any chance of getting my hosting service to upgrade? I'll email them and ask.

quixote: got the demo going, although I had to fix some of the urls within it as they appeared to be broken. It worked easily enough with cgi, the 1.3 version worked with python 2.2. However, again they have just released version 2 which has a couple of yield statements in it, limiting it python 2.3.

webware: I don't think I could see the advantage of 'servlets' and their web site (presumably powered by webware but it only says 'python powered') is not an impressive display of it's capabilities.

zope/plone: I've toyed with zope in the past (version 2) and found it's learning curve to be almighty. It may be very capable but you have to learn the zope way to do everything. The developers seem to be concentrating on inventing new object-orientated paradigms rather than keeping things simple and I hate huge complex object models. I don't like the web-based source editor (what, no syntax highlighting?: although I could use mozex and edit in vim). Also, non-cgi.

I was most interested in quixote until I came to the python 2.3 stumbling block with the new version 2. I don't want to settle for an old version. This led me on to think about what these frameworks actually give me:

• map urls to class/method calls
  • big deal
• session management
  • do I need it for my simple purposes?
• templating systems
  • Cheetah
• interface to cgi/mod_python/fastcgi/whatever
  • I can only have cgi anyway
• easy form generation
  • classes to build the contents of a form. How many forms will I need? Can't templates do this?
• already used on the internet and exposed to hackers
  • lets not be too ambitious with what we do. Keep drupal in the background for data entry.
• learning curves

Which has led me to... straight cgi. Run my own scripts from cgi, using the cgi module in the standard python library to get the url parameters and generate a nice error page when things go wrong. I can get it going easily enough and below, with just a few hours hacking, is something that can display nodes from the drupal database. The output is themed through the Cheetah template engine and gives me the same look as my drupal theme. Given that it is only cgi it is not massively scalable, but I understand every line of it and that means a lot in terms of development time. Given that it generates pages that look like the drupal pages, I can chop and change, leaving some bits to drupal (data entry) and others in python (presentation). It's a lash up but so what?

The main challenge is porting my wilki module to python, particularly the geshi syntax highlighting.

I am not planning to rewrite Drupal and I probably won't release Yet Another Python Web Framework. I may not even finish it. It is something for me to play with.

   1  #!/usr/bin/python
   2  #
   3  import Cheetah.Template
   4  import cgi
   5  import cgitb; cgitb.enable()
   6  import time
   7  import datetime
   8  import os
   9  import pdo
  10  
  11  def ThemeAndOutput( strPageTitle, strContent, oParams={}):
  12      "Theme the site"
  13  
  14      strSideBar = """
  15      <div class="block">
  16      <h2>Reports</h2>
  17      <ul>
  18          <li><a href="Intranet.cgi">Drupal Emulator</a></li>
  19      </ul>
  20      </div>"""
  21  
  22      oDict = {
  23          'Root': '/SMB',
  24          'Url': '/Intranet/Intranet.cgi',
  25          'Title': 'Drupal Emulator',
  26          'Slogan': 'There are no Problems, only Challenges',
  27          'PrimaryLinks': [ '<a href="http://intranet.org">Intranet</a>',
  28                              '<a href="http://www.google.co.uk">Google</a>'],
  29          'PageTitle': strPageTitle,
  30          'SideBar': strSideBar,
  31          'Footer': 'Copyright 2005 Peter Wilkinson'
  32      }
  33  
  34      #
  35      # Read page template and shove in the content.
  36      #
  37      strTemplate = open( 'PageTemplate.html').read() % { 'Content': strContent}
  38  
  39      #
  40      # Run template and content through cheetah in one go.
  41      # Is this faster? Doubt it.
  42      #
  43      oHtml = Cheetah.Template.Template( strTemplate, [oDict, oParams])
  44  
  45      print str(oHtml)
  46  
  47  def MainPage( oForm):
  48      "Main welcome page: tell user what is going on"
  49  
  50      #
  51      # Determine how many links to show.
  52      #
  53      strFrom = oForm.getfirst( 'from')
  54      if strFrom == None:
  55          strLimit = ' LIMIT 10'
  56      else:
  57          try:
  58              nLimit = int(strFrom)
  59              strLimit = ' LIMIT %d, 10' % nLimit
  60          except:
  61              strLimit = ' LIMIT 10'
  62  
  63      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  64      rs = c.open( 'SELECT * FROM node ORDER BY created DESC %s' % strLimit)
  65  
  66      strTitles = []
  67      while rs.next():
  68          strTitles.append( '<a href="Intranet.cgi?node=%s">%s</a>' %
  69                (rs.fields['nid'].value, rs.fields['title'].value))
  70  
  71      strPage = """
  72  <ul>
  73  #for $strTitle in $titles
  74  <li>$strTitle</li>
  75  #end for
  76  </ul>
  77      """
  78  
  79      ThemeAndOutput( "Welcome", strPage, { 'titles': strTitles})
  80  
  81  def NodePage( oForm):
  82      "Display a node"
  83  
  84      strNode = oForm.getfirst( 'node')
  85      if strNode == None:
  86          ErrorPage()
  87          return
  88      else:
  89          try:
  90              nNode = int(strNode)
  91          except:
  92              ErrorPage()
  93              return
  94  
  95      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  96      rs = c.open( 'SELECT * FROM node WHERE nid = %d' % nNode)
  97  
  98      if rs.next():
  99          strTitle = rs.fields['title'].value
 100          strBody = rs.fields['body'].value
 101      else:
 102          ErrorPage()
 103          return
 104  
 105      strPage = """
 106  <h2>$strTitle</h2>
 107  
 108  <pre>
 109  $strBody
 110  </pre>
 111      """
 112  
 113      ThemeAndOutput( strTitle, strPage,
 114            { 'strTitle': strTitle, 'strBody': strBody})
 115  
 116  def ErrorPage():
 117      strPage = """<p>There has been some kind of navigation error, the link
 118  you just clicked is broken</p>
 119  <p>Better <a href="mailto:someone@somewhere.com">complain about it</a>.</p>
 120      """
 121  
 122      ThemeAndOutput( "Drupal emulator", strPage)
 123  
 124  oForm=cgi.FieldStorage()
 125  
 126  node = oForm.getfirst( 'node')
 127  if node:
 128      NodePage( oForm)
 129  else:
 130      MainPage( oForm)
Toggle Line Numbers

The page template is kept in a seperate file. It is based on the phptemplate of my drupal theme and was ported over in less than an hour. Theming engines? A specialisation of templates.

   1  Content-Type: text/html
   2  
   3  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   4   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   5  <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
   6  <head>
   7  <title>$Title</title>
   8  <meta http-equiv="Content-Style-Type" content="text/css"/>
   9  <style type="text/css">@import url(style.css);</style>
  10  </head>
  11  <body>
  12      <div id="headerleft">
  13      </div>
  14      <div id="headermain">
  15          <div id="ie6hack">
  16          </div>
  17      </div>
  18      <div id="headertitle">
  19          <h1>
  20              <a href="$Url" title="$Title">
  21              $Title</a>
  22          </h1>
  23      </div>
  24      <div class="slogan">
  25          $Slogan
  26      </div>
  27      <div id="logo">
  28          <a href="$Url" title="$Title">
  29              <img src="ITL.gif"/>
  30          </a>
  31      </div>
  32      <div id="headerright">
  33      </div>
  34  #if len($PrimaryLinks) > 0
  35      <div id="primarylinks">
  36          <ul>
  37  #for $PrimaryLink in $PrimaryLinks
  38              <li>$PrimaryLink</li>
  39  #end for
  40          </ul>
  41      </div>
  42  #end if
  43  
  44      <div id="ie5forcemargininsidehack">
  45          <div id="main">
  46              <div class="tab">
  47                  <div class="tabbackground">
  48                      <div class="tableft">
  49                          <h3>
  50                              $PageTitle
  51                          </h3>
  52                      </div>
  53                  </div>
  54              </div>
  55              <div class="tabbody">
  56                  <div class="ieisbuggy">
  57                  </div>
  58                  <div id="content">
  59                      %(Content)s
  60                  </div>
  61                  <div class="ieisbuggy">
  62                  </div>
  63              </div>
  64          </div>
  65          <div id="footer">
  66              <div class="footerleft">
  67                  <p>
  68                  $Footer
  69                  </p>
  70              </div>
  71          </div>
  72      </div>
  73      <div id="sidebarright">
  74          <div id="sidebar">
  75              <div class="sidebarbox">
  76                  $SideBar
  77              </div>
  78          </div>
  79          <div id="sidebarbottom">
  80          </div>
  81      </div>
  82  </body>
Toggle Line Numbers

Update engineering log to Drupal 4.6.0.

Notes:

• Followed upgrade instructions, no big problems
• My Theme worked after installing phptemplate.
• Wilki module worked after reading and writing filter settings.
• Could not find 4.6 version of smileys module. Copied 4.5 version and edited smileys.module to change instances of drupal_specialChars to check_plain as described here.
• If this reaches my public blog then the sendhome module is working.

So far, see no big changes. It keeps moaning about some image library that I don't need not being installed and there are some icons in the admin logs. Looking forward to trying the improved search . I found google site search more useful than the old search but the engineering log is behind a firewall and beyond google's reach.

Latest candidate in my pursuit of the ideal Outliner is TreePad. This is the most appealing so far:

• two-pane outliner: tree hierarchy on the left, node contents on the right. Simpler than Microsoft OneNote with it's confusing mix of tabs levels.
• The node contents are rich text that can contain formatting, images, tables (unlike OneNote) etc. Node editor is virtually a word processor.
• Nodes may contain hyperlinks to executables, web pages, other nodes in other treepad documents (unlike OneNote and better than leo's cloning which only works in the context of a single document). Hyperlinks go in article body, unlike Leo where they go in the tree node title, not a good place to put it in my opinion.
• Tree nodes may have an icon associated with it. Leo loads it's selection of icons very slowly.
• Not the fastest program I have used: the help file is a 5M treepad document and takes a good time to load up (1.5G Centrino).
• Simple installation: one of it's features is that it can be installed on a USB flash key and carried around. I haven't found a licence agreement yet to tell me if it has to be licenced on each pc it is used on.
• Robust encryption. Dare I trust it with my credit card numbers and passwords?
• Can export in many formats including XML, OPML (which strips internal hyperlinks), html-ish files parsable by python. It is important to me that data can be extracted from any proprietary file format.
• Documents can be exported as entire websites, complete with Javascript driven tree. Kinda tempting to examine the possibility of organising a website around with this, carryng it in pocket, publishing it, searching it, refactoring it, writing it to pdf... I find blog notes the best place to record things but web interface is not the best editor. Could type stuff in here and bulk move it to the blog.

It is commercial (£20 ish) but it is very polished (unlike Leo).

Uploading images to this blog can be kinda fiddly. Say I have an image in the windows clipboard (e.g. a screen clip from OneNote and I want it on the blog, I have to fire up a graphics program, paste it in, save it to a file, fire up filezilla, upload it, delete the image file.

This little script does all this for me. It asks for a file name, writes the image in the clipboard to the file, uploads it and deletes the image. Job Done.

The uploading is done using pscp, a version of scp from the putty camp. It uses my putty private key so I don't have to give a password for access to the server (I didn't use the cygwin version of ssh as it didn't like having 'c:\' in file names). The script uses the Python Imaging Library to do the grabbing and wxPython to ask for the file name.

   1  #
   2  # Upload image in clipboard to server
   3  #
   4  import wx
   5  import ImageGrab
   6  import Image
   7  import os
   8  import tempfile
   9  
  10  class MyApp(wx.App):
  11      def OnInit(self):
  12          oImage = ImageGrab.grabclipboard()
  13  
  14          if not isinstance( oImage, Image.Image):
  15              #
  16              # In no way is wxPython an easy to use library.
  17              #
  18              dlg = wx.MessageDialog( None, 'No image in clipboard', 'Error', wx.OK)
  19              dlg.ShowModal()
  20              dlg.Destroy()
  21              return True
  22  
  23          dlg = wx.TextEntryDialog( None, 'Enter a file name', 'Upload Clipboard', '')
  24          if dlg.ShowModal() != wx.ID_OK:
  25              dlg.Destroy()
  26              return True
  27  
  28          strName = dlg.GetValue()
  29          dlg.Destroy()
  30  
  31          strTempFile = tempfile.mktemp( strName)
  32          oImage.save( open( strTempFile, 'wb'))
  33  
  34          strCmd = 'pscp -q -i "c:\my documents\puttykey.ppk" %s me@myserver.com:www/images/%s' % (strTempFile, strName)
  35  
  36          os.system( strCmd)
  37  
  38          os.unlink( strTempFile)
  39  
  40          return True
  41  
  42  app = MyApp(0)
  43  app.MainLoop()
Toggle Line Numbers

This is the result of alt-print screen and running this script:

The script itself inside VIM

ToDo:

• handle file names in clipboard
• detect upload errors
• allow resizing images

Read about the BBCode Firefox Extension which adds right menu for adding BBCode markup. I had already looked at BBCode and decided that it isn't much simpler than plain html markup and I'd rather only remember one of them. So I went looking to see if there is a similar firefox extension to do HTML markup for this blog and... the BBCode extension can be set up to do html markup as well.

Since I mentioned awstats on this blog I've been getting attempts to access the awstats.pl script on this site. awstats.pl is not accessable through this domain, it is provided by Site5 but I have to log in to netadmin to get to them.

Anyway, I had a quick search to see if there was a way to hack in via awstats and sure enough there is. The trick mentioned in this article is the one they are trying to get in with:

200.223.55.134 - - [11/Feb/2005:14:44:54 -0500] "GET
/stats/cgi-bin/awstats.pl?configdir=|echo%20;echo%20;id;echo%20;echo%20|
HTTP/1.0" 404 6186 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows NT 5.0)"

this is trying to execute the command id which shows the uid, gid and groups of the account it runs in. I guess this is probing for this vulnerability and seeing whether it gives root access.

The break-in attempts are coming from a variety of IPs, as is usual they are using proxys so there is no point trying to block them. They are getting 403s anyway, they aren't consuming much bandwidth.

Moral: keep an eye on your access logs, see what folk are up to.

Came in monday morning (when else) to find hard disk on Ubuntu server had gone down. It was giving repeated journal errors from the ext3 file system. I went to the IT dept to ask for a new hard disk and they gave me a brand new 40G Maxtor 6E040L0, a 40G 7200rpm beast. As this was better than the 16G IBM disk in my main PC which is suffering from about 5 years of accumulated crud I decided I would rather put the new disk in the main PC.

My old hard disk had Windows 2000 installed and years ago I was nieve enough to make it a 'dynamic disk' rather than a 'basic disk'. The advantages of a 'dynamic disk' are vague and only applicable to RAID arrays and suchlike. For a workstation there is no point and it becomes a liability as no tools like Norton Ghost, Partition Magic, Maxtor EZ-Drive or whatever work properly, they all run into limitations. After messing around trying to work out how to clone the Win2k partition to the new disk I decided upgrade to Windows XP on a clean install as I had a spare licence going unused on the old server PC. This PC had 96M of ram and XP is unusably slow on it.

I installed XP and it is wonderful, the system is so much faster and quieter than the cruddy old IBM disk.

XP salient points:

• would not recognise old non-plug and play DE220 ISA network card. Had to use a PCI DE528.
• Recognised Matrox 440 dual-head graphics card and ATI Rage IIC and offerred my three monitors. The graphic card setup dates back to Windows NT where the dual-head was a poor imitation of multi-head. I stuck with dual head using one of the Matrox channels and the ATI. I could plug another monitor in, for the hell of it, but:
  • I don't want to be irradiated.
  • Don't want to load the system to much as it only has 256M ram.
• although I was transferring the installation from a different PC I activated it online with no problems. I did wonder if I would have to ring Microsoft to justify it but no.

I am not sure that the old server hard disk is totally dead, I can read the partition table in my main PC but not in the server PC. I may be able to rescue the old installation, none of which is backed up. Thank god for this blog, where I noted down everything I did to set it up.

Search this site using Google.

WWW Peters Blog

Interesting to see that the new MSN Search can generate results in RSS format which is essentially XML and easy enough to crunch in python. You just submit a query and add '&format=rss' to the end, e.g.:

http://search.msn.com/results.aspx?q=peter's+blog&format=rss

gives this. What's useful about this?

• can interrogate a search engine without screen scraping (hastle, hoping web page format does not change) or using the google api (limited to 1000 searches/day). There may well be terms and conditions that will stop bots doing 10,000,000 searches a day.
• could set up aggregator to repeat search terms.
• could be a use for firefox's live bookmarks.

Searching MSN for 'peter's blog' puts this site first Google reserves this for Tom Peter's Blog.

This code goes in a Drupal block set to type 'php' and it shows a Drupal search form. When it loads it examines the 'referer' string that was used to find your site and, if the referrer was google, it will extract the query term and pre-populate the search box with it. This way your visitor can do a local search if they are not immediately inspired by what google led them to.

   1  <?php
   2  $referer = urldecode($_SERVER['HTTP_REFERER']);
   3  if (preg_match('|^http://(www)?\.?google.*|i', $referer)) {
   4    $query_terms = preg_replace('/^.*q=([^&]+)&?.*$/i','$1', $referer);
   5  } else {
   6    $query_terms = "";
   7  }
   8  
   9  $strHtml = <<< EOL2
  10  <form action="search" method="post">
  11    <div id="search">
  12    <input class="form-text" type="text" size="15" value="$query_terms" name="keys"
  13                            alt="Enter the terms you wish to search for." />
  14    <input class="form-submit" type="submit" value="Search" />
  15    </div>
  16  </form>
  17  
  18  EOL2;
  19  return $strHtml;
  20  ?>
Toggle Line Numbers

Here is an alternative version that shows a google site search box instead of a drupal one. It is better in that it uses googles search engine, which is more likely to work with the query term it arrives from, but it does put adverts on your results and relies on your site being thoroughly indexed.

The code below is set up to do a site search on my site so references to www.bisiand.me.uk should be changed.

   1  <?php
   2  $referer = urldecode($_SERVER['HTTP_REFERER']);
   3  if (preg_match('|^http://(www)?\.?google.*|i', $referer)) {
   4    $query_terms = preg_replace('/^.*q=([^&]+)&?.*$/i','$1', $referer);
   5  } else {
   6    $query_terms = "";
   7  }
   8  
   9  $strHtml = <<<EOL
  10  <!-- Search Google -->
  11  <center>
  12  <div style="text-align: center;">
  13  <FORM method=GET action=http://www.google.com/custom>
  14  <TABLE bgcolor=#FFFFCC cellspacing=0 border=0>
  15  <tr valign=top><td>
  16  <A HREF=http://www.google.com/search>
  17  <IMG SRC=http://www.google.com/logos/Logo_40wht.gif border=0 ALT=Google align=middle></A>
  18  </td>
  19  </tr><tr>
  20  <td>
  21  <INPUT TYPE=text name=q size=31 maxlength=100 style="width:100px" value="$query_terms">
  22  <INPUT type=submit name=sa VALUE="Google Search">
  23  <INPUT type=hidden name=cof VALUE="http://www.bisiand.me.uk">
  24  <input type=hidden name=domains value="www.bisiand.me.uk"><br>
  25  <input type=radio name=sitesearch value=""> Google
  26  <input type=radio name=sitesearch value="www.bisiand.me.uk" checked> Peters Blog
  27  </td></tr></TABLE>
  28  </FORM>
  29  </div>
  30  </center>
  31  <!-- Search Google -->
  32  EOL;
  33  return $strHtml;
  34  
  35  ?>
Toggle Line Numbers

I've been keeping an eye on my visitor logs to see how much my domain name problems have effected my traffic. According to Statcounter they had been climbing but yesterday there is a sudden dip. The Awstats logs provided by Site5 show no such dip.

I've seen a number of such dips in the Statcounter logs: their servers do not appear to be the most reliable. This is not a big complaint, I use them for free, more of a lamentation. Their professional service is too expensive for my simple ego brushing needs, $9 a month, but if I was paying that I would not want drop-outs approximately once a week.

The main advantage of Statcounter for me is that it counts visitors who have javascript enabled so it is essentially counting human beings rather than crawlers and referrer spam bots. It is also easy to set it up to ignore my own IP address. The Drupal statistics module does not have this feature but I could simply use phpmyadmin or another generic mysql database report generation tool to filter the drupal logs in any way I desire. The statistics module does list external referrers in reverse chronological order so it is useful for updating .htaccess referrer exclusion lists.

I checked the whois information for bisiand.me.uk and it turns out that it has been detagged, i.e. the old registrars have dumped it. It is in a limbo where it is still mine but I can't use it. I can claim it back by applying to nominet and sending them £15 + vat so that is what I am going to do. bisiand.me.uk is probably not dead, just in a coma.

I am getting bits and bobs of activity on bisiand.me.uk while the DNS fizzles out.

To get the new domain some pagerank I can just find the forums referencing my old site and post new items to say the site has moved.

I won't bother registering with sites like blogarama, they are just lists of blogs and they don't even have hard links which may give pagerank. I have registered with Feedster as that is a more significant and useful site. I should be pinging blog postings under the new domain name so blog traffic should grow.

I have a system set up whereby I can forward posts from my private work blog to this public blog. I've just got this system running again since changing the work server to Ubuntu and upgrading Drupal to 4.5.1.

The system works by having the work server email a posting to a python script on the server that uses the xmlrpc api to post the message. I came across two problems.

1. Ubuntu uses the postfix MTA. With the default setup email posted from php using the mail function just disappeared. I turned on php error logging by setting the following in /etc/php4/apache/php.ini:

   log_errors = On
   

   and was rewarded with the following entries in /var/log/apache/error.log

   sh: line 1: -t: command not found
   

   I worked out that this was due to the following in /etc/php4/apache/php.ini:

   ; For Unix only.  You may supply arguments as well (default: "sendmail -t -i").
   ;sendmail_path =
   

   I changed this to

   sendmail_path = /usr/sbin/sendmail -i -t
   

   and the problem was solved.
2. When the new post arrived in Drupal it was stored and published but the creation date was rubbish, sometime last year, so the posting was not at the top of the listing. This turned out to be something I have come across before so I changed the function blogapi_new_post in modules/blogapi.module in a similar way to how I did before:

      1    // check for bloggerAPI vs. metaWeblogAPI
      2    if (is_array($params[3])) {
      3      $edit['title'] = $params[3]['title'];
      4      $edit['body'] = $params[3]['description'];
      5      _blogapi_mt_extra($edit, $params[3]);
      6  
      7      //
      8      // PCW
      9      //
     10      if (array_key_exists('created', $params[3])) {
     11          $edit['created'] = $params[3]['created'];
     12      } else {
     13          $edit['created'] = time();
     14      }
     15    }
     16    else {
     17      $edit['title'] = blogapi_blogger_title($params[3]);
     18      $edit['body'] = $params[3];
     19      //
     20      // PCW
     21      //
     22      $edit['created'] = time();
     23    }
   Toggle Line Numbers

   This explicitly sets a creation date if none is found in the posting parameters.

Checking out my referrer logs and all the referrer spam I followed a few of the urls and it seems like a lot of them have been pulled by their hosting service: they all come up with the same page saying that the site has been pulled for abuse

If referrer spam bugs you as much as it does me then this is a useful site.

I was looking through my StatCounter logs looking at referrals from google. These give the search terms and it's interesting to see what people are searching for when they find my site. I noticed that someone in Australia was searching for 'StatCounter' itself. I checked with a google search and was surprised to find my post came third on the first page of results, after the StatCounter site at #1 and #2. This is odd, my pagerank is 2 (www.bisiand.me.uk, for some reason bisiand.me.uk is only 1) so why are my ramblings showing up? Does nobody else mention StatCounter?

I started a game, find random blog entry, pick a couple of keywords and do a google search. I found that out of about 5 goes, my blog came up in the first three pages of google results. Even something broad like 'Drupal' puts me on the second page.

So:

• do I only write about obscure topics that nobody else writes about
• does google indeed favour blogs over all those sites full of random search results and price comparisons
• it was a straw pole
• I chose the right keywords
• the googlebot recognises me as an authority on these things
• all the above
• all the first five but mainly the first one and very little of the fifth

I cannot get my post about getting msn desktop search to list exe files to appear on the first few pages of google. exe hits too many program names.

Right now my page loads are all taking a long time because the StatCounter server is not responding. Normally it's Technorati that drags me down.

Further to my Webcam Adventures there has been little progress. The development version of libusb-win32 that I am using has some problems:

• there seems to be a maximum transfer size hard coded into the library of 64 * 1024 which is not big enough for a video frame.
• it seems to do bulk transfers very slowly. What I get from the webcam appears to be the start of multiple frames rather than one complete frame.
• the same limit probably applies to isochronous transfers but whenever I try using usb_isochronous_setup_async my pc reboots. Each reboot takes about 4 minutes to come up which is incredibly tedious and frustrating.

While on the one hand it is flattering that a google search for 'libusb win32 isochronous' brings this blog up in second place, I also know that when that happens it means I am on the bleeding edge.

I use XMLRPC to update my blog from email. I use the python xmlrpclib to invoke the metaWeblog interface in Drupal.

The XML RPC interface can be accessed easily as follows:

>>> import xmlrpclib
>>> oDrupal = xmlrpclib.ServerProxy( 'http://www.my.web.site.com/xmlrpc.php')
>>> oDrupal.metaWeblog.getRecentPosts( '', '<username>', '<password>', 25)

From my server logs it seems that someone else has being trying to get in here so I've closed the door by editing the .htaccess file to prevent entry from the outside world:

  RewriteCond %{REMOTE_ADDR}       !^123.123.123.123$
  RewriteRule ^/xmlrpc.php          -   [F]

The above is saying that only the ip address 123.123.123.123 has access to the file xmlrpc.php. IP address is the address of my host.

For the first time in nearly a year my ntl broadband connection is down and I am having to blog via my mobile. I waited on the phone for 25 minutes to be told there is an outage in my area. When the 'ready' light on the cable modem stops flashing I am back in business, no idea when that might be.

Let's see:

strFred = "hello world"

Syntax highlighting from a mobile, cool.

Update: up again, about 6 hours later.

Had my first taste of a podcast. This is a kind of audio blog. The one I listened to was this. It comes across as being like a techy radio show, talking about mobile phones and the like. It was kind of interesting to listen to while I was on the rowing machine. I just downloaded the mp3 file and played it on the latest app to grab the association with mp3 files, Real Player in this case.

I could set up an aggregator capable of downloading these files as they appear and listen to them all the time but my rowing time is about the only time I have for this kind of thing. Rowing is like ironing, I need something interesting to watch on tv or listen to that will take my mind off my suffering.

Don't worry, I won't be generating podcasts any time soon.

Upgraded work engineering log to Drupal 4.5.0. Here's what I did:

• Followed the upgrade instructions (duh!)
• copied the contents of the bluemarine theme directory to a new directory and replaced the .css file with my old xtemplate themes .css file.
• Tweeked the template so that the main title is not a link and is aligned to the right of the logo, not under it (i.e. added align="left" to the image properties).
• added an entry for the new site-name style to change the colour and text size of the main title.
• Altered my wilki.module file to work with drupal 4.5.0. Mainly copied how it was done in the textile module as the drupal developer documents were a bit vague. However, for a filter it only means defining two functions.
• Altered my sendhome.module to work. This was mainly implementation of the sendhome_menu function as described in the developer documents.

In the boilerplate _menu hook function there is a line that goes:

    if ($may_cache) {

This line stops the simple callback for the /sendhome page from working. It's ok if I comment it out so I'll leave it at that for now. This module is only for me to forward articles from the work blog to my personal blog.

The upgrade gave me a chance to prune out modules I don't use. It seems to be running ok, the improvement over 4.4.0 is not remarkable but there are some nice new features.

Someone tried to comment-spam me with Seraphim Proudleduck stuff. All comments on my site go into approval by moi before the wider internet audience and specifically the google bot can see them. I don't get enough comments for this to be a bind and being able to kill comment spam makes up for the hastle.

I had a posting that should have started 'my screen' but I missed the c out. The resulting phrase was top listed by google for some reason and led to regular visits. Ok, I want visits but totally wasting peoples time with an article about screen protectors is not going to get me pagerank 7 real quick.

It bothered me that I could not edit old entries without telling the world about it. This may be ok for updates but I like to be able to correct typos in old entrys.

I changed ping.module so I only ping when new blog entrys are created:

   1  function ping_cron() {
   2    global $base_url;
   3  
   4    if (variable_get("site_name", 0) && variable_get("site_slogan", 0)) {
   5  // pcw    if (db_num_rows(db_query("SELECT nid FROM {node} WHERE status = 1 AND moderate = 0 AND
   6  //                         (created > '". variable_get("ping_cron_last", time()) ."' OR
   7  //                          changed > '". variable_get("ping_cron_last", time()) ."')"), 1)) {
   8  // only ping when created
   9      if (db_num_rows(db_query("SELECT nid FROM {node} WHERE type='blog' AND status = 1 AND moderate = 0 AND
  10                             (created > '". variable_get("ping_cron_last", time()) ."'"), 1)) {
  11        _ping_notify(variable_get("site_name", "") ." - ". variable_get("site_slogan", ""), $base_url);
  12      }
  13  
  14      variable_set("ping_cron_last", time());
  15    }
  16  }
Toggle Line Numbers

The ping itself is only sent on the next cron so I still have that long to change a posting.

Normal Paragraphs

Normal paragraph text is entered normally, Just type away. A blank line is considered the end of a paragraph and the start of a new one.

Wilki format does not support any bold, strong italic etc markup. For that you can use the raw html tags.

<b>Hello Peter</b>

<i>Hello Peter</i>

<u>Hello Peter</u>

<strong>Hello Peter</strong>

<small>Hello Peter</small>

Gives

Hello Peter

Hello Peter

Hello Peter

Hello Peter

Hello Peter

Bullet Points

Asterisks denote bullet lists:

* Bullet Lists
* Good for marketing presentations
  * they can be nested
     * to many levels
     * the text can go on for
       many lines if you
       so desire
  * back to this level
* Back to the start
* The list is terminated by a blank line

Gives

• Bullet Lists
• Good for marketing presentations
  • they can be nested
    • to many levels
    • the text can go on for many lines if you so desire
  • back to this level
• Back to the start
• The list is terminated by a blank line

Numbered Lists

# Much like bullet lists
# but with numbers
   * may contain bullet lists
       # which can themselves contain numbered lists
       # numbered lists can also
         have text that crosses lines
# continuing this numbered list

# This is a new numbered list, because of the blank line.

Gives:

1. Much like bullet lists
2. but with numbers
   • may contain bullet lists
     1. which can themselves contain numbered lists
     2. numbered lists can also have text that crosses lines
3. continuing this numbered list

1. This is a new numbered list, because of the blank line.

Definition Lists

Term: Definition

Note the space at the start of the line and after the colon. Gives:

Term: Definition

There can be many terms:

Term: Definition
Term 2: Definition 2
Term 3: Definition 3

Term: Definition Term 2: Definition 2 Term 3: Definition 3

Term: Definition which may
      cover multiple lines
Bullet Point: A list like this:
      * one
      * two
      * three
Cool?: Could it be more

Gives:

Term: Definition which may cover multiple lines Bullet Point: A list like this:

• one
• two
• three Cool?: Could it be more

Verbatim Blocks

Enter text that is not processed as markup:

Gives:

* an asterisk
# a hash

Code Blocks

<php>
function Fred()
{
   $strPHP = "this is some php code";
}
</php>

<python>
def Fred():
   strPHP = "this is some python code";
</python>

Gives the endlessly cool:

function Fred()
{
   $strPHP = "this is some php code";
}

def Fred():
   strPHP = "this is some python code";

Note that bullet lists, numbered lists and definition terms may have code blocks or verbatim blocks associated with them:

* Why keep on with the bullet points?
<php>
function Fred()
{
   $strPHP = "this is some php code";
}
</php>
* Because I can?
<python>
def Fred():
   strPHP = "this is some python code";
</python>

• Why keep on with the bullet points?

  function Fred()
  {
     $strPHP = "this is some php code";
  }
  

• Because I can

  def Fred():
     strPHP = "this is some python code";
  

Hyperlinks

This is a hyperlink: [my favorite search engine|http://www.google.com]

The title is optional: [http://www.google.com]

As are the brackets: [http://www.google.com]

I can link to nodes in my blog by number: [like this|400]

Node number without title (so the title is filled in for me) like this: [400].

Or I can ask for the node title: [Regular Expression Assertions]

If the URL appears to reference an image (.jpg, .gif, .pgn) then the
image is shown: [images/smileys/smile.png]

This is a hyperlink: my favorite search engine

The title is optional: http://www.google.com

As are the brackets: http://www.google.com

I can link to nodes in my blog by number: like this

Node number without title (so the title is filled in for me) like this: Logrotate failures.

Or I can ask for the node title: Regular Expression Assertions

If the URL appears to reference an image (.jpg, .gif, .pgn) then the image is shown:

I noticed in my Drupal logs the following error:

Weblogs replied flerror 1 message Can't accept the ping because the URL must begin with http://.

I had previously hacked ping.module to log the server response as I was suspicious that it was not working. I changed all the ping code to log both success and error messages (before it just logged errors without giving error details):

  $result = $client->send($message);

  if (!$result || $result->faultCode()) {
    watchdog("error", "failed to notify 'weblogs.com' (RSS)");
  }
  if( $result && !$result->faultCode) {
    watchdog("regular", "Weblogs replied " . $result->serialize());
  }

Anyway, I hacked in a fix to the problem by specifying the full url to my site:

  // PCW: complains about no http:// $message = new xmlrpcmsg("rssUpdate", array(new xmlrpcval($name), new xmlrpcval($feed)));

  $message = new xmlrpcmsg("rssUpdate", array(new xmlrpcval("Peter's Blog"), new xmlrpcval('http://bisiand.me.uk/rss.xml')));

I did a manual cron run and the error message has gone away. Obviously this is a hack just for me, some Drupal god may have already fixed this (I am still on 4.4.0) or come up with a proper fix.