Peter's Blog

This and this aren't about me. There are a surprising number of illustrious figures with that name though.

Gave qumana a try. It is a blog editing application that posts entries using whatever xmlrpc interface may be available. Here is the bullet list:

• very wysiwyg, with all the advantages and disadvantages that follow: easy to do simple/pretty stuff but interferes horribly with wilki formatting.
• easy to add adverts to your posts, should you be that concerned with making money.
• supports posting with tags and adds it's own tag by default, which upsets some purists who haven't figured out how to disable it.
• supports image uploading which had a silly bug when I tried it: if you don't select an image file it still tries to upload it and locks up.
• it doesn't honour the ancient ctrl-ins, shift-ins copy and paste keys I prefer. This may be a consequence of being written in java which also makes it boot sluggishly.
• comes with a 'drop pad' onto which you can drag and drop stuff for posting: very nice, except you cannot drag the drop pad onto a second monitor (so time for a 30inch wide screen).

For me it interfered with my wilki formatting too much to be useful on this blog. If I was happy to let it take care of the formatting then it would be just fine.

There are three ways to blog from a pocketpc that I have found:

1. open your blog in a web browser and edit it as normal. This doesn't work so well in practise unless your web page has been designed with a css style sheet that works nicely on a pda. I tried modifying my drupal theme accordingly and made it presentable but it's not up to data entry.
2. try using an application for posting to blogs. There are a couple of these about but those that I tried were buggy or had tiny little edit boxes.
3. write the article and submit by email. This is how I am posting this, I use phatnotes, a notetaking application that can send the notes via email. I don't use pocket outlook for the simple reason that it doesn't seem to save sent mail so I cannot edit the article and send it again to modify it. I use my old mailbot script on the server to capture posts and poke them into drupal.

So far this has worked ok and I can compose posts offline and upload them at my leisure.

Disadvantages:

• No preview unless I go online, hence I keep posts simple.
• cutting and pasting urls is fiddly and I cannot be bothered with it.
• my script adds tags by scanning the database for existing tags, searching the post for the tag words and adding the tags that it finds. This can give irrelevant tags and I cannot define new tags. I need a neat way to specify tags in the post.

Advantages:

• the convenience of whipping my pda out and having a quick blog. I don't have to go upstairs and get my laptop.
• can lay on settee and blog in comfort.

I've moved this site to my Linode. I have a couple more months before my Site5 account runs out so I have time to move back if necessary. I moved the site as follows:

• Dump the sql:

  cd ~/www
  mysqldump -u <user> -p drupal > pb.sql
  

• Zip the useful stuff:

  zip -r pb cron.php database/ fail.html favicon.ico files/ images/ includes/
  index.php misc/ modules/ pb.sql robots.txt scripts/ sites/ themes/ tmp/ xmlrpc.php
  

  Will I ever learn to love tar?
• copy the zip to new site by running sftp on new site.
• unzip the zip file in /var/www
• load the database into mysql:

  mysqladmin create drupal -u <user> -p
  mysql -u <user> -p drupal < pb.sql
  

• go to domain registrar and change DNS address of site from site5 to linode. It only took three hours or so for this to propogate enough for me to access the linode via the petersblog.org name.

And if you are reading this then you are reading the site from the Linode as I haven't posted it on the old site. I can leave the old site floating for a while until the old address is flushed from all the DNS caches out there. I'll leave bisiand.me.uk pointing there until the account expires: unless of couse something happens to convince me to stick with Site5.

I have no problems with Site5, I would recommend them to anybody, it's just that Linode is more technically challenging and hence more interesting and fun.

blog, a short for of 'web log', a kind of diary that 100 million people can have access to, but maybe a dozen actually read.

Two concepts I have been contemplating recently are starting to blur together: outliners and tags.

I have read of people who have moved from Outliners to Wikis as a means of organising their notes. I have found Wiki's rather simplistic and unattractive, especially as web-based editors are sluggish compared to desktop applications.

However, now I have changed my site to use awTags I see how tags can be used to implement a wiki. But this is better than a wiki: a link leads to a list of related articles rather than just a single page. In the manner of an outliner, an article leads on to a list of child articles according to which tag you follow. The article itself can be tagged with a number of different tags representing different concepts that the article itself can be filed under and these links can be regarded as parent relationships. This is just what TheBrain is trying to do: Mind Mapping but without the fancy graphics.

I have altered my wilki module accordingly and have used it right there. By typing in something as simple as

[wilki|tags/wilki]

I created a link to all my wilki module related postings filed under the wilki tag. Alternatively, I could have linked to the wilki introductory article and from there a reader, once they know what the wilki module is, can follow the wilki tag if they so desire.

This is all pretty abstract and I will have to see if it is actually of any use in the real world. On a practical level it will be easier for me to reference other articles by linking to the tag name, rather than trying to find the drupal node number of a specific article I am thinking of. The downside is that going to the introductory article is probably a better pattern to follow, especially as articles are listed in reverse chronological order, putting the most informative introduction at the very end.

I've been contemplating some python web development. This is mainly for the following reasons:

• I'd like my blog to have features for threading subjects based on keywords, a kind of related articles list. This would be dynamically updated, old articles could link to newer articles. This is possible in Drupal but the taxonomy system is laborious to configure through the web interface. There is a new keyword feature (folksonomies) but it appears to be a library for other modules to use. There are some modules that promise this kind of thing but they are not what I am looking for. The most important thing is that old articles are displayed with links to new articles, which involves messing around with themeing to add links to a cached page or figuring out how to flush the cache and put articles through a filter that would add the links.
• I'd rather spend my limited programming time being creative with technologies I am already familiar with. I don't like php enough to want to learn it any more. I know and like python.
• I don't want to spend my time trying to figure out how the Drupal code base works. Ok, I'm lazy and I have a short attention span and I'm not very interested in how it works.
• If I do this as a spare time project I want to enjoy it, I don't want it to feel like a maintenance exercise. I can do that at work.
• I have found that using librarys etc written by third partys can give you 95% of what you want, then you spend weeks trying to hack them to get the extra 5%. The only exception to this rule I can think of is the python standard library which is way cool.

So I looked at a few web application frameworks. This is a little lightweight and short on detail, I did most of it a few weeks ago and I didn't take notes.

cherrypy: Requires python 2.3 and my hosting service will only give me python 2.2 with cgi. I think generator functions are way cool and I'd love to use them. Do I stand any chance of getting my hosting service to upgrade? I'll email them and ask.

quixote: got the demo going, although I had to fix some of the urls within it as they appeared to be broken. It worked easily enough with cgi, the 1.3 version worked with python 2.2. However, again they have just released version 2 which has a couple of yield statements in it, limiting it python 2.3.

webware: I don't think I could see the advantage of 'servlets' and their web site (presumably powered by webware but it only says 'python powered') is not an impressive display of it's capabilities.

zope/plone: I've toyed with zope in the past (version 2) and found it's learning curve to be almighty. It may be very capable but you have to learn the zope way to do everything. The developers seem to be concentrating on inventing new object-orientated paradigms rather than keeping things simple and I hate huge complex object models. I don't like the web-based source editor (what, no syntax highlighting?: although I could use mozex and edit in vim). Also, non-cgi.

I was most interested in quixote until I came to the python 2.3 stumbling block with the new version 2. I don't want to settle for an old version. This led me on to think about what these frameworks actually give me:

• map urls to class/method calls
  • big deal
• session management
  • do I need it for my simple purposes?
• templating systems
  • Cheetah
• interface to cgi/mod_python/fastcgi/whatever
  • I can only have cgi anyway
• easy form generation
  • classes to build the contents of a form. How many forms will I need? Can't templates do this?
• already used on the internet and exposed to hackers
  • lets not be too ambitious with what we do. Keep drupal in the background for data entry.
• learning curves

Which has led me to... straight cgi. Run my own scripts from cgi, using the cgi module in the standard python library to get the url parameters and generate a nice error page when things go wrong. I can get it going easily enough and below, with just a few hours hacking, is something that can display nodes from the drupal database. The output is themed through the Cheetah template engine and gives me the same look as my drupal theme. Given that it is only cgi it is not massively scalable, but I understand every line of it and that means a lot in terms of development time. Given that it generates pages that look like the drupal pages, I can chop and change, leaving some bits to drupal (data entry) and others in python (presentation). It's a lash up but so what?

The main challenge is porting my wilki module to python, particularly the geshi syntax highlighting.

I am not planning to rewrite Drupal and I probably won't release Yet Another Python Web Framework. I may not even finish it. It is something for me to play with.

   1  #!/usr/bin/python
   2  #
   3  import Cheetah.Template
   4  import cgi
   5  import cgitb; cgitb.enable()
   6  import time
   7  import datetime
   8  import os
   9  import pdo
  10  
  11  def ThemeAndOutput( strPageTitle, strContent, oParams={}):
  12      "Theme the site"
  13  
  14      strSideBar = """
  15      <div class="block">
  16      <h2>Reports</h2>
  17      <ul>
  18          <li><a href="Intranet.cgi">Drupal Emulator</a></li>
  19      </ul>
  20      </div>"""
  21  
  22      oDict = {
  23          'Root': '/SMB',
  24          'Url': '/Intranet/Intranet.cgi',
  25          'Title': 'Drupal Emulator',
  26          'Slogan': 'There are no Problems, only Challenges',
  27          'PrimaryLinks': [ '<a href="http://intranet.org">Intranet</a>',
  28                              '<a href="http://www.google.co.uk">Google</a>'],
  29          'PageTitle': strPageTitle,
  30          'SideBar': strSideBar,
  31          'Footer': 'Copyright 2005 Peter Wilkinson'
  32      }
  33  
  34      #
  35      # Read page template and shove in the content.
  36      #
  37      strTemplate = open( 'PageTemplate.html').read() % { 'Content': strContent}
  38  
  39      #
  40      # Run template and content through cheetah in one go.
  41      # Is this faster? Doubt it.
  42      #
  43      oHtml = Cheetah.Template.Template( strTemplate, [oDict, oParams])
  44  
  45      print str(oHtml)
  46  
  47  def MainPage( oForm):
  48      "Main welcome page: tell user what is going on"
  49  
  50      #
  51      # Determine how many links to show.
  52      #
  53      strFrom = oForm.getfirst( 'from')
  54      if strFrom == None:
  55          strLimit = ' LIMIT 10'
  56      else:
  57          try:
  58              nLimit = int(strFrom)
  59              strLimit = ' LIMIT %d, 10' % nLimit
  60          except:
  61              strLimit = ' LIMIT 10'
  62  
  63      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  64      rs = c.open( 'SELECT * FROM node ORDER BY created DESC %s' % strLimit)
  65  
  66      strTitles = []
  67      while rs.next():
  68          strTitles.append( '<a href="Intranet.cgi?node=%s">%s</a>' %
  69                (rs.fields['nid'].value, rs.fields['title'].value))
  70  
  71      strPage = """
  72  <ul>
  73  #for $strTitle in $titles
  74  <li>$strTitle</li>
  75  #end for
  76  </ul>
  77      """
  78  
  79      ThemeAndOutput( "Welcome", strPage, { 'titles': strTitles})
  80  
  81  def NodePage( oForm):
  82      "Display a node"
  83  
  84      strNode = oForm.getfirst( 'node')
  85      if strNode == None:
  86          ErrorPage()
  87          return
  88      else:
  89          try:
  90              nNode = int(strNode)
  91          except:
  92              ErrorPage()
  93              return
  94  
  95      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  96      rs = c.open( 'SELECT * FROM node WHERE nid = %d' % nNode)
  97  
  98      if rs.next():
  99          strTitle = rs.fields['title'].value
 100          strBody = rs.fields['body'].value
 101      else:
 102          ErrorPage()
 103          return
 104  
 105      strPage = """
 106  <h2>$strTitle</h2>
 107  
 108  <pre>
 109  $strBody
 110  </pre>
 111      """
 112  
 113      ThemeAndOutput( strTitle, strPage,
 114            { 'strTitle': strTitle, 'strBody': strBody})
 115  
 116  def ErrorPage():
 117      strPage = """<p>There has been some kind of navigation error, the link
 118  you just clicked is broken</p>
 119  <p>Better <a href="mailto:someone@somewhere.com">complain about it</a>.</p>
 120      """
 121  
 122      ThemeAndOutput( "Drupal emulator", strPage)
 123  
 124  oForm=cgi.FieldStorage()
 125  
 126  node = oForm.getfirst( 'node')
 127  if node:
 128      NodePage( oForm)
 129  else:
 130      MainPage( oForm)

The page template is kept in a seperate file. It is based on the phptemplate of my drupal theme and was ported over in less than an hour. Theming engines? A specialisation of templates.

   1  Content-Type: text/html
   2  
   3  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   4   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   5  <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
   6  <head>
   7  <title>$Title</title>
   8  <meta http-equiv="Content-Style-Type" content="text/css"/>
   9  <style type="text/css">@import url(style.css);</style>
  10  </head>
  11  <body>
  12      <div id="headerleft">
  13      </div>
  14      <div id="headermain">
  15          <div id="ie6hack">
  16          </div>
  17      </div>
  18      <div id="headertitle">
  19          <h1>
  20              <a href="$Url" title="$Title">
  21              $Title</a>
  22          </h1>
  23      </div>
  24      <div class="slogan">
  25          $Slogan
  26      </div>
  27      <div id="logo">
  28          <a href="$Url" title="$Title">
  29              <img src="ITL.gif"/>
  30          </a>
  31      </div>
  32      <div id="headerright">
  33      </div>
  34  #if len($PrimaryLinks) > 0
  35      <div id="primarylinks">
  36          <ul>
  37  #for $PrimaryLink in $PrimaryLinks
  38              <li>$PrimaryLink</li>
  39  #end for
  40          </ul>
  41      </div>
  42  #end if
  43  
  44      <div id="ie5forcemargininsidehack">
  45          <div id="main">
  46              <div class="tab">
  47                  <div class="tabbackground">
  48                      <div class="tableft">
  49                          <h3>
  50                              $PageTitle
  51                          </h3>
  52                      </div>
  53                  </div>
  54              </div>
  55              <div class="tabbody">
  56                  <div class="ieisbuggy">
  57                  </div>
  58                  <div id="content">
  59                      %(Content)s
  60                  </div>
  61                  <div class="ieisbuggy">
  62                  </div>
  63              </div>
  64          </div>
  65          <div id="footer">
  66              <div class="footerleft">
  67                  <p>
  68                  $Footer
  69                  </p>
  70              </div>
  71          </div>
  72      </div>
  73      <div id="sidebarright">
  74          <div id="sidebar">
  75              <div class="sidebarbox">
  76                  $SideBar
  77              </div>
  78          </div>
  79          <div id="sidebarbottom">
  80          </div>
  81      </div>
  82  </body>

Update engineering log to Drupal 4.6.0.

Notes:

• Followed upgrade instructions, no big problems
• My Theme worked after installing phptemplate.
• Wilki module worked after reading and writing filter settings.
• Could not find 4.6 version of smileys module. Copied 4.5 version and edited smileys.module to change instances of drupal_specialChars to check_plain as described here.
• If this reaches my public blog then the sendhome module is working.

So far, see no big changes. It keeps moaning about some image library that I don't need not being installed and there are some icons in the admin logs. Looking forward to trying the improved search . I found google site search more useful than the old search but the engineering log is behind a firewall and beyond google's reach.

Latest candidate in my pursuit of the ideal Outliner is TreePad. This is the most appealing so far:

• two-pane outliner: tree hierarchy on the left, node contents on the right. Simpler than Microsoft OneNote with it's confusing mix of tabs levels.
• The node contents are rich text that can contain formatting, images, tables (unlike OneNote) etc. Node editor is virtually a word processor.
• Nodes may contain hyperlinks to executables, web pages, other nodes in other treepad documents (unlike OneNote and better than leo's cloning which only works in the context of a single document). Hyperlinks go in article body, unlike Leo where they go in the tree node title, not a good place to put it in my opinion.
• Tree nodes may have an icon associated with it. Leo loads it's selection of icons very slowly.
• Not the fastest program I have used: the help file is a 5M treepad document and takes a good time to load up (1.5G Centrino).
• Simple installation: one of it's features is that it can be installed on a USB flash key and carried around. I haven't found a licence agreement yet to tell me if it has to be licenced on each pc it is used on.
• Robust encryption. Dare I trust it with my credit card numbers and passwords?
• Can export in many formats including XML, OPML (which strips internal hyperlinks), html-ish files parsable by python. It is important to me that data can be extracted from any proprietary file format.
• Documents can be exported as entire websites, complete with Javascript driven tree. Kinda tempting to examine the possibility of organising a website around with this, carryng it in pocket, publishing it, searching it, refactoring it, writing it to pdf... I find blog notes the best place to record things but web interface is not the best editor. Could type stuff in here and bulk move it to the blog.

It is commercial (£20 ish) but it is very polished (unlike Leo).

Uploading images to this blog can be kinda fiddly. Say I have an image in the windows clipboard (e.g. a screen clip from OneNote and I want it on the blog, I have to fire up a graphics program, paste it in, save it to a file, fire up filezilla, upload it, delete the image file.

This little script does all this for me. It asks for a file name, writes the image in the clipboard to the file, uploads it and deletes the image. Job Done.

The uploading is done using pscp, a version of scp from the putty camp. It uses my putty private key so I don't have to give a password for access to the server (I didn't use the cygwin version of ssh as it didn't like having 'c:\' in file names). The script uses the Python Imaging Library to do the grabbing and wxPython to ask for the file name.

   1  #
   2  # Upload image in clipboard to server
   3  #
   4  import wx
   5  import ImageGrab
   6  import Image
   7  import os
   8  import tempfile
   9  
  10  class MyApp(wx.App):
  11      def OnInit(self):
  12          oImage = ImageGrab.grabclipboard()
  13  
  14          if not isinstance( oImage, Image.Image):
  15              #
  16              # In no way is wxPython an easy to use library.
  17              #
  18              dlg = wx.MessageDialog( None, 'No image in clipboard', 'Error', wx.OK)
  19              dlg.ShowModal()
  20              dlg.Destroy()
  21              return True
  22  
  23          dlg = wx.TextEntryDialog( None, 'Enter a file name', 'Upload Clipboard', '')
  24          if dlg.ShowModal() != wx.ID_OK:
  25              dlg.Destroy()
  26              return True
  27  
  28          strName = dlg.GetValue()
  29          dlg.Destroy()
  30  
  31          strTempFile = tempfile.mktemp( strName)
  32          oImage.save( open( strTempFile, 'wb'))
  33  
  34          strCmd = 'pscp -q -i "c:\my documents\puttykey.ppk" %s me@myserver.com:www/images/%s' % (strTempFile, strName)
  35  
  36          os.system( strCmd)
  37  
  38          os.unlink( strTempFile)
  39  
  40          return True
  41  
  42  app = MyApp(0)
  43  app.MainLoop()

This is the result of alt-print screen and running this script:

The script itself inside VIM

ToDo:

• handle file names in clipboard
• detect upload errors
• allow resizing images