Peter's Blog

Redefining the Impossible

Items filed under wilki


Gave qumana a try. It is a blog editing application that posts entries using whatever xmlrpc interface may be available. Here is the bullet list:

  • very wysiwyg, with all the advantages and disadvantages that follow: easy to do simple/pretty stuff but interferes horribly with wilki formatting.
  • easy to add adverts to your posts, should you be that concerned with making money.
  • supports posting with tags and adds it's own tag by default, which upsets some purists who haven't figured out how to disable it.
  • supports image uploading which had a silly bug when I tried it: if you don't select an image file it still tries to upload it and locks up.
  • it doesn't honour the ancient ctrl-ins, shift-ins copy and paste keys I prefer. This may be a consequence of being written in java which also makes it boot sluggishly.
  • comes with a 'drop pad' onto which you can drag and drop stuff for posting: very nice, except you cannot drag the drop pad onto a second monitor (so time for a 30inch wide screen).

For me it interfered with my wilki formatting too much to be useful on this blog. If I was happy to let it take care of the formatting then it would be just fine.


Filed under: blog wilki

2 Comments

What's the single most tedious thing about blogging: having to press the 'Add Blog Item ' button. I think I've hit on why taking notes with EverNote is fast: there is a blank note at the bottom of the screen that you can just start typing in.

I've realised that this could be done in a web app, simply by having a box on every page ready to start typing in. Google mail does something like this already: there is a little text box at the bottom of each message that you can select and start typing a reply: when selected it resizes itself and formatting toolbars and stuff appear. It's all very AJAXy and slick. Most of the time you can ignore it because it's not too big, once you explicitly start using it all the associated tools appear.

Compare to posting in drupal where there's all kinds of stuff filling the page: Input format, date edit boxes, categories, tags, I have to scroll down two pages to find the 'preview' and 'submit' buttons.

Here's a rough outline of the quick blogging features:

  • Regular blog page appears with a textarea box ready for typing in
  • Title is first line (if preceeded with a - or something), then a list of tags line, each tag preceeded with a dot or something similarly lightweight.
  • Rest of post is in wilki format.
  • Big button that uses AJAX to generate a preview which appears under the text box
  • After preview a post button appears.

Everything goes in the one textarea box, no need to tab or click between controls. Now I know what I want, how to implement it?

I am growing disillusioned with EverNote, mainly because of the buggy handling of formatting: if you mark something bold, for example, it has an annoying habit of not turning the bold off, you have to fiddle around selecting a big block and turning it all off explicitly (very much like Microsoft Word). I'm happy to use markup to make things bold, it's simple and understandable.

I want all my notes on a server where I can get to them from anywhere.


Filed under: blogging drupal evernote wilki

8 Comments

I am now committed to using awtags, so much so that I am adding awtags links all the time. To save myself some typing I have altered my wilki module to enable me to add a link to a tag very simply:

[awtags]

will link me to the awtags tag. If no tag is found with the given name then the linker goes on to look for a node number, article title or whatever that matches.

Previously I had to enter

[tags/awtags]

which was becoming a bind.


Filed under: awtags drupalawtags wilki

Add a comment

I noticed I broke the syntax highlighting in my wilki module when I moved the site to Linode. I forgot to change a hard-coded link to the geshi syntax highlighting library. Fixed now and I took the opportunity to upgrade to the latest version of Geshi and, I'm pleased to say, it Just Worked, didn't need any code changes from me.

Geshi is a very nice library, for me it has only two problems:

  1. it is php, not python
  2. it does not appear to have a mode for highlighting generic config files as does vim.

Silly things like fixing syntax highlighting can cause duplicate postings to appear in RSS aggregators as they nievely think the articles have been edited. Sorry for this, there is not much I can do about it.


Filed under: drupal wilki

Add a comment

Wilki is the name I have given to my version of the Drupal wiki module. This module implements a markup language, making it easier to create things like bullet lists and hyperlinks. It also allows me to syntax highlight source code. Read more about it here.


Filed under: wilki

Add a comment

Two concepts I have been contemplating recently are starting to blur together: outliners and tags.

I have read of people who have moved from Outliners to Wikis as a means of organising their notes. I have found Wiki's rather simplistic and unattractive, especially as web-based editors are sluggish compared to desktop applications.

However, now I have changed my site to use awTags I see how tags can be used to implement a wiki. But this is better than a wiki: a link leads to a list of related articles rather than just a single page. In the manner of an outliner, an article leads on to a list of child articles according to which tag you follow. The article itself can be tagged with a number of different tags representing different concepts that the article itself can be filed under and these links can be regarded as parent relationships. This is just what TheBrain is trying to do: Mind Mapping but without the fancy graphics.

I have altered my wilki module accordingly and have used it right there. By typing in something as simple as

[wilki|tags/wilki]

I created a link to all my wilki module related postings filed under the wilki tag. Alternatively, I could have linked to the wilki introductory article and from there a reader, once they know what the wilki module is, can follow the wilki tag if they so desire.

This is all pretty abstract and I will have to see if it is actually of any use in the real world. On a practical level it will be easier for me to reference other articles by linking to the tag name, rather than trying to find the drupal node number of a specific article I am thinking of. The downside is that going to the introductory article is probably a better pattern to follow, especially as articles are listed in reverse chronological order, putting the most informative introduction at the very end.


2 Comments

I've been contemplating some python web development. This is mainly for the following reasons:

  • I'd like my blog to have features for threading subjects based on keywords, a kind of related articles list. This would be dynamically updated, old articles could link to newer articles. This is possible in Drupal but the taxonomy system is laborious to configure through the web interface. There is a new keyword feature (folksonomies) but it appears to be a library for other modules to use. There are some modules that promise this kind of thing but they are not what I am looking for. The most important thing is that old articles are displayed with links to new articles, which involves messing around with themeing to add links to a cached page or figuring out how to flush the cache and put articles through a filter that would add the links.
  • I'd rather spend my limited programming time being creative with technologies I am already familiar with. I don't like php enough to want to learn it any more. I know and like python.
  • I don't want to spend my time trying to figure out how the Drupal code base works. Ok, I'm lazy and I have a short attention span and I'm not very interested in how it works.
  • If I do this as a spare time project I want to enjoy it, I don't want it to feel like a maintenance exercise. I can do that at work.
  • I have found that using librarys etc written by third partys can give you 95% of what you want, then you spend weeks trying to hack them to get the extra 5%. The only exception to this rule I can think of is the python standard library which is way cool.

So I looked at a few web application frameworks. This is a little lightweight and short on detail, I did most of it a few weeks ago and I didn't take notes.

cherrypy: Requires python 2.3 and my hosting service will only give me python 2.2 with cgi. I think generator functions are way cool and I'd love to use them. Do I stand any chance of getting my hosting service to upgrade? I'll email them and ask.

quixote: got the demo going, although I had to fix some of the urls within it as they appeared to be broken. It worked easily enough with cgi, the 1.3 version worked with python 2.2. However, again they have just released version 2 which has a couple of yield statements in it, limiting it python 2.3.

webware: I don't think I could see the advantage of 'servlets' and their web site (presumably powered by webware but it only says 'python powered') is not an impressive display of it's capabilities.

zope/plone: I've toyed with zope in the past (version 2) and found it's learning curve to be almighty. It may be very capable but you have to learn the zope way to do everything. The developers seem to be concentrating on inventing new object-orientated paradigms rather than keeping things simple and I hate huge complex object models. I don't like the web-based source editor (what, no syntax highlighting?: although I could use mozex and edit in vim). Also, non-cgi.

I was most interested in quixote until I came to the python 2.3 stumbling block with the new version 2. I don't want to settle for an old version. This led me on to think about what these frameworks actually give me:

  • map urls to class/method calls
    • big deal
  • session management
    • do I need it for my simple purposes?
  • templating systems
  • interface to cgi/mod_python/fastcgi/whatever
    • I can only have cgi anyway
  • easy form generation
    • classes to build the contents of a form. How many forms will I need? Can't templates do this?
  • already used on the internet and exposed to hackers
    • lets not be too ambitious with what we do. Keep drupal in the background for data entry.
  • learning curves

Which has led me to... straight cgi. Run my own scripts from cgi, using the cgi module in the standard python library to get the url parameters and generate a nice error page when things go wrong. I can get it going easily enough and below, with just a few hours hacking, is something that can display nodes from the drupal database. The output is themed through the Cheetah template engine and gives me the same look as my drupal theme. Given that it is only cgi it is not massively scalable, but I understand every line of it and that means a lot in terms of development time. Given that it generates pages that look like the drupal pages, I can chop and change, leaving some bits to drupal (data entry) and others in python (presentation). It's a lash up but so what?

The main challenge is porting my wilki module to python, particularly the geshi syntax highlighting.

I am not planning to rewrite Drupal and I probably won't release Yet Another Python Web Framework. I may not even finish it. It is something for me to play with.

   1  #!/usr/bin/python
   2  #
   3  import Cheetah.Template
   4  import cgi
   5  import cgitb; cgitb.enable()
   6  import time
   7  import datetime
   8  import os
   9  import pdo
  10  
  11  def ThemeAndOutput( strPageTitle, strContent, oParams={}):
  12      "Theme the site"
  13  
  14      strSideBar = """
  15      <div class="block">
  16      <h2>Reports</h2>
  17      <ul>
  18          <li><a href="Intranet.cgi">Drupal Emulator</a></li>
  19      </ul>
  20      </div>"""
  21  
  22      oDict = {
  23          'Root': '/SMB',
  24          'Url': '/Intranet/Intranet.cgi',
  25          'Title': 'Drupal Emulator',
  26          'Slogan': 'There are no Problems, only Challenges',
  27          'PrimaryLinks': [ '<a href="http://intranet.org">Intranet</a>',
  28                              '<a href="http://www.google.co.uk">Google</a>'],
  29          'PageTitle': strPageTitle,
  30          'SideBar': strSideBar,
  31          'Footer': 'Copyright 2005 Peter Wilkinson'
  32      }
  33  
  34      #
  35      # Read page template and shove in the content.
  36      #
  37      strTemplate = open( 'PageTemplate.html').read() % { 'Content': strContent}
  38  
  39      #
  40      # Run template and content through cheetah in one go.
  41      # Is this faster? Doubt it.
  42      #
  43      oHtml = Cheetah.Template.Template( strTemplate, [oDict, oParams])
  44  
  45      print str(oHtml)
  46  
  47  def MainPage( oForm):
  48      "Main welcome page: tell user what is going on"
  49  
  50      #
  51      # Determine how many links to show.
  52      #
  53      strFrom = oForm.getfirst( 'from')
  54      if strFrom == None:
  55          strLimit = ' LIMIT 10'
  56      else:
  57          try:
  58              nLimit = int(strFrom)
  59              strLimit = ' LIMIT %d, 10' % nLimit
  60          except:
  61              strLimit = ' LIMIT 10'
  62  
  63      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  64      rs = c.open( 'SELECT * FROM node ORDER BY created DESC %s' % strLimit)
  65  
  66      strTitles = []
  67      while rs.next():
  68          strTitles.append( '<a href="Intranet.cgi?node=%s">%s</a>' %
  69                (rs.fields['nid'].value, rs.fields['title'].value))
  70  
  71      strPage = """
  72  <ul>
  73  #for $strTitle in $titles
  74  <li>$strTitle</li>
  75  #end for
  76  </ul>
  77      """
  78  
  79      ThemeAndOutput( "Welcome", strPage, { 'titles': strTitles})
  80  
  81  def NodePage( oForm):
  82      "Display a node"
  83  
  84      strNode = oForm.getfirst( 'node')
  85      if strNode == None:
  86          ErrorPage()
  87          return
  88      else:
  89          try:
  90              nNode = int(strNode)
  91          except:
  92              ErrorPage()
  93              return
  94  
  95      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  96      rs = c.open( 'SELECT * FROM node WHERE nid = %d' % nNode)
  97  
  98      if rs.next():
  99          strTitle = rs.fields['title'].value
 100          strBody = rs.fields['body'].value
 101      else:
 102          ErrorPage()
 103          return
 104  
 105      strPage = """
 106  <h2>$strTitle</h2>
 107  
 108  <pre>
 109  $strBody
 110  </pre>
 111      """
 112  
 113      ThemeAndOutput( strTitle, strPage,
 114            { 'strTitle': strTitle, 'strBody': strBody})
 115  
 116  def ErrorPage():
 117      strPage = """<p>There has been some kind of navigation error, the link
 118  you just clicked is broken</p>
 119  <p>Better <a href="mailto:someone@somewhere.com">complain about it</a>.</p>
 120      """
 121  
 122      ThemeAndOutput( "Drupal emulator", strPage)
 123  
 124  oForm=cgi.FieldStorage()
 125  
 126  node = oForm.getfirst( 'node')
 127  if node:
 128      NodePage( oForm)
 129  else:
 130      MainPage( oForm)

The page template is kept in a seperate file. It is based on the phptemplate of my drupal theme and was ported over in less than an hour. Theming engines? A specialisation of templates.

   1  Content-Type: text/html
   2  
   3  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   4   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   5  <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
   6  <head>
   7  <title>$Title</title>
   8  <meta http-equiv="Content-Style-Type" content="text/css"/>
   9  <style type="text/css">@import url(style.css);</style>
  10  </head>
  11  <body>
  12      <div id="headerleft">
  13      </div>
  14      <div id="headermain">
  15          <div id="ie6hack">
  16          </div>
  17      </div>
  18      <div id="headertitle">
  19          <h1>
  20              <a href="$Url" title="$Title">
  21              $Title</a>
  22          </h1>
  23      </div>
  24      <div class="slogan">
  25          $Slogan
  26      </div>
  27      <div id="logo">
  28          <a href="$Url" title="$Title">
  29              <img src="ITL.gif"/>
  30          </a>
  31      </div>
  32      <div id="headerright">
  33      </div>
  34  #if len($PrimaryLinks) > 0
  35      <div id="primarylinks">
  36          <ul>
  37  #for $PrimaryLink in $PrimaryLinks
  38              <li>$PrimaryLink</li>
  39  #end for
  40          </ul>
  41      </div>
  42  #end if
  43  
  44      <div id="ie5forcemargininsidehack">
  45          <div id="main">
  46              <div class="tab">
  47                  <div class="tabbackground">
  48                      <div class="tableft">
  49                          <h3>
  50                              $PageTitle
  51                          </h3>
  52                      </div>
  53                  </div>
  54              </div>
  55              <div class="tabbody">
  56                  <div class="ieisbuggy">
  57                  </div>
  58                  <div id="content">
  59                      %(Content)s
  60                  </div>
  61                  <div class="ieisbuggy">
  62                  </div>
  63              </div>
  64          </div>
  65          <div id="footer">
  66              <div class="footerleft">
  67                  <p>
  68                  $Footer
  69                  </p>
  70              </div>
  71          </div>
  72      </div>
  73      <div id="sidebarright">
  74          <div id="sidebar">
  75              <div class="sidebarbox">
  76                  $SideBar
  77              </div>
  78          </div>
  79          <div id="sidebarbottom">
  80          </div>
  81      </div>
  82  </body>

Add a comment

Update engineering log to Drupal 4.6.0.

Notes:

  • Followed upgrade instructions, no big problems
  • My Theme worked after installing phptemplate.
  • Wilki module worked after reading and writing filter settings.
  • Could not find 4.6 version of smileys module. Copied 4.5 version and edited smileys.module to change instances of drupal_specialChars to check_plain as described here.
  • If this reaches my public blog then the sendhome module is working.

So far, see no big changes. It keeps moaning about some image library that I don't need not being installed and there are some icons in the admin logs. Looking forward to trying the improved search . I found google site search more useful than the old search but the engineering log is behind a firewall and beyond google's reach.


Filed under: blog drupal google theme wilki

Add a comment

I like the concept of outliners, previously I used Bonsai but today I've been giving The Vim Outliner a try. It works as a very simple markup language and then adds syntax highlighting to make headers more visible. For example:

This is a Heading.
   This is a Subheading.
        | This is some text within the subheading.
        | Blah Blah Blah Blah
   This is another subheading
        | More plain text.
        | Blah Blah Blah

gets displayed like this:

This is a Heading.
    This is a Subheading.
        | This is some text within the subheading.
        | Blah Blah Blah Blah
    This is another subheading
        | More plain text.
        | Blah Blah Blah

(notice how I can use Vim's built-in syntax highlighting to html conversion to generate this).

This makes outlining in VIM very easy and the syntax highlighting helps readability a great deal. TVL defines some useful shortcut keys such as - and + to fold and unfold sections easily. The outline text is stored in .otl files and Vim's file type system automatically applies the outlining features to these files when they are opened.

I find it interesting to compare what is effectively a new markup language with other markup languages I have encountered such as reStructuredText and indeed I am writing this using my Wilki Module which is another markup language. These have all used markup on the headers and plain body text but in TVO the body text is both marked with a '|' character and the headers are plain text. This would make TVO files tedious to type except Vim has been set up to automatically insert the indents and the '|' character so it is easy to type away as normal.

Markup languages are most useful when they can be easily converted into other formats. The Vim Outliner comes with an otl to html converter written in Ruby (python ripoff programming language named after a Kenny Rogers song about a prostitute). It is not hard to do simplistic conversion in python:

   1  #
   2  # otl2html.py: convert .otl files to html
   3  #
   4  import re
   5  
   6  oREIndent = re.compile( '^(\s*)(\|\s)?(.*)')
   7  
   8  nHeaderStack = [ 0]
   9  nTextIndent = None
  10  
  11  for strLine in open( r'c:\tmp\try2.otl').readlines():
  12    oMatch = oREIndent.match( strLine)
  13    if oMatch:
  14      strIndent = oMatch.group(1)
  15      strTextLeader = oMatch.group(2)
  16      strText = oMatch.group(3)
  17    else:
  18      strIndent = ''
  19      strTextLeader = ''
  20      strText = strLine
  21    if strTextLeader != None:
  22      #
  23      # This is a text block
  24      #
  25      if nTextIndent == None:
  26        nTextIndent = len( strTextLeader.expandtabs(4))
  27        print "<p>"
  28      else:
  29        strText = (strTextLeader + strText)[nTextIndent:]
  30      print strText
  31    else:
  32      #
  33      # Dealing with header
  34      #
  35  
  36      #
  37      # Terminate text block.
  38      #
  39      if nTextIndent != None:
  40        print "</p>"
  41        nTextIndent = None
  42  
  43      #
  44      # Lets assume it's me and a tab = 4 spaces
  45      #
  46      nIndent = len( strIndent.expandtabs(4))
  47      while nHeaderStack[-1] > nIndent:
  48        nHeaderStack.pop()
  49      if nHeaderStack[0] < nIndent:
  50        nHeaderStack.append( nIndent)
  51      print '<h%d>%s</h%d>' % (len(nHeaderStack), strText, len(nHeaderStack))

Add a comment

Updated site to Drupal 4.5.0. Used same strategy as at work. All went fairly smoothly.

Tweeked the site theme a bit to make it a bit less Drupally (I don't blame them for keeping the new Drupal site theme under wraps), not too happy with the results, needs further tweeking. Spent a while fighting an annoying IE bug with 100% width tables, margins and borders. I've resolved it for now by setting the table width to 99% but it still irks me. I tried changing em dimensions to px to no avail.

Smileys still work smile

print "Wilki Module still works"

Filed under: drupal python theme wilki

Add a comment