Peter's Blog

For the record I am still alive.

I'm busy working on a new blogging platform and I'm not quite ready to make it live yet. I decided to write it in Ruby on Rails directly rather than use an existing blogging application. I wanted something that ran under ruby on rails for the hackability and there were four candidates:

typo: Most popular rails blogging app but nobody seems to rate it as technically excellent. I tried the demo but was put off by the ajax: lots of complexity to go wrong there. mephisto: This seems to be highly regarded on a technical level but a few things put me off:

• it hasn't had an official release since 2006, the mephisto community seems to work in terms of svn revision numbers.
• it needs a pagination plugin to give basic functionality required by any blog with more than ten posts. I'd rather keep plugin requirements to a minimum (fewer things to break when upgrades come along).
• the database schema seems to be growing in complexity: 17 tables? The number of fields in the table that holds posts is also proliferating: 27 fields?
• the layout is configured by editing liquid templates. Apart from making it possible to edit layouts through the admin panel and to support themes, to me this just adds another layer of complexity and more to learn. I am happy with Rails built-in templating system. It took me no time to port my drupal theme to rails.

simplelog: a simple blogging platform but it didn't grab me. The link on the site to a wiki gives an access error.

radiant: Could be used for blogging but Radiant is primarily a cms. The author of Radiant uses Mephisto...

Development of my application has been quite rapid and it's very nearly ready for launch. However I have a problem with the Ruby on Rails testing framework: it's addictive and I can't bare the thought of putting up a live website without making sure it is thoroughly tested first. I'm in that testing/polishing phase that is hard to break out of if you have any kind of obsession with detail.

One thing is for sure: Rails is a joy to work with.

In the meantime, this old blog platform has lost it's lustre and I'm not so included to use it.

UPDATE: I forgot to mention hobix. I can't get past the website...

This blog is now running on my new rails based blogging platform which I have called PetersBlogger (I'm not good at names).

It's mostly complete. Features:

• post articles (or you wouldn't be reading this)
• wilki formatting including syntax highlighting with ultraviolet e.g.

  strThis = "that"
  

• comments, including
  • moderation list
  • Peter's simplistic captcha system
• tagging
• pingomatic submission
• Technorati tags
• rss feeds (no atom yet)
• uses fragment caching for performance

Todo list:

• AJAX formatting preview
• atom feeds (do I care?)
• search (or rely on google)
• tag cloud
• archives by month/year: you DO care what I wrote in Feb 2005 right?
• gimmicks

Please let me know if you spot any sillies (that's if comment submission is working at all).

It's nice to use because the administration interface isn't split out into a seperate set of web pages, when I 'log in' I simply get extra buttons to press in the pages that you are seeing.

I've missed blogging while I've been working on PetersBlogger. My old drupal blog was limping along: while administering the site it kept logging me out. I can admit now that I was running an old version of drupal (4.7?) which I couldn't upgrade due to module dependencies (mainly awtags). The old code wasn't running that smoothly under php5 despite initial appearancies.

I'm happy with my new blog platform, it's running really nicely on my slicehost slice and I've learnt an awful lot about rails while developing it. I will try to do a braindump of rails experiencies in forthcoming posts (cross fingers).

Drupal is a fine CMS and I'm using Drupal 5 on my company intranet. My thoughts on a drupal vs ruby on rails debate would be:

• drupal/php/apache is easier to deploy than a rails/ruby/mongrel/nginx deploy: I moved my drupal site between about four different hosts and did it each time in less than an hour. A rails setup is more convoluted (especially mongrel_cluster). However, this site is running much faster under rails than it was under drupal. The difference is probably down to php and apache being more mainstream and slightly more refined and hence easy to set up. I'm not saying rails is a total pain, just that if I only had ten minutes in which to deploy a site I would be reaching for drupal. If I only wanted to install the source and a few modules and never touch any coding then I would be happy with drupal.
• I love ruby and rails development. There is no comparison, php seems to me as much a bastard language as visual basic 6. Ruby was cleanly developed as an object orientated language, php is having object orientation grafted on as an afterthough.
• If I look through my drupal source I see no unit testing. Ruby/Rails has unit testing built in and I'm completely sold on it. I don't think I will ever trust any code (especially code I write) if it hasn't been unit tested. Some interesting aspects of unit testing:
  • I understand now the 'test driven' approach to development where you design a module's api by first roughing out how the tests will work: the unit test is your first experience of using the new api. It's during testing that you learn how nice an api will be.
  • If the documentation of a new ruby/rails tool is dubious, look in the unit tests to see how the author intended it to be used. If there are no tests then run.
  • If I run into a tricky bug, it's better to first reproduce it in unit tests, fix it there and then be happy that the bug will never recur. It's easier to debug code in a unit test (essentially a command line application) than on a live website.
• all ruby on rails applications follow basically the same architecture. It is fairly easy to figure out how a new application works. Every php application is different. I know I'm comparing a web framework to a programming language but none of the php applications (drupal, phpmyadmin) or am familiar with (wordpress) share a common architecture. New php application to maintain? New learning curve.
• my new site is being hammered with attempts to post to /comments/reply, even though it doesn't use that url for comment submission. The comment spammers know drupal and know where to test the locks. If any of them ever bother to try to find my comment submission url they will only find the same captcha that protected my drupal site (which was much easier to implement in rails, simply as a validation on the model).

I wanted PetersBlogger to have a preview facility when composing posts that would automatically run the text I was editing through my wilki filter as I typed and give me a live preview of how the post would turn out when it was formatted and displayed. No pressing a preview button for me, I wanted it to just happen. Sounds like a job for AJAX. But how to do this in rails? Well it turns out to be very easy.

I put this in my form where I edit a post:

<p><%= f.text_area 'body', :size => "100x40", :html => { :id => :body} %></p>
<%= observe_field :post_body, :url => { :action => :wilkify },
    :frequency => 3.00,
    :update => :preview,
    :with => "'text='+escape( $('post_body').value)"
%>
<div id="preview"/>

This puts some javascript in the page that monitors the text_area where I am editing and, if it changes it will invoke the 'wilkify' action in my controller, passing it what I have typed thus far.

The 'wilkify' action is very simple, it just gets the text, wilkifies it and makes sure it is rendered as a simple piece of html, without the main site layout:

def wilkify
  strBody = params[:text]
  strBody = CGI::unescape( strBody)
  @text = Wilki( strBody)
  render :layout => false
end

I have a small view defined to wrap the preview text in a couple of div's, just as if they were appearing on the site:

<div class="node">
  <div class="bodynode">
    <div class="entry">
      <%= @text %>
    </div>
  </div>
</div>

The wikified text is then inserted into the edit form in the div with the id 'preview'.

I have to add the prototype javascript library to the head section of the main application layout:

<%= javascript_include_tag 'prototype' %>

It is not entirely perfect and I would never let strangers use it because they could type malformed html and totally mess things up but it is good enough for me.

I want to use rails as the front end for an application that I am developing. The application requires that I maintain a persistant object, i.e. one that lives on between multiple page requests. Given that my application would live in a single mongrel instance, is there an easy way to do this?

Well, I studied the code in the mongrel/rails stack (open source ftw) and it turns out that it is easy. Once the rails framework is loaded by mongrel, the dispatcher will handle any request that comes in and once the request is completed it will reload rails own reloadable classes (controllers, models etc) so that they are 'clean' and ready for the next request.

It is very easy to have an object that survives between individual page requests.

Here is an object:

   1  class SillyCounter
   2    def initialize
   3      @count = 0
   4    end
   5  
   6    def Inc
   7      return @count += 1
   8    end
   9  end
  10  
  11  $oSillyCounter = SillyCounter.new

I put this in lib/SillyCounter.rb since I like CamelCase.

I load it in config/environment.rb, right at the bottom:

require 'lib/SillyCounter'

and in a view I added:

<p><%= $oSillyCounter.Inc %> pages served since last reboot</p>

Relaunch mongrel and it works!

CAVEAT: I'll emphasise that this will only work for a single process server! But my application is single user and will never need to serve 1000 requests/second! I would use dRb (distributed ruby) if I needed something scalable.

Todo: find a rails problem that takes more than 20 lines to get around.

PHP programmers out there may be interested to know that this is the first time I have used a $ in a variable name in ruby.

I've been vimming a lot recently yet I haven't given any vim tips for an eternity.

With this in .vimrc/_vimrc:

set numberwidth=5
set numbers

vim shows line numbers on each line. I've held out against this for years but now I'm using >= 22" widescreen monitors I am converted.

The only problem is when I am vimming over an ssh tunnel (which I do a lot to maintain PetersBlogger) and I want to copy and paste: the pasted stuff includes the line numbers. When this crops up I do

:nonumber

to turn it off (kinda like doing a :noautoindent before pasting indented code).

UPDATE: this is even better if the background of the line numbers is changed so you can clearly see the gutter they are in:

:color murphy
:hi LineNr guifg=white guibg=darkgreen
:hi LineNr ctermfg=white ctermbg=darkgreen

I forgot to mention the best ruby tip of all, the biggest time saver in the world ever.

p whatever

Simply prints the value of a variable on a new line. It is equivalent to:

print "#{whatever.inspect}\n"

or

puts "#{whatever.inspect}"

Who needs debuggers? Well I do occasionally but a well aimed print can shed light into the darkest gloom.

Moral: read more books.

Since I added the page count thing yesterday (that only I can see since I don't want everyone to know how often I tweek the site and reboot) I got this:

I think I've managed to map all the urls from my old drupal blog to this new blog: according to statcounter (which is a bit slow loading today) I had 1,103 page loads yesterday compared to 1,167 last week on the old blog. Any missed urls would lead to the 404 page which doesn't have the hit counter or statcounter on it (hum, could catch the 404's and redirect them to main page with a flash notice..).

Statcounter works under javascript and so does not count bots and paranoid people with javascript disabled. It is interesting therefore that the latter outnumber the javascript users by 2.5:1. And none of this includes rss/atom feeds.

For the record, here are the routes I used to map my drupal urls to PetersBlogger urls. This is from config/routes.rb:

   1  map.root :controller => 'post', :action => 'list'
   2  
   3  #
   4  # map the various rss feeds
   5  #
   6  map.connect 'tag/:tagname/feed', :controller => 'tag', :action => 'rss'
   7  map.connect 'tags/:tagname/feed', :controller => 'tag', :action => 'rss'
   8  map.connect 'tag/feed/:tagname', :controller => 'tag', :action => 'rss'
   9  map.connect 'blog/feed/1', :controller => 'post', :action => 'rss'
  10  map.connect 'blog/feed', :controller => 'post', :action => 'rss'
  11  map.connect 'blog/1/feed', :controller => 'post', :action => 'rss'
  12  map.connect 'rss.xml', :controller => 'post', :action => 'rss'
  13  
  14  #
  15  # map the atom feeds
  16  #
  17  map.connect 'tags/:tagname/atom.xml', :controller => 'tag', :action => 'atom'
  18  map.connect 'node/:id/atom.xml', :controller => 'post', :action => 'atom'
  19  map.connect 'atom/feed', :controller => 'post', :action => 'atom'
  20  
  21  #
  22  # Map pages
  23  #
  24  map.connect 'node/view/:id', :controller => 'post', :action => 'show'
  25  map.connect 'node/:id', :controller => 'post', :action => 'show'
  26  # not sure what below was for
  27  map.connect 'node/:id/tag/function.require', :controller => 'post', :action => 'show'
  28  map.connect 'node/:id/function.require', :controller => 'post', :action => 'show'
  29  map.connect 'tag/:tagname', :controller => 'post', :action => 'listbytag'
  30  map.connect 'tags/:tagname', :controller => 'post', :action => 'listbytag'
  31  map.connect 'tags/:tagnamex/tags/:tagname', :controller => 'post', :action => 'listbytag'
  32  map.connect 'blog/tags/:tagname', :controller => 'post', :action => 'listbytag'
  33  map.connect 'tag', :controller => 'tag', :action => 'list'
  34  
  35  # Install the default routes as the lowest priority.
  36  map.connect ':controller/:action/:id'
  37  map.connect ':controller/:action/:id.:format'

I fixed most of the above by searching the access logs for 404's (page not found) and looking for patterns. I'm not sure why drupal felt the need to support so many variants on urls for the same page/feed or how people managed to find them. Note the silly 'tags/:tagnamex/tags/:tagname': I had old urls of the form '/tag/213/tag/34' because of sloppy relative linking.

Rails routing is sublimely simple and flexible. It would have taken me days to get all the above working with mod_rewrite rules.

I've been testing my C libraries by compiling them into ruby extensions and then using ruby's unit testing to test them. Under Windows I had it all running nicely and I could run a main test script that would invoke all the other test scripts. Sweet.

Then it came to getting it running under linux (the target system) and while the unit tests ran fine individually, when I tried my main test script I was getting weird errors from my test extensions, including segment violations (!).

It took me a while to track down the problem but it turned out to be due to the behaviour of gcc and the linux program loader. All my test libraries were loaded into one ruby process and because they all contained functions with similar names it became a bit of a lottery as to which function in which library was called. I am used to the behaviour of windows dll's where all functions are by default hidden from the outside world unless you explicity export them. gcc/elf/whatever seems to export everything from shared libraries (linux equivalent of a dll) by default with hilarious results.

In version 4 of gcc they have added a facility for hiding these internal functions. I solved my problem as follows:

• Add '-fvisibility=hidden' to the gcc arguments to cause all functions to be hidden by default.
• Add this code to my stddefs.h:

     1  #ifdef _MSC_VER
     2    #ifdef BUILDING_DLL
     3      #define DLLEXPORT __declspec(dllexport)
     4    #else
     5      #define DLLEXPORT __declspec(dllimport)
     6    #endif
     7    #define DLLLOCAL
     8  #else
     9    #define DLLEXPORT __attribute__ ((visibility("default")))
    10    #define DLLLOCAL __attribute__ ((visibility("hidden")))
    11  #endif
  

• Change the functions that ruby calls to load the ruby extension to the following form (i.e. add DLLEXPORT):

     1  /*
     2   * Initialise the test class.
     3   */
     4  DLLEXPORT void Init_testtimer( void)
     5  {
     6      VALUE oMyClass = rb_define_class( "TestTimer", rb_cObject);
     7      rb_define_method( oMyClass, "initialize", test_initialise, 1);
     8      rb_define_method( oMyClass, "TimerSet", test_TimerSet, 1);
     9      rb_define_method( oMyClass, "TimerUpdate", test_TimerUpdate, 1);
    10      rb_define_method( oMyClass, "TimerStart", test_TimerStart, 1);
    11      rb_define_method( oMyClass, "TimerTest", test_TimerTest, 0);
    12  }
  

These functions are the only ones that need to be visible outside the library.

Results:

If you are really bored, you might like to check out my new Archive Page.

I thought about putting it in the sidebar but I didn't for a few reasons:

• it's ugly
• it takes time to generate the page and I didn't want the rendering of every url that wasn't already cached to require this hit.
• who cares anyway? It's mainly there to open up the older posts to search bots without them having to go through page/next 100 times.

On a more technical level, it uses the following route to give 'nice urls' for the year/month combinations:

map.connect 'post/on/:year/:month', :controller => 'post', :action => 'listbyyearmonth'
map.connect 'post/on/:year', :controller => 'post', :action => 'listbyyearmonth'

so March 2005 would be '/post/on/2005/3' which is cool.