Peter's Blog

Up till now to put pictures in this blog I have had to:

• go into an image editor to resize the file to suit the blog (about 450-500 pixels wide).
• fire up an sftp program to upload the file to /var/www/PetersBlogger/public/images
• add a tag such as

  [/images/TheNewPicture.jpg]
  

  to the wilki.

It was a bit laborious, especially the SFTP bit which is why I didn't do it very often.

I've now added better support for pictures to PetersBlogger. Here is an example:

Girls Aloud

if you click on this you should be treated to a larger version of the image, thanks to lightbox a javascript library.

PetersBlogger now supports a database of pictures so I can add a new picture, give it a name and upload it through the administration pages. The code uses the name as a caption under the photo and as alt text. I can use the name directly in my wilki text to choose which pictures to display.

The database of pictures is handled by the attachmentfu rails plugin. This article gives the steps required to get attachmentfu up and running.

The setup automatically handles resizing images to three sizes: thumbnail for listing in my picture admin page, a version that is no more than 500 pixels in either dimension for showing in the blog and a version that is no more than 800x600 for showing zoomed when you click on a picture. Attachmentfu has the sense to keep the aspect ratio correct automatically so I don't have to worry about portrait/landscape. I had to hack the imagescience library that does the resizing as it was hard-wired to save jpegs using JPEG_QUALITYSUPERB mode so the blog-sized images were taking up 150k. I changed this to JPEG_QUALITYAVERAGE and the size dropped to 15k or so, but it was a bit noisy so I've settled on JPEG_QUALITYGOOD and 38k images. You are on broadband?

I was slightly worried about how to change pictures, i.e. to upload a different version of a picture to replace an existing one, since the attachmentfu article I mentioned does not cover this. The answer is to use the exact same view code in the 'edit' page as in the 'new' page. Combined with rails scaffold code it Just Works! It even handles deleting the three old pictures and creating the new files.

My main motivation for doing this was so I can do it on the websites I host for other people and make it easier for them to upload pictures. PetersCMS is still in the planning stages but one problem has been cracked.

In a mammoth programming session I implemented proper smileys

[[':-)', 'smile'],
  [':-(', 'sad'],
  ['8-)', 'cool'],
  [';-)', 'wink']].each do |strCode, strFile|
  strText.gsub!( strCode, "<img alt=\"#{strFile}\" src=\"/images/smileys/#{strFile}.png\" />")
end

My fingers were bleeding from all the typing Worked first time though

There is even alt text for blind people

I figured out how to scrape the Wow Armory for up-to-date information about my characters using ruby. This means I can do this directly in PetersBlogger without using third party signature generators that disappear overnight or get absorbed into wow gold sites.

   1    require 'net/http'
   2  
   3    #
   4    # Scrape wow character info.
   5    #
   6    def WoWInfo
   7      begin
   8        strRealm = 'Eonar'
   9  
  10        oInfo = []
  11  
  12        ['Pookypoo', 'Maevyn', 'Maezyn', 'Maexyn'].each do |strCharacter|
  13          oCharInfo = []
  14  
  15          #
  16          # Open url.
  17          # Need to specify firefox as user agent as this makes the server return an XML
  18          # file.
  19          # Look the data up in the european armory. Change this for US.
  20          # If this is not done we get html.
  21          oResp = Net::HTTP.start( "armory.wow-europe.com", 80) do |http|
  22                http.get( "/character-sheet.xml?r=#{strRealm.tr( ' ', '+')}&n=#{strCharacter}",
  23                            { 'user-agent' => 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4'})
  24          end
  25  
  26          oDoc = REXML::Document.new oResp.body
  27          oDoc.elements[1].elements.each( 'characterInfo/character') do |oElement|
  28            oCharInfo << {:race => oElement.attributes['race']}
  29            oCharInfo << {:class => oElement.attributes['class']}
  30            oCharInfo << {:level => oElement.attributes['level']}
  31          end
  32  
  33          oDoc.elements[1].elements.each( 'characterInfo/characterTab/talentSpec') do |oElement|
  34            oCharInfo << {:spec => "#{oElement.attributes['treeOne']}/#{oElement.attributes['treeTwo']}/#{oElement.attributes['treeThree']}"}
  35          end
  36  
  37          oDoc.elements[1].elements.each( 'characterInfo/characterTab/professions/skill') do |oElement|
  38            oCharInfo << {oElement.attributes['key'].to_sym => oElement.attributes['value']}
  39          end
  40  
  41          oInfo << {strCharacter => oCharInfo}
  42        end
  43  
  44        return oInfo
  45      rescue
  46        return []
  47      end
  48    end

This lot generates the info in this page which is linked to under 'My Characters' over to the right. It should be fairly up-to-date as the cache will flush once a day causing the armory to be requeried. Currently the output is presented in a rails view thus:

   1  <% cache( :part => :wow) do %>
   2    <table>
   3    <% WoWInfo().each do |oCharInfo| %>
   4      <% oCharInfo.each_pair do |strChar, oInfo| %>
   5        <tr><td colspan="2"><b><%= strChar %></b></td></tr>
   6        <% oInfo.each do |oItem| %>
   7          <% oItem.each_pair do |strItem, strValue| %>
   8            <tr>
   9              <td><%= strItem %>:</td><td><%= strValue %></td>
  10            </tr>
  11          <% end %>
  12        <% end %>
  13      <% end %>
  14    <% end %>
  15    </table>
  16  <% end %>

All the work is done by the WoWInfo method which I put in the application helper. I didn't use a controller method for this.

I'm going to think about a nicer presentation with some pictures of the actual guys rather than generic pictures of bald redhead dwarves.

UPDATE: I've added my old Aerie Peak toons to the output but the code above still stands. Lugulas was my old auction alt who I've never named here before, hence level 10 with level 75 (dis)enchanting. I find myself missing them...

Rake is the ruby equivalent of make except it is much better in that you are scripting in ruby and not the nasty primeval make language. Rails uses rake a lot as a simple place to gather together a load of administration tasks such as running tests, running database migrations etc.

When I moved all my drupal blog posts from drupal over to PetersBlogger I decided to use a nice organised rake script rather than do my usual practise of having a directory full of utility scripts.

In my rails installation I created a file called lib/tasks/jobs.rake and in it I put my import code:

   1  task (:import_drupal => :environment) do
   2    print "Importing Drupal\n"
   3    oTagTable = {}
   4    #
   5    # Import all the articles.
   6    #
   7    Drupal::Article.find(:all, :conditions => "type <> 'image'").each do |oArticle|
   8      print "#{oArticle.title}\n"
   9      oTime = Time.at( oArticle.created)
  10      oPost = Post.create( :nid => oArticle.nid,
  11                          :title => oArticle.title,
  12        :body => oArticle.body,
  13        :created => oTime)
  14  
  15      #
  16      # import the tags related to the article.
  17      #
  18      oArticle.tags.each do |oTag|
  19        if oTagTable[oTag.tag]
  20          oNewTag = oTagTable[oTag.tag]
  21        else
  22          oNewTag = Tag.new( :tag=>oTag.tag)
  23          oTagTable[oTag.tag] = oNewTag
  24        end
  25  
  26        oPost.tags << oNewTag
  27      end
  28  
  29      #
  30      # Import the comments for the article.
  31      #
  32      oArticle.comments.each do |oComment|
  33        strWho = oComment.name
  34        strComment = oComment.comment
  35        if strWho == nil or strWho == ""
  36          strWho = "Peter"
  37        end
  38        if strComment == nil or strComment.strip == ""
  39          strComment = "{nothing to say}"
  40        end
  41        oPost.comments << Comment.create( :who => strWho,
  42            :body => strComment,
  43            :created => Time.at( oComment.timestamp), :published => 1)
  44        if not oPost.comments[-1].valid?
  45          p oPost.comments[-1]
  46          p oPost.comments[-1].errors
  47        end
  48      end
  49  
  50      oPost.save!
  51    end
  52  end

This creates a rake task called 'import_drupal' which I can invoke simply by running

rake import_drupal

The task runs with the appropriate rails environment so all my models are in place. I had to create a model to access the records in the drupal database thus:

   1  module Drupal
   2    class Comment < ActiveRecord::Base
   3      establish_connection configurations['drupal']
   4      set_table_name 'comments'
   5    end
   6    class Tag < ActiveRecord::Base
   7      establish_connection configurations['drupal']
   8      set_primary_key 'tid'
   9      has_and_belongs_to_many :articles,
  10        :class_name => 'Drupal::Article',
  11        :join_table => 'awtags_node',
  12        :foreign_key => 'tid',
  13        :association_foreign_key => 'nid'
  14      set_table_name 'awtags'
  15    end
  16    class Article < ActiveRecord::Base
  17      establish_connection configurations['drupal']
  18      set_primary_key 'nid'
  19      self.inheritance_column = 'poopy'
  20      has_many :comments, :table_name => 'comments', :foreign_key => 'nid', :class_name => 'Drupal::Comment'
  21      has_and_belongs_to_many :tags,
  22        :class_name => 'Drupal::Tag',
  23        :join_table => 'awtags_node',
  24        :foreign_key => 'nid',
  25        :association_foreign_key => 'tid'
  26      set_table_name 'node'
  27    end
  28  end

This model handles all the relationships between the records, including the has_and_belongs_to_many relationship between posts and tags (provided by awtags).

I needed an entry in config/database.yml to define the connection to the drupal database:

drupal:
  adapter: mysql
  database: petersblog
  username: secret
  password: doyouthinkidpostithere

and it all works nicely. If I invoke it thusly:

rake import_drupal RAILS_ENV=production

then it will import the posts into the rails production database.

I'm giving away a few of my schema secrets here which pedantic rails developers might leap on, specifically I have used my own fields called 'nid' and 'created' to perform roles that rails already provides 'id' and 'created_on' for. The reason for the duplication was that during development I was having problems setting these fields to the values from the drupal database, rails was insisting on giving them it's own values. By having my own fields I had total control. There are no doubt ways around this but I didn't have time to find them (although I did find other people moaning about the same problem).

And that's how I managed to save 1500 posts from the hell that is php.

If you are really bored, you might like to check out my new Archive Page.

I thought about putting it in the sidebar but I didn't for a few reasons:

• it's ugly
• it takes time to generate the page and I didn't want the rendering of every url that wasn't already cached to require this hit.
• who cares anyway? It's mainly there to open up the older posts to search bots without them having to go through page/next 100 times.

On a more technical level, it uses the following route to give 'nice urls' for the year/month combinations:

map.connect 'post/on/:year/:month', :controller => 'post', :action => 'listbyyearmonth'
map.connect 'post/on/:year', :controller => 'post', :action => 'listbyyearmonth'

so March 2005 would be '/post/on/2005/3' which is cool.

Since I added the page count thing yesterday (that only I can see since I don't want everyone to know how often I tweek the site and reboot) I got this:

I think I've managed to map all the urls from my old drupal blog to this new blog: according to statcounter (which is a bit slow loading today) I had 1,103 page loads yesterday compared to 1,167 last week on the old blog. Any missed urls would lead to the 404 page which doesn't have the hit counter or statcounter on it (hum, could catch the 404's and redirect them to main page with a flash notice..).

Statcounter works under javascript and so does not count bots and paranoid people with javascript disabled. It is interesting therefore that the latter outnumber the javascript users by 2.5:1. And none of this includes rss/atom feeds.

For the record, here are the routes I used to map my drupal urls to PetersBlogger urls. This is from config/routes.rb:

   1  map.root :controller => 'post', :action => 'list'
   2  
   3  #
   4  # map the various rss feeds
   5  #
   6  map.connect 'tag/:tagname/feed', :controller => 'tag', :action => 'rss'
   7  map.connect 'tags/:tagname/feed', :controller => 'tag', :action => 'rss'
   8  map.connect 'tag/feed/:tagname', :controller => 'tag', :action => 'rss'
   9  map.connect 'blog/feed/1', :controller => 'post', :action => 'rss'
  10  map.connect 'blog/feed', :controller => 'post', :action => 'rss'
  11  map.connect 'blog/1/feed', :controller => 'post', :action => 'rss'
  12  map.connect 'rss.xml', :controller => 'post', :action => 'rss'
  13  
  14  #
  15  # map the atom feeds
  16  #
  17  map.connect 'tags/:tagname/atom.xml', :controller => 'tag', :action => 'atom'
  18  map.connect 'node/:id/atom.xml', :controller => 'post', :action => 'atom'
  19  map.connect 'atom/feed', :controller => 'post', :action => 'atom'
  20  
  21  #
  22  # Map pages
  23  #
  24  map.connect 'node/view/:id', :controller => 'post', :action => 'show'
  25  map.connect 'node/:id', :controller => 'post', :action => 'show'
  26  # not sure what below was for
  27  map.connect 'node/:id/tag/function.require', :controller => 'post', :action => 'show'
  28  map.connect 'node/:id/function.require', :controller => 'post', :action => 'show'
  29  map.connect 'tag/:tagname', :controller => 'post', :action => 'listbytag'
  30  map.connect 'tags/:tagname', :controller => 'post', :action => 'listbytag'
  31  map.connect 'tags/:tagnamex/tags/:tagname', :controller => 'post', :action => 'listbytag'
  32  map.connect 'blog/tags/:tagname', :controller => 'post', :action => 'listbytag'
  33  map.connect 'tag', :controller => 'tag', :action => 'list'
  34  
  35  # Install the default routes as the lowest priority.
  36  map.connect ':controller/:action/:id'
  37  map.connect ':controller/:action/:id.:format'

I fixed most of the above by searching the access logs for 404's (page not found) and looking for patterns. I'm not sure why drupal felt the need to support so many variants on urls for the same page/feed or how people managed to find them. Note the silly 'tags/:tagnamex/tags/:tagname': I had old urls of the form '/tag/213/tag/34' because of sloppy relative linking.

Rails routing is sublimely simple and flexible. It would have taken me days to get all the above working with mod_rewrite rules.

I wanted PetersBlogger to have a preview facility when composing posts that would automatically run the text I was editing through my wilki filter as I typed and give me a live preview of how the post would turn out when it was formatted and displayed. No pressing a preview button for me, I wanted it to just happen. Sounds like a job for AJAX. But how to do this in rails? Well it turns out to be very easy.

I put this in my form where I edit a post:

<p><%= f.text_area 'body', :size => "100x40", :html => { :id => :body} %></p>
<%= observe_field :post_body, :url => { :action => :wilkify },
    :frequency => 3.00,
    :update => :preview,
    :with => "'text='+escape( $('post_body').value)"
%>
<div id="preview"/>

This puts some javascript in the page that monitors the text_area where I am editing and, if it changes it will invoke the 'wilkify' action in my controller, passing it what I have typed thus far.

The 'wilkify' action is very simple, it just gets the text, wilkifies it and makes sure it is rendered as a simple piece of html, without the main site layout:

def wilkify
  strBody = params[:text]
  strBody = CGI::unescape( strBody)
  @text = Wilki( strBody)
  render :layout => false
end

I have a small view defined to wrap the preview text in a couple of div's, just as if they were appearing on the site:

<div class="node">
  <div class="bodynode">
    <div class="entry">
      <%= @text %>
    </div>
  </div>
</div>

The wikified text is then inserted into the edit form in the div with the id 'preview'.

I have to add the prototype javascript library to the head section of the main application layout:

<%= javascript_include_tag 'prototype' %>

It is not entirely perfect and I would never let strangers use it because they could type malformed html and totally mess things up but it is good enough for me.

I've missed blogging while I've been working on PetersBlogger. My old drupal blog was limping along: while administering the site it kept logging me out. I can admit now that I was running an old version of drupal (4.7?) which I couldn't upgrade due to module dependencies (mainly awtags). The old code wasn't running that smoothly under php5 despite initial appearancies.

I'm happy with my new blog platform, it's running really nicely on my slicehost slice and I've learnt an awful lot about rails while developing it. I will try to do a braindump of rails experiencies in forthcoming posts (cross fingers).

Drupal is a fine CMS and I'm using Drupal 5 on my company intranet. My thoughts on a drupal vs ruby on rails debate would be:

• drupal/php/apache is easier to deploy than a rails/ruby/mongrel/nginx deploy: I moved my drupal site between about four different hosts and did it each time in less than an hour. A rails setup is more convoluted (especially mongrel_cluster). However, this site is running much faster under rails than it was under drupal. The difference is probably down to php and apache being more mainstream and slightly more refined and hence easy to set up. I'm not saying rails is a total pain, just that if I only had ten minutes in which to deploy a site I would be reaching for drupal. If I only wanted to install the source and a few modules and never touch any coding then I would be happy with drupal.
• I love ruby and rails development. There is no comparison, php seems to me as much a bastard language as visual basic 6. Ruby was cleanly developed as an object orientated language, php is having object orientation grafted on as an afterthough.
• If I look through my drupal source I see no unit testing. Ruby/Rails has unit testing built in and I'm completely sold on it. I don't think I will ever trust any code (especially code I write) if it hasn't been unit tested. Some interesting aspects of unit testing:
  • I understand now the 'test driven' approach to development where you design a module's api by first roughing out how the tests will work: the unit test is your first experience of using the new api. It's during testing that you learn how nice an api will be.
  • If the documentation of a new ruby/rails tool is dubious, look in the unit tests to see how the author intended it to be used. If there are no tests then run.
  • If I run into a tricky bug, it's better to first reproduce it in unit tests, fix it there and then be happy that the bug will never recur. It's easier to debug code in a unit test (essentially a command line application) than on a live website.
• all ruby on rails applications follow basically the same architecture. It is fairly easy to figure out how a new application works. Every php application is different. I know I'm comparing a web framework to a programming language but none of the php applications (drupal, phpmyadmin) or am familiar with (wordpress) share a common architecture. New php application to maintain? New learning curve.
• my new site is being hammered with attempts to post to /comments/reply, even though it doesn't use that url for comment submission. The comment spammers know drupal and know where to test the locks. If any of them ever bother to try to find my comment submission url they will only find the same captcha that protected my drupal site (which was much easier to implement in rails, simply as a validation on the model).

This blog is now running on my new rails based blogging platform which I have called PetersBlogger (I'm not good at names).

It's mostly complete. Features:

• post articles (or you wouldn't be reading this)
• wilki formatting including syntax highlighting with ultraviolet e.g.

  strThis = "that"
  

• comments, including
  • moderation list
  • Peter's simplistic captcha system
• tagging
• pingomatic submission
• Technorati tags
• rss feeds (no atom yet)
• uses fragment caching for performance

Todo list:

• AJAX formatting preview
• atom feeds (do I care?)
• search (or rely on google)
• tag cloud
• archives by month/year: you DO care what I wrote in Feb 2005 right?
• gimmicks

Please let me know if you spot any sillies (that's if comment submission is working at all).

It's nice to use because the administration interface isn't split out into a seperate set of web pages, when I 'log in' I simply get extra buttons to press in the pages that you are seeing.