Peter's Blog

I have been using the Twitter gem to post tweets regarding new blog updates. During some testing I was seeing bare passwords in urls which prompted me into migrating to using Oauth to authenticate with twitter. However this proved to be a bit of a minefield. Oauth tends to be described in explicit technical details and the gems available have cursory examples based on the simplest use cases and they never explain how to get the tokens.

So here is how to use oauth with twitter in enough detail for other people to do it. The main problem is that even if you are writing, say, a command line twitter app you will still need to write a web based app to get twitter to provide you with the right authentication keys as the process involves being redirected through the twitter web site.

First go here and ask twitter nicely for your Consumer Token and your Consumer Secret. I won't bore you with what these are, if you want to be bored read the oauth docs.

Stick these in a yaml file like this:

---
accountname:
  consumer_secret: ulasdfjasdhflkshdflkjsdfhasdfasdfasdffs
  consumer_token: 9asdjfhkasjdfhkhkjhkjA

I put this in RAILS_ROOT/config/twitter.yml

Now install the twitter gem:

sudm gem install twitter

Here is a library to encapsulate reading and writing these tokens. I called it PetersTwit.rb and put it in RAILS_ROOT/lib:

   1  #
   2  # Simple wrapper for twitter oauth tokens
   3  #
   4  
   5  require 'rubygems'
   6  require 'twitter'
   7  require 'yaml'
   8  
   9  class PetersTwit
  10    #
  11    # Initialise oauth stuff. strConfig points to the yaml config file,
  12    # strAccount is the name of an account in the file (need not be same name
  13    # as twitter account)
  14    #
  15    def initialize( strConfig, strAccount)
  16      @strConfig = strConfig
  17      @strAccount = strAccount
  18  
  19      @oTokens = nil
  20  
  21      begin
  22        @oAllTokens = YAML::load_file( strConfig)
  23        @oTokens = @oAllTokens[@strAccount]
  24      rescue
  25        @oAllTokens = {}
  26      end
  27  
  28      if not @oTokens
  29        @oTokens = {
  30          'consumer_token' => nil,
  31          'consumer_secret' => nil,
  32          'access_token' => nil,
  33          'access_secret' => nil,
  34        }
  35  
  36        @oAllTokens[@strAccount] = @oTokens
  37      end
  38    end
  39  
  40    def consumer
  41      OAuth::Consumer.new(
  42         @oTokens['consumer_token'],
  43         @oTokens['consumer_secret'],
  44         {:site => 'http://twitter.com'}
  45      )
  46    end
  47  
  48    def SetAccessToken( strToken, strSecret)
  49      @oTokens['access_token'] = strToken
  50      @oTokens['access_secret'] = strSecret
  51      @oAllTokens[@strAccount] = @oTokens
  52      Save()
  53    end
  54  
  55  
  56    def Save
  57      YAML::dump( @oAllTokens, open( @strConfig, "w"))
  58    end
  59  
  60    def Auth
  61      oAuth = Twitter::OAuth.new( @oTokens['consumer_token'], @oTokens['consumer_secret'])
  62      oAuth.authorize_from_access( @oTokens['access_token'], @oTokens['access_secret'])
  63  
  64      return oAuth
  65    end
  66  end
Toggle Line Numbers

Now add the following actions to a handy rails controller you may have:

   1    def twitter_oauth_login
   2      oTwit = PetersTwit.new( File.join( RAILS_ROOT, 'config', 'twitter.yml'), 'therealpeter')
   3  
   4      request_token = oTwit.consumer.get_request_token
   5      session[:request_token] = request_token.token
   6      session[:request_token_secret] = request_token.secret
   7  # Send to twitter.com to authorize
   8      redirect_to request_token.authorize_url
   9    end
  10  
  11    def twitter_oauth_register
  12      oTwit = PetersTwit.new( File.join( RAILS_ROOT, 'config', 'twitter.yml'), 'therealpeter')
  13  
  14      request_token = OAuth::RequestToken.new( oTwit.consumer, session[:request_token], session[:request_token_secret])
  15  
  16      #
  17      # Exchange the request token for an access token.
  18      #
  19      access_token = request_token.get_access_token( :oauth_verifier => params[:oauth_verifier])
  20  
  21      oTwit.SetAccessToken( access_token.token, access_token.secret)
  22  
  23      redirect_to( :action => :index)
  24    end
Toggle Line Numbers

When you register your application with twitter you can say it is a web application and where it asks for a callback url you can point to the url of the twitter_oauth_register. I did this but it never called me back

Run the rails app and go to the twitter_oauth_login url which should redirect you to twitter where you log into your twitter account (I'm not phishing here, honest) and authorise twitter to let your application connect to twitter without any more tedious password nonsense.

In an ideal world twitter will redirect you back to your app but this is broken for me. Instead Twitter acts as if I asked for a desktop app and displays a seven digit PIN. This can still be used by manually entering the url for:

http://yourserver/your_controller/twitter_oauth_register?oauth_verifier=1234567

where 1234567 is the PIN that twitter gave you.

After this you should have values for access_token and access_secret appear in your yaml file. These are good, keep these safe as they will let you post to twitter without passwords and crap. You can use them anywhere: web app or command line app. They allow your app to access twitter under a particular twitter user name.

Posting to twitter from the command line is now easy:

require 'PetersTwit'
require 'twitter'

oPTwit = PetersTwit.new( File.join( RAILS_ROOT, 'config', 'twitter.yml'), 'therealpeter')
oTwit = Twitter::Base.new( oPTwit.Auth())

oTwit.update( 'OMG jonas brothers')

When developing this the biggest conceptual hurdles came in the handling of the exchange of request tokens and access tokens. What can be confusing is that this can only be done once: if you get a set of request tokens you have one chance to swap them for access tokens and if you screw it up you need a fresh set of request tokens. But you don't need to know that if you follow my recipe, it's an implementation detail.

10 lines of ruby, one blank. Sorry no comments.

Twitter Client

Gotta love ruby gems.

My conscience is leading me more and more into formal testing and I have been trying out cucumber the latest ruby Behaviour Driven Development offering. Cucumber is still very new and there is little documentation available so here are some notes.

The basic idea of cucumber is along the lines that your tests are almost in spoken language rather that a programming language and hence non technical people like the marketing department, clients etc can read them and understand them, although I just cannot believe that they could ever write them in any useful way. The tests describe the behaviour of the systems and hence document it in a similar way to a requirements specification. If one decided, for example, to go with the herd one could take the behaviour listings as a specification for reimplementing everything in .NET.

Or something like that.

Anyway, here is a cucumber 'feature' that I have come up with for pctv. This describes the features of the system. Pctv shows a list of tv programs, one can click on a program and see a list of episodes of that program.

Feature: Looking at programme listing
    In order to watch programs
    I should be able to
    See listings of what is on

    Scenario: Look at programs
        Given I have some programmes
            And I visit "/"
        Then I should see "Dora the Explorer"
            And I should see "Loose Women"

    Scenario: Examine Dora the Explorer
        Given I have some programmes
            And I visit "/"
        When I follow "Dora the Explorer"
        Then I should see "Dora the Explorer"
            And I should see "Tomorrows Episode"
            And I should not see "Yesterdays Episode"
            And I should not see "Loose Women"
            And the "Series Link" checkbox should not be checked

    Scenario: Mark Dora series link
        Given I have some programmes
            And I visit "/"
        When I follow "Dora the Explorer"
            And I check "Series Link"
            And I press "Update"
        Then I should see "Dora the Explorer"
            And I should see "Tomorrows Episode"
            And I should not see "Yesterdays Episode"
            And I should not see "Loose Women"
            And the "Series Link" checkbox should be checked

    Scenario: Star Trek series link
        Given I have some programmes
            And I visit "/"
        When I follow "Star Trek"
        Then I should see "Star Trek"
            And the "Series Link" checkbox should be checked
            And I should see "Boldly Gone"
            And I should see "Boldly Going"
            And I should not see "Yesterdays Episode"
            And I should not see "Loose Women"

    Scenario: Unmark Star Trek series link
        Given I have some programmes
            And I visit "/"
        When I follow "Star Trek"
            And I uncheck "Series Link"
            And I press "Update"
        Then I should see "Star Trek"
            And the "Series Link" checkbox should not be checked
            And I should see "Boldly Gone"
            And I should see "Boldly Going"
            And I should not see "Yesterdays Episode"
            And I should not see "Loose Women"

    Scenario: Visit recordings page
        Given I have some programmes
            And I visit "/"
        When I follow "Recordings"
        Then I should see "Star Trek"
            And I should see "Boldly Gone"
            And I should see "Recorded"
            And I should not see "Boldly Going"

    Scenario: Examine Categories of Women
        Given I have some programmes
            And I visit "/"
        When I follow "Women"
        Then I should see "Loose Women"
            And I should not see "Dora the Explorer"
            And I should not see "Star Trek"

    Scenario: Search for Dora
        Given I have some programmes
            And I visit "/programmes"
        When I follow "Search"
            And I fill in "Search" with "Dora"
            And I press "Seek"
        Then I should see "Dora the Explorer"
            And I should not see "Loose Women"
            And I should not see "Star Trek"

I think this works at the level of being quite readable and describes nicely how one interacts with pctv.

This uses the webrat gem which simulates the interactions with a web browser (seeing text, clicking links etc).

The Given, When and Then keywords translate directly to code steps which establish the state of the system, an event that changes the system and the results of the change respectively. When cucumber is set up in a rails application with:

script/generate cucumber

it will generate a file called /features/step_definitions/webrat_steps.rb which contains some useful webrat examples such as:

When /^I press "(.*)"$/ do |button|
  clicks_button(button)
end

Then /^I should see "(.*)"$/ do |text|
  response.body.should =~ /#{text}/m
end

So the When term is converted into webrat operations and the Then term examines the html generated by the code. It should be noted that webrat does nothing in respect to examining the output, that can be done directly on the response object or it can be done using rspecs have_tag method:

Then /it should have the tag "(.*)" containing the text "(.*)"/ do |strTag, strText|
  response.should have_tag( strTag, /#{strText}/)
end

The above is one of mine and I'm a bit uneasy about it. I'm not sure if the cucumber level of testing should be above the implemntation and above the level of examining the html in the response or whether it should be at the level of what a user would actually see on a web site. The studying of the html should be at the level of view tests using rspec-rails but this is just a feeling.

Here is another one of mine:

Given /^I visit "(.*)"$/ do |url|
  visits(url)
end

This seems like an obvious one to need and I'm not sure why the generated webrat steps don't include it, it makes me wonder if I haven't quite grasped the philosophy.

Cucumber doesn't seem to support fixtures and googling for this I came across a thread where someone was asking about this issue and the author of cucumber said 'google for it': there is no FM to R. Other posters in the thread were more helpful but I still found that the fixtures were only loaded in the first scenario and for following scenarios the database stayed empty. I decided to just code something:

   1  Given /I have some programmes/ do
   2    strYesterday = (DateTime.now - 1).strftime( "%Y%m%d")
   3    strTomorrow = (DateTime.now + 1).strftime( "%Y%m%d")
   4  
   5    oChannel1 = Channel.create( :dvbname => 'Nick Jr')
   6    oChannel2 = Channel.create( :dvbname => 'ITV ONE')
   7  
   8    oCategory1 = Category.create( :name => 'Children')
   9    oCategory2 = Category.create( :name => 'Crap')
  10  
  11    oProgram = Programme.create( :title => "Dora the Explorer", :category => oCategory1)
  12    oEpisode = Episode.create( :programme => oProgram, :subtitle => "Episode 1", :description => "Yesterdays Episode", :channel => oChannel1, :starting => DateTime.parse( strYesterday + "080000"), :ending => DateTime.parse( strYesterday + "080000"))
  13    oEpisode2 = Episode.create( :programme => oProgram, :subtitle => "Episode 2", :description => "Tomorrows Episode", :channel => oChannel1, :starting => DateTime.parse( strTomorrow + "080000"), :ending => DateTime.parse( strTomorrow + "080000"))
  14  
  15    oProgram2 = Programme.create( :title => "Loose Women", :category => oCategory2)
  16    oEpisode2 = Episode.create( :programme => oProgram2, :description => "You did it where?", :channel => oChannel2, :starting => DateTime.parse( strTomorrow + "123000"), :ending => DateTime.parse( strTomorrow + "133000"))
  17  
  18    oProgram3 = Programme.create( :title => "Star Trek", :category => oCategory1, :series_link => true)
  19    oEpisode1 = Episode.create( :programme => oProgram3, :description => "Boldly Gone", :channel => oChannel2, :starting => DateTime.parse( strTomorrow + "013000"), :ending => DateTime.parse( strYesterday + "023000"),
  20                                  :recording_status => 'recorded', :filename => 'Star Trek.avi')
  21    oEpisode2 = Episode.create( :programme => oProgram3, :description => "Boldly Going", :channel => oChannel2, :starting => DateTime.parse( strTomorrow + "013000"), :ending => DateTime.parse( strTomorrow + "023000"))
  22  end
Toggle Line Numbers

I put the following in /features/support/env.rb

require "webrat/rails"

and run it using

spec features

and I get:

   1  Feature: Looking at programme listing  # features/programmes.feature
   2      In order to watch programs
   3      I should be able to
   4      See listings of what is on
   5    Scenario: Look at programs               # features/programmes.feature:6
   6      Given I have some programmes           # features/step_definitions/pctv_steps.rb:1
   7      And I visit "/"                        # features/step_definitions/pctv_steps.rb:24
   8      Then I should see "Dora the Explorer"  # features/step_definitions/webrat_steps.rb:85
   9      And I should see "Loose Women"         # features/step_definitions/webrat_steps.rb:85
  10  
  11    Scenario: Examine Dora the Explorer                     # features/programmes.feature:12
  12      Given I have some programmes                          # features/step_definitions/pctv_steps.rb:1
  13      And I visit "/"                                       # features/step_definitions/pctv_steps.rb:24
  14      When I follow "Dora the Explorer"                     # features/step_definitions/webrat_steps.rb:10
  15      Then I should see "Dora the Explorer"                 # features/step_definitions/webrat_steps.rb:85
  16      And I should see "Tomorrows Episode"                  # features/step_definitions/webrat_steps.rb:85
  17      And I should not see "Yesterdays Episode"             # features/step_definitions/webrat_steps.rb:89
  18      And I should not see "Loose Women"                    # features/step_definitions/webrat_steps.rb:89
  19      And the "Series Link" checkbox should not be checked  # features/step_definitions/webrat_steps.rb:97
  20  
  21    Scenario: Mark Dora series link                     # features/programmes.feature:22
  22      Given I have some programmes                      # features/step_definitions/pctv_steps.rb:1
  23      And I visit "/"                                   # features/step_definitions/pctv_steps.rb:24
  24      When I follow "Dora the Explorer"                 # features/step_definitions/webrat_steps.rb:10
  25      And I check "Series Link"                         # features/step_definitions/webrat_steps.rb:69
  26      And I press "Update"                              # features/step_definitions/webrat_steps.rb:6
  27      Then I should see "Dora the Explorer"             # features/step_definitions/webrat_steps.rb:85
  28      And I should see "Tomorrows Episode"              # features/step_definitions/webrat_steps.rb:85
  29      And I should not see "Yesterdays Episode"         # features/step_definitions/webrat_steps.rb:89
  30      And I should not see "Loose Women"                # features/step_definitions/webrat_steps.rb:89
  31      And the "Series Link" checkbox should be checked  # features/step_definitions/webrat_steps.rb:93
  32  
  33    Scenario: Star Trek series link                     # features/programmes.feature:34
  34      Given I have some programmes                      # features/step_definitions/pctv_steps.rb:1
  35      And I visit "/"                                   # features/step_definitions/pctv_steps.rb:24
  36      When I follow "Star Trek"                         # features/step_definitions/webrat_steps.rb:10
  37      Then I should see "Star Trek"                     # features/step_definitions/webrat_steps.rb:85
  38      And the "Series Link" checkbox should be checked  # features/step_definitions/webrat_steps.rb:93
  39      And I should see "Boldly Gone"                    # features/step_definitions/webrat_steps.rb:85
  40      And I should see "Boldly Going"                   # features/step_definitions/webrat_steps.rb:85
  41      And I should not see "Yesterdays Episode"         # features/step_definitions/webrat_steps.rb:89
  42      And I should not see "Loose Women"                # features/step_definitions/webrat_steps.rb:89
  43  
  44    Scenario: Unmark Star Trek series link                  # features/programmes.feature:45
  45      Given I have some programmes                          # features/step_definitions/pctv_steps.rb:1
  46      And I visit "/"                                       # features/step_definitions/pctv_steps.rb:24
  47      When I follow "Star Trek"                             # features/step_definitions/webrat_steps.rb:10
  48      And I uncheck "Series Link"                           # features/step_definitions/webrat_steps.rb:73
  49      And I press "Update"                                  # features/step_definitions/webrat_steps.rb:6
  50      Then I should see "Star Trek"                         # features/step_definitions/webrat_steps.rb:85
  51      And the "Series Link" checkbox should not be checked  # features/step_definitions/webrat_steps.rb:97
  52      And I should see "Boldly Gone"                        # features/step_definitions/webrat_steps.rb:85
  53      And I should see "Boldly Going"                       # features/step_definitions/webrat_steps.rb:85
  54      And I should not see "Yesterdays Episode"             # features/step_definitions/webrat_steps.rb:89
  55      And I should not see "Loose Women"                    # features/step_definitions/webrat_steps.rb:89
  56  
  57    Scenario: Visit recordings page        # features/programmes.feature:58
  58      Given I have some programmes         # features/step_definitions/pctv_steps.rb:1
  59      And I visit "/"                      # features/step_definitions/pctv_steps.rb:24
  60      When I follow "Recordings"           # features/step_definitions/webrat_steps.rb:10
  61      Then I should see "Star Trek"        # features/step_definitions/webrat_steps.rb:85
  62      And I should see "Boldly Gone"       # features/step_definitions/webrat_steps.rb:85
  63      And I should see "Recorded"          # features/step_definitions/webrat_steps.rb:85
  64      And I should not see "Boldly Going"  # features/step_definitions/webrat_steps.rb:89
  65  
  66    Scenario: Examine Categories of Women       # features/programmes.feature:67
  67      Given I have some programmes              # features/step_definitions/pctv_steps.rb:1
  68      And I visit "/"                           # features/step_definitions/pctv_steps.rb:24
  69      When I follow "Women"                     # features/step_definitions/webrat_steps.rb:10
  70      Then I should see "Loose Women"           # features/step_definitions/webrat_steps.rb:85
  71      And I should not see "Dora the Explorer"  # features/step_definitions/webrat_steps.rb:89
  72      And I should not see "Star Trek"          # features/step_definitions/webrat_steps.rb:89
  73  
  74    Scenario: Search for Dora                # features/programmes.feature:75
  75      Given I have some programmes           # features/step_definitions/pctv_steps.rb:1
  76      And I visit "/programmes"              # features/step_definitions/pctv_steps.rb:24
  77      When I follow "Search"                 # features/step_definitions/webrat_steps.rb:10
  78      And I fill in "Search" with "Dora"     # features/step_definitions/webrat_steps.rb:14
  79        Could not find field labeled "Search" (RuntimeError)
  80        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/core/flunk.rb:4:in `flunk'
  81        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/core/locators.rb:16:in `field_labeled'
  82        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/core/locators.rb:10:in `field'
  83        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/core/scope.rb:183:in `locate_field'
  84        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/core/scope.rb:41:in `fills_in'
  85        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/rails.rb:88:in `send'
  86        /usr/lib/ruby/gems/1.8/gems/webrat-0.3.2/lib/webrat/rails.rb:88:in `method_missing'
  87        /usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/integration.rb:498:in `__send__'
  88        /usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/integration.rb:498:in `method_missing'
  89        ./features/step_definitions/webrat_steps.rb:15:in `And /^I fill in "(.*)" with "(.*)"$/'
  90        features/programmes.feature:79:in `And I fill in "Search" with "Dora"'
  91      And I press "Seek"                     # features/step_definitions/webrat_steps.rb:6
  92      Then I should see "Dora the Explorer"  # features/step_definitions/webrat_steps.rb:85
  93      And I should not see "Loose Women"     # features/step_definitions/webrat_steps.rb:89
  94      And I should not see "Star Trek"       # features/step_definitions/webrat_steps.rb:89
  95      And I press "Seek"                     # features/step_definitions/webrat_steps.rb:6
  96      Then I should see "Dora the Explorer"  # features/step_definitions/webrat_steps.rb:85
  97      And I should not see "Loose Women"     # features/step_definitions/webrat_steps.rb:89
  98      And I should not see "Star Trek"       # features/step_definitions/webrat_steps.rb:89
  99  
 100  
 101  58 steps passed
 102  1 steps failed
 103  4 steps skipped
Toggle Line Numbers

The error above is because webrat can't find the Search box in my form, although it looks good to me. Not the easist thing to debug, dump the html into the console and peer at it, viewing source in a web browser is easier. Webrat has a function save_and_open_page that only seems to work in OSX.

UPDATE: found the problem with the search term. My search form was set up like this:

<% form_tag(:controller => :programmes, :action => :search) do %>
  <p>
    <%= label_tag "Search" %> <%= text_field_tag :search_term %>

    <%= submit_tag "Seek" %>
  </p>
<% end %>

I kinda assumed that webrat used the fact that the label was before the text field in the html to associate the two but it turns out there is a correct way to do this association. The following works, the :search_term parameter to label_tag makes it generate a 'for' attribute that points to the text input field:

<% form_tag(:controller => :programmes, :action => :search) do %>
  <p>
    <%= label_tag :search_term, "Search" %> <%= text_field_tag :search_term %>

    <%= submit_tag "Seek" %>
  </p>
<% end %>

Webrat is helping me validate my html.

I still have a few problems with this:

• in the example of a checkbox, I can set up a test to check a checkbox and I can see it checked but as an engineer I'm itching to look in the database to make sure it is checked. I have to resist the urge to test at the cucumber level and tell myself the view, controller and model tests would have that sorted.
• to test that a checkbox can be unchecked I stuffed the database with a record that is already checked. I could do these steps:

      Scenario: Mark Dora series link
          Given I have some programmes
              And I visit "/"
          When I follow "Dora the Explorer"
              And I check "Series Link"
              And I press "Update"
              And I uncheck "Series Link"
              And I press "Update"
          Then I should see "Dora the Explorer"
              And I should see "Tomorrows Episode"
              And I should not see "Yesterdays Episode"
              And I should not see "Loose Women"
              And the "Series Link" checkbox should not be checked
  

  and I have to rely on having already tested that checking works. Writing now and thinking more, this seems ok.
• my specs are lacking in justifications. This happens in most specifications, things are detailed with no details on why things are this way. For example, in my test I click on 'Dora the Explorer' and I don't see 'Loose Women' because clicking on Dora the Explorer takes me to the episodes of Dora and Dora doesn't usually feature loose women. The test above doesn't state this unless I add comments which breaks the idea of the formal definition language. Maybe it needs something like 'because'?

      Then I should not see "Loose Women" Because that is not an episode of Dora
      And I should not see "Yesterdays Episode" Because it should only show future episodes
  

  However since I can define the Then terms myself there is nothing stopping me adding Because to my tests and this could then force me to put in my justifications!
• tests where you don't want to see something are fragile. I could be happy that I don't see "Yesterdays Episode" and then I later break things to that the text "Yesterday's Episode" is generated but the test still passes.

I can definitely see some use for this in keeping auditors happy.

This is how I gather my twitterings into the blog. I just scrape the rss feed. The twitter api may allow for something more sophisticated but this Just Works.

It's just too easy in ruby/rails:

   1  class TwitterHandler
   2    def update( strFile)
   3      require 'open-uri'
   4      require 'simple-rss'
   5  
   6      oRss = SimpleRSS.parse( open( strFile))
   7  
   8      oRss.items.each do |oItem|
   9        strBody = oItem.description
  10        strBody.gsub!( /^petersblog: /, '')
  11        strTitle = "Twittering: #{oItem.pubDate.strftime( "%d %B %Y %H:%M")}"
  12  
  13        if not Post.find_by_title( strTitle)
  14          oPost = Post.new( :title => strTitle, :body => strBody, :created => oItem.pubDate)
  15          oPost.Taglist = "twittering"
  16        end
  17      end
  18    end
  19  end
  20  
  21  The above is only wrapped in a class because that is how the first rspec example I read was laid out.
  22  
  23  Then there is a rake task to run the above:
  24  
  25  <ruby>
  26  task (:twitter => :environment) do
  27    require 'lib/TwitterHandler'
  28  
  29    oTwitter = TwitterHandler.new
  30    oTwitter.update( "http://twitter.com/statuses/user_timeline/16738335.rss")
  31  end
Toggle Line Numbers

Then a cron job:

13,28,43,58 * * * *  bash -c "cd /var/www/PetersBlogger/current; rake -s RAILS_ENV=production twitter"

This polls every 15 minutes. The '-s' stops rake outputting a needless 'In directory blah' message which cron dutifully emails to me.

I'm enjoying using twitter, I like the medium, short and pithy. Not really suitable for code snippets though.

I was working on this site and noticed how slow it was, particularly on rss feeds. This was just after managing to completely screw up my ruby gem installation (apt-get remove rubygems/apt-get install rubygems didn't reinstall rubygems) which I only cured by upgrading the server to ubuntu hardy. I wasn't sure whether I had blown something in the upgrade (the server did a fsck when I rebooted it but maybe only because it had been 'up' for 100 days) or whether the code was just slow.

How to pinpoint the cause of the slowness? Profiling. Some research revealed I can profile ruby code as follows:

• install a ruby profiler:

  gen install ruby-prof
  

• insert some profiling code around your suspect slow code (in my case around the rss formatting):

     1  gem install ruby-prof
     2  
     3     require 'ruby-prof'
     4  
     5     RubyProf.start
     6  
     7     {code to profile}
     8  
     9   result = RubyProf.stop
    10     printer = RubyProf::FlatPrinter.new(result)
    11     printer.print(File.open( '/tmp/prof.txt', "w"), 0)
  Toggle Line Numbers

• run script/server and try downloading rss
• wait because it is very slow
• examine the profile

  vim /tmp/prof.txt
  

• lament

This told me that 70% of my time was being spent in a gsub in my wilki code that I added last week. gsub was being called 1800 times (!) and when I commented out the new gsub call the rss was rendered much faster.

The thing is, my wilki formatting is supposed to be cached, it should only be done when a posting is modified, not every time the post is rendered. Turns out to be because updating a field in an ActiveRecord doesn't automatically cause it to be saved because it is dirty, an explicit save helped. Problem solved, now downloading the rss feed doesn't max out my cpu for 20 seconds

NB (clarification): the rss doesn't go through the rails fragment cache.

My websites are all migrating to a CMS kinda vibe where pages are edited through the web browser. Easy for me, easy for anyone else. I've already got the technology to upload pictures but it has an annoying problem and it seems to be built into the attachment_fu plugin. The problem is that one must choose how big one wants the pictures to be when they are first uploaded and the size is defined in the database model. attachment_fu will then resize the image accordingly. That is all very well if one is organised and prepared and knows in advance how big they want their pictures or are running a simple gallery where all pictures are the same size.

But what about where one is trying to do some nice layouts on a page and wants to adjust the sizes of the pictures? Say I decide I want a particular picture on a page to be 100 pixels wide? I have two options:

• I can go into gimp/paintshop pro/whatever, resize the image and upload it (again) but that is annoying.
• I can alter the img tags in my html to include a width attribute. I see a lot of this on other websites, as the page downloads you see lots of little images slowly filled in as huge 1000x1000 jpg's are downloaded and resized in the browser to 100x100 because the page designer couldn't be bothered to resize the image in gimp/paintshop pro/whatever.

Now there is another way. Here is a rails helper that allows the width of an image to be specified in a view, automatically resizing the image as appropriate:

   1  # Methods added to this helper will be available to all templates in the application.
   2  module ApplicationHelper
   3  
   4     #
   5     # Takes the url of an image and a desired width and ensures a cached resized
   6     # version of the image is available and returns the url of said image.
   7     #
   8     # the url should not be a relative url as it is assumed work relative to RAILS_ROOT/public
   9     # This also means the image must be on this server.
  10     #
  11     def cacheimage( strUrl, nWidth)
  12       # Find original file in file system
  13       strFile = "#{RAILS_ROOT}/public" + strUrl
  14  
  15       # create name for cached and resized image
  16       #
  17       # /blah/blah.jpg --> /blah/blah-123.jpg
  18       #
  19       strBits = strUrl.split( '.')
  20       strBits[-2] += "-#{nWidth}"
  21       strCacheName = strBits.join( '.')
  22  
  23       strCacheUrl  = "/imagecache#{strCacheName}"
  24       strCacheFileName = "#{RAILS_ROOT}/public#{strCacheUrl}"
  25  
  26       if not File.exists?( strCacheFileName)
  27         require 'image_science'
  28  
  29         FileUtils.mkpath( File.dirname( strCacheFileName))
  30  
  31         ImageScience.with_image( strFile) do |oImg|
  32           oImg.resize( nWidth, nWidth * oImg.height / oImg.width) do |oResized|
  33             oResized.save strCacheFileName
  34           end
  35         end
  36       end
  37  
  38       return strCacheUrl
  39     end
  40  end
Toggle Line Numbers

That wasn't too difficult. You need to create an 'imagecache' directory under /public to store a cache of resized images and make sure mongrel or whatever can write to it.

In a view one would put:

<%= image_tag cacheimage( "/images/prettypicture.jpg", 500) %>

This will cause a version of prettypicture.jpg to be created that is resized to 500 pixels wide with a proportional change in height (in a typical html page layout the width of things is often more important than the height except maybe for headers and footers). The resized version is stored in a cache and will be displayed from then on.

If the original version of prettypicture.jpg is changed then one must remember to erase the resized versions from the cache as this isn't automatic.

Another helper is:

   def applyimagewidth( strText)
      strText.gsub!( /([^"!<>{}]*)(jpg|JPG|gif|GIF|png|PNG)x(\d+)/) do |oMatch|
         cacheimage( "#{$1}#{$2}", $3.to_i)
      end

      return strText
   end

(UPDATE: warning- the gsub above is VERY SLOW, it can take minutes to scan a few k of text, even if there are no matches).

which allows the width to be appended to the image file name. This allows a view such as

<%= textilize applyimagewidth( @page.body) %>

and thus one's textile source could be:

!/imagecache/images/DSC_4064-300.JPG!

to show an image with a width of 300 pixels. I could hack the redcloth source directly to add this feature but I'd rather keep out of there.

I've done this in a simplistic hacky way but it should suffice (like all my stuff).

Another fusion of WoW and rails.

While planning Maexyn's conversion to alchemy I thought it would be nice to be able to get a listing of my characters bag and bank items so I could look through my stocks and see what was what. It occurred to me that a nice way to do this would be to scan through the lua files that are dumped by the TBags mod since that holds a database of contents of all my characters bags and bank. Doing this in ruby turned out to be very easy:

• Install ruby lua support:

  sudo aptitude install liblua5.1-0-dev
  sudo gem install rua
  

• Ruby code to convert lua data file into ruby data:

  oRua = Rua.new(:all)
  oRua.eval( open( File.dirname( __FILE__) + "/../../db/TBag.lua").read)
  

• oRua now has an array of items that maps onto the data stored in the lua file.

  oRua['TBnkItm'] has a list of items in the bank
  oRua['TInvItm'] has a list of items in the inventory
  

The end result is this page where I can see what stocks I have. Auctioneer also stores its data in lua so in a similar way I added a lookup so I see the values to these stocks. I can also use the item numbers to add links to the items to thottbot for reference. It isn't perfect, it should show vendor values, especially for soulbound items.

Anyway with this I can plan some sales, particularly of Maexyn's enchanting materials.

Currently the auctioneer data is way out of date, I need to get it running again and do a few scans.

This isn't here as a way of boasting about my vast wealth, I guess a week's karak farming would be better that this junk pile (minions scourgestones?!?).

N.B: an alternative implementation would be simply to write something in the lua language that reads the two lua databases and then dumps out the html. For me it is easier to do it in ruby than learn lua.

I've been trying out rspec, a new testing framework for ruby that allows one to better follow the principles of Behaviour Driven Design (BDD). BDD is a step on from Test Driven Design (TDD) approach exemplified in the standard ruby unit test framework. TDD is the process of starting ones development by writing the tests that your new module should pass. As you write each test you then implement the code to pass it and hence, voila, you have nicely designed and robust software. In BDD you approach testing the code from how it is supposed to behave. When liasing with clients this is a better approach as everyone can talk in the same language, essentially in terms of requirements so one can try to get less bogged down in implementation details.

Here are some rspec tests for a class I have been working on:

   1  describe ReportHelper do
   2    before(:each) do
   3      #
   4      # Create a new clean Report class for each individual test
   5      #
   6      @oReport = ReportHelper::Report.new
   7    end
   8  
   9    it "should allow a column to be added" do
  10      @oReport.AddColumn( 'First Column')
  11    end
  12  
  13    it "should allow multiple columns to be added" do
  14      @oReport.AddColumn( 'First Column')
  15      @oReport.AddColumn( 'Second Column')
  16    end
  17  
  18    it "should allow rows to be added to the report" do
  19      @oReport.AddColumn( 'First Column')
  20      @oReport.AddColumn( 'Second Column')
  21      @oReport.AddRow do
  22        # pass
  23      end
  24    end
  25  
  26    it "should only allow rows to be added if columns exist" do
  27      lambda{ @oReport.AddRow}.should raise_error(RuntimeError, 'No columns defined')
  28    end
  29  
  30    it "should return simple cell values" do
  31      @oReport.AddColumn( 'First Column')
  32      @oReport.AddColumn( 'Second Column')
  33  
  34      @oReport.AddRow do |oRow|
  35        oRow[ 'First Column'] = 98
  36        oRow[ 'Second Column'] = 198
  37      end
  38  
  39      @oReport[0]['First Column'].should == 98
  40      @oReport[0]['Second Column'].should == 198
  41    end
  42  
  43  end
Toggle Line Numbers

Each test has a descriptive string that can be interpreted in two ways: as a test that must be passed or as a requirement for the software. The rspec toolset includes something that will pull out all these strings and give a summary of what testing has been done/what the requirements for the software are. If an auditor asks how the software is tested, bang, there is a full report of the tests. At a practical level (i.e. makes life easier for me) I prefer this to the ruby unit test system where one has to think up meaningful_method_names for each test. The strings are easier to write and read and reduce the need for comments.

Example report:

ReportHelper

- should allow a column to be added
- should allow multiple columns to be added
- should allow rows to be added to the report
- should only allow rows to be added if columns exist
- should return simple cell values

Rspec contains sweeter syntactic sugar than the ruby unit test module. For example, rspec overloads every object in the system with a 'should' method for doing the equivalent of assert tests:

    @oReport[0]['First Column'].should == 98

Compare this to test/unit:

    assert_equal 98, @oReport[0]['First Column']

apart from more typing, assert_equal has the expectation/actual things in what is intuitively to me the wrong order and I'm always having to correct these lines because the error messages have the terms swapped.

Rspec can test assertions more cleanly:

    lambda{ @oReport.AddRow}.should raise_error(RuntimeError, 'No columns defined')

although it's a pity that the lambda is necessary (and probably no 1 noob trap) but with test/unit the following always seemed like a lot of typing:

    oError = assert_raises RuntimeError do
      @oReport.AddRow
    end
    assert_equal 'No columns defined', oError.message

rspec is nicely integrated into rails, plugins being available to generate rspec tested controllers and models and suchlike, rake tasks to generate documents (woohoo) etc. It is integrated into netbeans and runs ok on Windows although when the tests are run (CTRL-F6) there is a tedious few seconds wasted in startup time. It is possible to run an rspec server in the background to eliminate this startup but it doesn't work on windows native ruby, it gives an error lamenting Microsofts glorious omission of a fork api.

There is much more to Rspec than this. One aim seems to be to write psuedo testing languages and have clients use these to write stories about what the software will do, these stories turning into a test framework. As the slashdot tag goes, 'goodluckwiththat'. At a practical level, I'm liking rspec even if only as a unit test framework with added syntactic sugar.

Up till now to put pictures in this blog I have had to:

• go into an image editor to resize the file to suit the blog (about 450-500 pixels wide).
• fire up an sftp program to upload the file to /var/www/PetersBlogger/public/images
• add a tag such as

  [/images/TheNewPicture.jpg]
  

  to the wilki.

It was a bit laborious, especially the SFTP bit which is why I didn't do it very often.

I've now added better support for pictures to PetersBlogger. Here is an example:

Girls Aloud

if you click on this you should be treated to a larger version of the image, thanks to lightbox a javascript library.

PetersBlogger now supports a database of pictures so I can add a new picture, give it a name and upload it through the administration pages. The code uses the name as a caption under the photo and as alt text. I can use the name directly in my wilki text to choose which pictures to display.

The database of pictures is handled by the attachmentfu rails plugin. This article gives the steps required to get attachmentfu up and running.

The setup automatically handles resizing images to three sizes: thumbnail for listing in my picture admin page, a version that is no more than 500 pixels in either dimension for showing in the blog and a version that is no more than 800x600 for showing zoomed when you click on a picture. Attachmentfu has the sense to keep the aspect ratio correct automatically so I don't have to worry about portrait/landscape. I had to hack the imagescience library that does the resizing as it was hard-wired to save jpegs using JPEG_QUALITYSUPERB mode so the blog-sized images were taking up 150k. I changed this to JPEG_QUALITYAVERAGE and the size dropped to 15k or so, but it was a bit noisy so I've settled on JPEG_QUALITYGOOD and 38k images. You are on broadband?

I was slightly worried about how to change pictures, i.e. to upload a different version of a picture to replace an existing one, since the attachmentfu article I mentioned does not cover this. The answer is to use the exact same view code in the 'edit' page as in the 'new' page. Combined with rails scaffold code it Just Works! It even handles deleting the three old pictures and creating the new files.

My main motivation for doing this was so I can do it on the websites I host for other people and make it easier for them to upload pictures. PetersCMS is still in the planning stages but one problem has been cracked.

Much as I hate flash, it is a nice way to embed a sound or video player in a page. Here's how I did it in rails using the JW FLV Media Player. I added the following to my view:

   1  <script type="text/javascript" src="/javascripts/swfobject.js"></script>
   2  
   3  <% for oRssItem in @rssitems %>
   4    <hr/>
   5    <h3><%= oRssItem.title %></h3>
   6    <p><%= oRssItem.description %></p>
   7    <table width="470">
   8      <tr>
   9         <td wIdth="50%"><%= oRssItem.created.strftime( "%m %B %y") %></td>
  10         <td align="right"><%= link_to "Download", "/#{ oRssItem.filename}" %></td>
  11      </tr>
  12    </table>
  13  
  14    <div id="player<%=oRssItem.id%>">This text will be replaced</div>
  15  
  16    <script type="text/javascript">
  17      var so = new SWFObject('/mediaplayer.swf','mpl','470','20','8');
  18      so.addParam('allowscriptaccess','always');
  19      so.addParam('allowfullscreen','true');
  20      so.addVariable('width','470');
  21      so.addVariable('height','20');
  22      so.addVariable('file','/<%= oRssItem.filename%>');
  23      so.write('player<%=oRssItem.id%>');
  24    </script>
  25  <% end %>
Toggle Line Numbers

This generates a page full of podcast entries, each with a little media player thing so you can listen to them easily. This does mp3 files but it can handle video too.

I am aware that the swfobject.js would be better loaded in the html header.

Here's a demo: I don't know if this will work in the rss feed (UPDATE: no!) so bear with me and please be gentle on my bandwidth, it's quite a catchy tune:

The main reason I am blogging this is because it only needs a change to a view file and installation of two files in the /public directory (the player and a javascript file). Since view files are awkward to comment legibly I decided to document it all here so I can find it again.

In a mammoth programming session I implemented proper smileys

[[':-)', 'smile'],
  [':-(', 'sad'],
  ['8-)', 'cool'],
  [';-)', 'wink']].each do |strCode, strFile|
  strText.gsub!( strCode, "<img alt=\"#{strFile}\" src=\"/images/smileys/#{strFile}.png\" />")
end

My fingers were bleeding from all the typing Worked first time though

There is even alt text for blind people

I figured out how to scrape the Wow Armory for up-to-date information about my characters using ruby. This means I can do this directly in PetersBlogger without using third party signature generators that disappear overnight or get absorbed into wow gold sites.

   1    require 'net/http'
   2  
   3    #
   4    # Scrape wow character info.
   5    #
   6    def WoWInfo
   7      begin
   8        strRealm = 'Eonar'
   9  
  10        oInfo = []
  11  
  12        ['Pookypoo', 'Maevyn', 'Maezyn', 'Maexyn'].each do |strCharacter|
  13          oCharInfo = []
  14  
  15          #
  16          # Open url.
  17          # Need to specify firefox as user agent as this makes the server return an XML
  18          # file.
  19          # Look the data up in the european armory. Change this for US.
  20          # If this is not done we get html.
  21          oResp = Net::HTTP.start( "armory.wow-europe.com", 80) do |http|
  22                http.get( "/character-sheet.xml?r=#{strRealm.tr( ' ', '+')}&n=#{strCharacter}",
  23                            { 'user-agent' => 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4'})
  24          end
  25  
  26          oDoc = REXML::Document.new oResp.body
  27          oDoc.elements[1].elements.each( 'characterInfo/character') do |oElement|
  28            oCharInfo << {:race => oElement.attributes['race']}
  29            oCharInfo << {:class => oElement.attributes['class']}
  30            oCharInfo << {:level => oElement.attributes['level']}
  31          end
  32  
  33          oDoc.elements[1].elements.each( 'characterInfo/characterTab/talentSpec') do |oElement|
  34            oCharInfo << {:spec => "#{oElement.attributes['treeOne']}/#{oElement.attributes['treeTwo']}/#{oElement.attributes['treeThree']}"}
  35          end
  36  
  37          oDoc.elements[1].elements.each( 'characterInfo/characterTab/professions/skill') do |oElement|
  38            oCharInfo << {oElement.attributes['key'].to_sym => oElement.attributes['value']}
  39          end
  40  
  41          oInfo << {strCharacter => oCharInfo}
  42        end
  43  
  44        return oInfo
  45      rescue
  46        return []
  47      end
  48    end
Toggle Line Numbers

This lot generates the info in this page which is linked to under 'My Characters' over to the right. It should be fairly up-to-date as the cache will flush once a day causing the armory to be requeried. Currently the output is presented in a rails view thus:

   1  <% cache( :part => :wow) do %>
   2    <table>
   3    <% WoWInfo().each do |oCharInfo| %>
   4      <% oCharInfo.each_pair do |strChar, oInfo| %>
   5        <tr><td colspan="2"><b><%= strChar %></b></td></tr>
   6        <% oInfo.each do |oItem| %>
   7          <% oItem.each_pair do |strItem, strValue| %>
   8            <tr>
   9              <td><%= strItem %>:</td><td><%= strValue %></td>
  10            </tr>
  11          <% end %>
  12        <% end %>
  13      <% end %>
  14    <% end %>
  15    </table>
  16  <% end %>
Toggle Line Numbers

All the work is done by the WoWInfo method which I put in the application helper. I didn't use a controller method for this.

I'm going to think about a nicer presentation with some pictures of the actual guys rather than generic pictures of bald redhead dwarves.

UPDATE: I've added my old Aerie Peak toons to the output but the code above still stands. Lugulas was my old auction alt who I've never named here before, hence level 10 with level 75 (dis)enchanting. I find myself missing them...

One of the things that is offputting about learning any of the web application stacks that I have tried (rails, turbogears, django) is the command line operations that need to be performed. When you start using them they seem like magic but after a while they become old friends. Hence this article is a summary of the rails command line operations that I use routinely.

All these commands except for the first one should be executed in the root directory of your application. Note that for the commands that start 'script' you are running ruby scripts and under windows you will need to type 'ruby script/blah'.

rails {app name}

The basic one, creates a new rails application. It will create a subdirectory with the given name in the current directory. This subdirectory is the root directory of your application and will be filled with all the appropriate files and subdirectories to start developing. It is a big code generator.

script/server

This could be considered the next step after creating your application. You cd to the application's root directory and run this command. It will start a development web server and you will be able to connect to localhost:3000 in a web browser and admire your new creation. This is the way to debug your application. Any print statements will be dumped out here.

script/generate model {model name}

This command will generate a new model in your application. A model is roughly equivalent to a table in the database.

script/generate controller {controller name}

This command will generate a new controller in your application in the file 'app/controllers/{controller name}_controller.rb'. A controller is essentially the logic that takes a web page request and decides what to do with it. It will also create a directory called 'app/views/{controller name}' to store the views for this controller, views being the templates that determine how the web pages will look.

rake db:migrate

This runs the migrations that will either define the database schema, modify it or undo those modifications. It defaults to updating the development database but can take the argument 'RAILS_ENV=production' to update the production database.

script/console

script/console is useful for debugging. It gives you a ruby irb command line prompt where you can play with the inner workings of your code, primarily you have direct access to your models and hence the database. It can be an easier way of manipulating the database from a command line than typing SQL into the mysql or sqlite command line clients.

rake --tasks

Lists what rake tasks are available.

rake db:test:prepare

Prepares the test database for running unit tests.

rake test

Runs all unit, functional and integration tests on your model.

I think that covers it. There are many more options than these are the ones that I have found to be most useful.

Rake is the ruby equivalent of make except it is much better in that you are scripting in ruby and not the nasty primeval make language. Rails uses rake a lot as a simple place to gather together a load of administration tasks such as running tests, running database migrations etc.

When I moved all my drupal blog posts from drupal over to PetersBlogger I decided to use a nice organised rake script rather than do my usual practise of having a directory full of utility scripts.

In my rails installation I created a file called lib/tasks/jobs.rake and in it I put my import code:

   1  task (:import_drupal => :environment) do
   2    print "Importing Drupal\n"
   3    oTagTable = {}
   4    #
   5    # Import all the articles.
   6    #
   7    Drupal::Article.find(:all, :conditions => "type <> 'image'").each do |oArticle|
   8      print "#{oArticle.title}\n"
   9      oTime = Time.at( oArticle.created)
  10      oPost = Post.create( :nid => oArticle.nid,
  11                          :title => oArticle.title,
  12        :body => oArticle.body,
  13        :created => oTime)
  14  
  15      #
  16      # import the tags related to the article.
  17      #
  18      oArticle.tags.each do |oTag|
  19        if oTagTable[oTag.tag]
  20          oNewTag = oTagTable[oTag.tag]
  21        else
  22          oNewTag = Tag.new( :tag=>oTag.tag)
  23          oTagTable[oTag.tag] = oNewTag
  24        end
  25  
  26        oPost.tags << oNewTag
  27      end
  28  
  29      #
  30      # Import the comments for the article.
  31      #
  32      oArticle.comments.each do |oComment|
  33        strWho = oComment.name
  34        strComment = oComment.comment
  35        if strWho == nil or strWho == ""
  36          strWho = "Peter"
  37        end
  38        if strComment == nil or strComment.strip == ""
  39          strComment = "{nothing to say}"
  40        end
  41        oPost.comments << Comment.create( :who => strWho,
  42            :body => strComment,
  43            :created => Time.at( oComment.timestamp), :published => 1)
  44        if not oPost.comments[-1].valid?
  45          p oPost.comments[-1]
  46          p oPost.comments[-1].errors
  47        end
  48      end
  49  
  50      oPost.save!
  51    end
  52  end
Toggle Line Numbers

This creates a rake task called 'import_drupal' which I can invoke simply by running

rake import_drupal

The task runs with the appropriate rails environment so all my models are in place. I had to create a model to access the records in the drupal database thus:

   1  module Drupal
   2    class Comment < ActiveRecord::Base
   3      establish_connection configurations['drupal']
   4      set_table_name 'comments'
   5    end
   6    class Tag < ActiveRecord::Base
   7      establish_connection configurations['drupal']
   8      set_primary_key 'tid'
   9      has_and_belongs_to_many :articles,
  10        :class_name => 'Drupal::Article',
  11        :join_table => 'awtags_node',
  12        :foreign_key => 'tid',
  13        :association_foreign_key => 'nid'
  14      set_table_name 'awtags'
  15    end
  16    class Article < ActiveRecord::Base
  17      establish_connection configurations['drupal']
  18      set_primary_key 'nid'
  19      self.inheritance_column = 'poopy'
  20      has_many :comments, :table_name => 'comments', :foreign_key => 'nid', :class_name => 'Drupal::Comment'
  21      has_and_belongs_to_many :tags,
  22        :class_name => 'Drupal::Tag',
  23        :join_table => 'awtags_node',
  24        :foreign_key => 'nid',
  25        :association_foreign_key => 'tid'
  26      set_table_name 'node'
  27    end
  28  end
Toggle Line Numbers

This model handles all the relationships between the records, including the has_and_belongs_to_many relationship between posts and tags (provided by awtags).

I needed an entry in config/database.yml to define the connection to the drupal database:

drupal:
  adapter: mysql
  database: petersblog
  username: secret
  password: doyouthinkidpostithere

and it all works nicely. If I invoke it thusly:

rake import_drupal RAILS_ENV=production

then it will import the posts into the rails production database.

I'm giving away a few of my schema secrets here which pedantic rails developers might leap on, specifically I have used my own fields called 'nid' and 'created' to perform roles that rails already provides 'id' and 'created_on' for. The reason for the duplication was that during development I was having problems setting these fields to the values from the drupal database, rails was insisting on giving them it's own values. By having my own fields I had total control. There are no doubt ways around this but I didn't have time to find them (although I did find other people moaning about the same problem).

And that's how I managed to save 1500 posts from the hell that is php.

If you are really bored, you might like to check out my new Archive Page.

I thought about putting it in the sidebar but I didn't for a few reasons:

• it's ugly
• it takes time to generate the page and I didn't want the rendering of every url that wasn't already cached to require this hit.
• who cares anyway? It's mainly there to open up the older posts to search bots without them having to go through page/next 100 times.

On a more technical level, it uses the following route to give 'nice urls' for the year/month combinations:

map.connect 'post/on/:year/:month', :controller => 'post', :action => 'listbyyearmonth'
map.connect 'post/on/:year', :controller => 'post', :action => 'listbyyearmonth'

so March 2005 would be '/post/on/2005/3' which is cool.

Since I added the page count thing yesterday (that only I can see since I don't want everyone to know how often I tweek the site and reboot) I got this:

I think I've managed to map all the urls from my old drupal blog to this new blog: according to statcounter (which is a bit slow loading today) I had 1,103 page loads yesterday compared to 1,167 last week on the old blog. Any missed urls would lead to the 404 page which doesn't have the hit counter or statcounter on it (hum, could catch the 404's and redirect them to main page with a flash notice..).

Statcounter works under javascript and so does not count bots and paranoid people with javascript disabled. It is interesting therefore that the latter outnumber the javascript users by 2.5:1. And none of this includes rss/atom feeds.

For the record, here are the routes I used to map my drupal urls to PetersBlogger urls. This is from config/routes.rb:

   1  map.root :controller => 'post', :action => 'list'
   2  
   3  #
   4  # map the various rss feeds
   5  #
   6  map.connect 'tag/:tagname/feed', :controller => 'tag', :action => 'rss'
   7  map.connect 'tags/:tagname/feed', :controller => 'tag', :action => 'rss'
   8  map.connect 'tag/feed/:tagname', :controller => 'tag', :action => 'rss'
   9  map.connect 'blog/feed/1', :controller => 'post', :action => 'rss'
  10  map.connect 'blog/feed', :controller => 'post', :action => 'rss'
  11  map.connect 'blog/1/feed', :controller => 'post', :action => 'rss'
  12  map.connect 'rss.xml', :controller => 'post', :action => 'rss'
  13  
  14  #
  15  # map the atom feeds
  16  #
  17  map.connect 'tags/:tagname/atom.xml', :controller => 'tag', :action => 'atom'
  18  map.connect 'node/:id/atom.xml', :controller => 'post', :action => 'atom'
  19  map.connect 'atom/feed', :controller => 'post', :action => 'atom'
  20  
  21  #
  22  # Map pages
  23  #
  24  map.connect 'node/view/:id', :controller => 'post', :action => 'show'
  25  map.connect 'node/:id', :controller => 'post', :action => 'show'
  26  # not sure what below was for
  27  map.connect 'node/:id/tag/function.require', :controller => 'post', :action => 'show'
  28  map.connect 'node/:id/function.require', :controller => 'post', :action => 'show'
  29  map.connect 'tag/:tagname', :controller => 'post', :action => 'listbytag'
  30  map.connect 'tags/:tagname', :controller => 'post', :action => 'listbytag'
  31  map.connect 'tags/:tagnamex/tags/:tagname', :controller => 'post', :action => 'listbytag'
  32  map.connect 'blog/tags/:tagname', :controller => 'post', :action => 'listbytag'
  33  map.connect 'tag', :controller => 'tag', :action => 'list'
  34  
  35  # Install the default routes as the lowest priority.
  36  map.connect ':controller/:action/:id'
  37  map.connect ':controller/:action/:id.:format'
Toggle Line Numbers

I fixed most of the above by searching the access logs for 404's (page not found) and looking for patterns. I'm not sure why drupal felt the need to support so many variants on urls for the same page/feed or how people managed to find them. Note the silly 'tags/:tagnamex/tags/:tagname': I had old urls of the form '/tag/213/tag/34' because of sloppy relative linking.

Rails routing is sublimely simple and flexible. It would have taken me days to get all the above working with mod_rewrite rules.

I want to use rails as the front end for an application that I am developing. The application requires that I maintain a persistant object, i.e. one that lives on between multiple page requests. Given that my application would live in a single mongrel instance, is there an easy way to do this?

Well, I studied the code in the mongrel/rails stack (open source ftw) and it turns out that it is easy. Once the rails framework is loaded by mongrel, the dispatcher will handle any request that comes in and once the request is completed it will reload rails own reloadable classes (controllers, models etc) so that they are 'clean' and ready for the next request.

It is very easy to have an object that survives between individual page requests.

Here is an object:

   1  class SillyCounter
   2    def initialize
   3      @count = 0
   4    end
   5  
   6    def Inc
   7      return @count += 1
   8    end
   9  end
  10  
  11  $oSillyCounter = SillyCounter.new
Toggle Line Numbers

I put this in lib/SillyCounter.rb since I like CamelCase.

I load it in config/environment.rb, right at the bottom:

require 'lib/SillyCounter'

and in a view I added:

<p><%= $oSillyCounter.Inc %> pages served since last reboot</p>

Relaunch mongrel and it works!

CAVEAT: I'll emphasise that this will only work for a single process server! But my application is single user and will never need to serve 1000 requests/second! I would use dRb (distributed ruby) if I needed something scalable.

Todo: find a rails problem that takes more than 20 lines to get around.

PHP programmers out there may be interested to know that this is the first time I have used a $ in a variable name in ruby.

I wanted PetersBlogger to have a preview facility when composing posts that would automatically run the text I was editing through my wilki filter as I typed and give me a live preview of how the post would turn out when it was formatted and displayed. No pressing a preview button for me, I wanted it to just happen. Sounds like a job for AJAX. But how to do this in rails? Well it turns out to be very easy.

I put this in my form where I edit a post:

<p><%= f.text_area 'body', :size => "100x40", :html => { :id => :body} %></p>
<%= observe_field :post_body, :url => { :action => :wilkify },
    :frequency => 3.00,
    :update => :preview,
    :with => "'text='+escape( $('post_body').value)"
%>
<div id="preview"/>

This puts some javascript in the page that monitors the text_area where I am editing and, if it changes it will invoke the 'wilkify' action in my controller, passing it what I have typed thus far.

The 'wilkify' action is very simple, it just gets the text, wilkifies it and makes sure it is rendered as a simple piece of html, without the main site layout:

def wilkify
  strBody = params[:text]
  strBody = CGI::unescape( strBody)
  @text = Wilki( strBody)
  render :layout => false
end

I have a small view defined to wrap the preview text in a couple of div's, just as if they were appearing on the site:

<div class="node">
  <div class="bodynode">
    <div class="entry">
      <%= @text %>
    </div>
  </div>
</div>

The wikified text is then inserted into the edit form in the div with the id 'preview'.

I have to add the prototype javascript library to the head section of the main application layout:

<%= javascript_include_tag 'prototype' %>

It is not entirely perfect and I would never let strangers use it because they could type malformed html and totally mess things up but it is good enough for me.

I've missed blogging while I've been working on PetersBlogger. My old drupal blog was limping along: while administering the site it kept logging me out. I can admit now that I was running an old version of drupal (4.7?) which I couldn't upgrade due to module dependencies (mainly awtags). The old code wasn't running that smoothly under php5 despite initial appearancies.

I'm happy with my new blog platform, it's running really nicely on my slicehost slice and I've learnt an awful lot about rails while developing it. I will try to do a braindump of rails experiencies in forthcoming posts (cross fingers).

Drupal is a fine CMS and I'm using Drupal 5 on my company intranet. My thoughts on a drupal vs ruby on rails debate would be:

• drupal/php/apache is easier to deploy than a rails/ruby/mongrel/nginx deploy: I moved my drupal site between about four different hosts and did it each time in less than an hour. A rails setup is more convoluted (especially mongrel_cluster). However, this site is running much faster under rails than it was under drupal. The difference is probably down to php and apache being more mainstream and slightly more refined and hence easy to set up. I'm not saying rails is a total pain, just that if I only had ten minutes in which to deploy a site I would be reaching for drupal. If I only wanted to install the source and a few modules and never touch any coding then I would be happy with drupal.
• I love ruby and rails development. There is no comparison, php seems to me as much a bastard language as visual basic 6. Ruby was cleanly developed as an object orientated language, php is having object orientation grafted on as an afterthough.
• If I look through my drupal source I see no unit testing. Ruby/Rails has unit testing built in and I'm completely sold on it. I don't think I will ever trust any code (especially code I write) if it hasn't been unit tested. Some interesting aspects of unit testing:
  • I understand now the 'test driven' approach to development where you design a module's api by first roughing out how the tests will work: the unit test is your first experience of using the new api. It's during testing that you learn how nice an api will be.
  • If the documentation of a new ruby/rails tool is dubious, look in the unit tests to see how the author intended it to be used. If there are no tests then run.
  • If I run into a tricky bug, it's better to first reproduce it in unit tests, fix it there and then be happy that the bug will never recur. It's easier to debug code in a unit test (essentially a command line application) than on a live website.
• all ruby on rails applications follow basically the same architecture. It is fairly easy to figure out how a new application works. Every php application is different. I know I'm comparing a web framework to a programming language but none of the php applications (drupal, phpmyadmin) or am familiar with (wordpress) share a common architecture. New php application to maintain? New learning curve.
• my new site is being hammered with attempts to post to /comments/reply, even though it doesn't use that url for comment submission. The comment spammers know drupal and know where to test the locks. If any of them ever bother to try to find my comment submission url they will only find the same captcha that protected my drupal site (which was much easier to implement in rails, simply as a validation on the model).

This blog is now running on my new rails based blogging platform which I have called PetersBlogger (I'm not good at names).

It's mostly complete. Features:

• post articles (or you wouldn't be reading this)
• wilki formatting including syntax highlighting with ultraviolet e.g.

  strThis = "that"
  

• comments, including
  • moderation list
  • Peter's simplistic captcha system
• tagging
• pingomatic submission
• Technorati tags
• rss feeds (no atom yet)
• uses fragment caching for performance

Todo list:

• AJAX formatting preview
• atom feeds (do I care?)
• search (or rely on google)
• tag cloud
• archives by month/year: you DO care what I wrote in Feb 2005 right?
• gimmicks

Please let me know if you spot any sillies (that's if comment submission is working at all).

It's nice to use because the administration interface isn't split out into a seperate set of web pages, when I 'log in' I simply get extra buttons to press in the pages that you are seeing.

For the record I am still alive.

I'm busy working on a new blogging platform and I'm not quite ready to make it live yet. I decided to write it in Ruby on Rails directly rather than use an existing blogging application. I wanted something that ran under ruby on rails for the hackability and there were four candidates:

typo: Most popular rails blogging app but nobody seems to rate it as technically excellent. I tried the demo but was put off by the ajax: lots of complexity to go wrong there. mephisto: This seems to be highly regarded on a technical level but a few things put me off:

• it hasn't had an official release since 2006, the mephisto community seems to work in terms of svn revision numbers.
• it needs a pagination plugin to give basic functionality required by any blog with more than ten posts. I'd rather keep plugin requirements to a minimum (fewer things to break when upgrades come along).
• the database schema seems to be growing in complexity: 17 tables? The number of fields in the table that holds posts is also proliferating: 27 fields?
• the layout is configured by editing liquid templates. Apart from making it possible to edit layouts through the admin panel and to support themes, to me this just adds another layer of complexity and more to learn. I am happy with Rails built-in templating system. It took me no time to port my drupal theme to rails.

simplelog: a simple blogging platform but it didn't grab me. The link on the site to a wiki gives an access error.

radiant: Could be used for blogging but Radiant is primarily a cms. The author of Radiant uses Mephisto...

Development of my application has been quite rapid and it's very nearly ready for launch. However I have a problem with the Ruby on Rails testing framework: it's addictive and I can't bare the thought of putting up a live website without making sure it is thoroughly tested first. I'm in that testing/polishing phase that is hard to break out of if you have any kind of obsession with detail.

One thing is for sure: Rails is a joy to work with.

In the meantime, this old blog platform has lost it's lustre and I'm not so included to use it.

UPDATE: I forgot to mention hobix. I can't get past the website...

I wanted to code some new features for one of my Radiant sites but I didn't really want to do it in terms creating new radius tags. What I am doing involves creating html tables from a database and doing this with radius tags has a few problems:

• it is beyond my understanding of what radius tags are capable of. For example, how can I pass a radiant tag the id of a record to display, if the record does not correspond to a Radiant page? I don't want to create Radiant pages for every record. The documentation for the tagging engine isn't strong and I don't have the time available to figure out how to use it.
• what if I want to port my application to some other platform? It would still run on rails but would not necessarily use the radius tagging engine: frankly I am happy using erb/rhtml for my templating (the 'end users' won't need to fiddle with the presentation of this stuff).

So the question came to be: how can one write regular rails code with Radiant? How to bolt some regular rails Model/View/Controller code inside Radiant. How to use my existing Rails knowledge? I had to study the mailing list for a while but the answer was very straightforward:

1. Install the share_layouts extension in your Radiant site.
2. Create a new Radiant Extension of your own using the neat tutorial.
3. Create a regular controller in your extension using the syntax from the extension tutorial, i.e.:

   script/generate extension_controller MyRailExtension myrails
   

4. Add the following code to your new controller so that it uses a radiant layout to render the page

   radiant_layout 'Layout name'
   

5. That's it!

You then carry on developing your application with regular rails MVC code in the app/myrails directory of your extension. The command to add models/controller are in the extension tutorial. The code generators create the stubs for the standard Rails testing framework so I have been testing my Radiant extension by following the tutorial in my Rails Book. This was one of my reasons for choosing Rails in the first place, you can buy books about it.

Radiant is very easy to extend, both by defining new radius tags or simply writing rails code. This makes it potentially a very powerful platform. There is nothing to stop me writing the administration pages of my extension in the same way as regular Radiant extensions and coding the presentation part in plain rails. I want to do some AJAX and the querying of JSON snippets would be done through my rails controller, once I read that part of my book

Incidently, on the subject of Radiant development I once complained how slow radiant development was on my pc. This turns out to be because of the way I was connecting to a Mysql database over an ssh tunnel for every page access. I ought to use sqlite for development on the pc as connection would be instant (I don't want/need a mysql server running on my development pc).

When researching VPS's to host websites a few months ago I came across SliceHost who were very highly regarded but had a long waiting list, with waiting times measured in weeks unless you were willing to commit to a top range deal for a year in advance.

Hence I went with vpslink and a 512Mb VPS. Vpslink are cheap but have the limitation that you get no 'burst' ram so if your 512M runs out then phut, apache crashes. This was happening to me until I changed from the preforking apache to the multithreaded one that uses less memory (fewer processes) after which it was rock solid. However, I no longer trusted it.

Last Saturday I was still mulling over hosting and checked Slicehost again to find that their waiting times were down to less than a week for all their packages. Their prices are slightly better than VPSlink so I applied for one and by tuesday I got an email with a link to sign up. Signing up was easy enough and within minutes I had 1024Mb slice for three months.

My first impression when I logged into it with ssh was that it was very fast. Typing into the console, the keypresses appeared immediately. In a way it's a detail but it shows there is low latency between me and the server (I'm not sure where the server is but I saw something about swbell in the iptables which makes me suspect it is in sw USA). Everything I did with it (installing packages etc) happened very quickly.

But the thing that impressed me most was... the documentation on their site. Very good indeed, great guides on setting up servers for Apache, Rails, Mongrel, iptables etc. Simple step-by-step instructions to go from a raw OS install (ubuntu gutsy for me) to a running site. Other hosting services I have used have had scrappy and out-of-date documents (oneandone) or I had to spend ages trawling forums for answers (site5: and with a shared hosting service like site5 setting anything up is intrinsically more difficult because of the shared hosting shackles). Slicehost appears to be a 'run-by-developers-for-developers' deal and feels very web 2 and geeky. Perfect.

Armed with the documentation my site is running under apache with a mongrel cluster serving up my rails app! And it is running very nicely, very snappy. The mongrel cluster is using 10% of my memory under zero load (I wouldn't dare do this on vpslink) and setting it up is not the most straightforward thing and I couldn't do it again without the slicehost docs. I would ditch apache and use pure mongrel but:

• apache makes virtual sites very easy
• if I ever wanted to move one of my drupal sites to this server I would need php and apache's url rewriting facilities (to get the clean urls working: the 'new wave' servers like lighttpd are great for serving static files but have too many shortcomings for anything beyond that).
• I want multiple rails apps and I'm unsure whether mongrel can do that (I've been bitten by cherrypy and it's one-site-under-mod-python ethos so don't trust anything else yet).

So far Slicehost is looking very promising. And no, there are no referral links here. Vpslink are good but their slightly more expensive spry brand VPS's may be better for reliability. There aren't any other vps providers that I would consider. They are either the pile-high-sell-cheap types (oneandone) or poor/terse support (linode).

Petersblog is still on site5/drupal. Maybe, just maybe, I can make it my christmas holiday project to port it to rails. Maybe.

I'm going to blog this so I can remember it. How to get Rails/ActiveRecord to wrap an existing table in a database, i.e. one that you don't want created or manipulated via migrations:

ruby script/generate model my_table --skip-migration

where my_table is the name of the table you are wrapping. This assumes you are already connected to the database (easy: edit config/database.yml).

It may be possible to simply derive a new class from ActiveRecord but the above is probably ensuring that everything is done properly.

In my case the database is being stuffed from some complex python code that I don't have the time/inclination to port to ruby/rails and I would rather the creation of the tables was still done through python. ActiveRecord being the wonder that it is will pick up the schema of the table and create wrappers automatically so you still don't need to tediously reiterate the column names/functions as you would in lesser frameworks.

It should be entirely possible to create views in MySQL and wrap them for Rails in this fashion although I found that creating views on a 20,000 record table causes the mysqld to take 99% cpu time for ten minutes afterwards. I know what you're saying, 'check your indexes', well I have.

This site was offline yesterday while site5 moved the hosting to a server in Texas (yee ha). Everything was offline for a few hours during which time I changed the DNS so the domains (four in all on this host) pointed to the new server on a different IP address.

When the server came up my generic domain which I use for forgettable experiments was ok but the domain pointers (i.e. secondary mappings of domain names to the same ip address as the primary domain name), including petersblog.org were still broken. I tried to kick petersblog.org into life by deleting the domain pointer and creating it again but then I ran into site5's checking against a whois database, presumably to make sure I wasn't trying to hijack someone else's domain name. It seems that this whois database updates comparatively slowly: when I did a whois lookup the records were already up to date. It wouldn't let me create the domain pointer.

Another domain name on a domain pointer started working later in the day so I ought to have been more patient.

This morning I was able to create the domain pointer and hence I am writing this. Fortunately deleting and recreating the domain pointer has not zapped the awstats log history which is recording half a million hits on petersblog.org over the last year.

A third site was running on Ruby on Rails and Radiant and that was broken: it loaded a page that was totally blank. I presumed that the rails on the new site5 server is different or broken so I did something that was on my todo list anyway, move it to the vpslink server which I have as a better Rails platform.

I've had the site5 account for a year, the contract is up and I may be in a position to abandon it or upgrade. Right now they are doing a special offer of 750g for $7.50 a month (yes, that's 750 GIGABYTES) which is almost irresistable. 750g backup space anyone?

Site5 have been ok up to now for basic web hosting and php stuff. Their rails is not totally hassle free (you can get things running if you spend enough time poring through the forums) and I prefer root access for this. It is still hosting a couple of drupal sites including this blog, mainly because they are tied to php4 which I don't want on my vpslink server (less is more).

The vpslink box has apache randomly crashing, presumably through lack of burst memory. I'm not sure I want to commit myself to it long term. I'm tempted again to get a dedicated host and one of these huge bucket site5 accounts for rsync backups while they are still available.

Disclosure: I was in the site5 affiliate program at one time and even earned a few $$$ from it. I'm not actively in it now, although there may be some old affiliate links knocking around here. I'd rather not take their money and be in a better position to criticise them.

I've had a few emails from people wanting to pay to put links here but I haven't taken them up. You will note that this is one of the few sites on the net not plastered with google ads. I like it that way.

Had to update one of my production Ruby on Rails applications. I've taken the time to install the Subclipse plugin into Aptana and it's oh so smooth now I can develop my application on my work pc, editing and debugging, check the changes into subversion from within Aptana, check the new version out onto my server and rock away.

My application had to do a huge long database query, returning 19,000 records in a .csv file. I had some problems with Apache deciding that the fcgi process had timed out and throwing a 500 error. To resolve this I altered the file /etc/apache2/mods-enabled/fcgid.conf thusly:

<IfModule mod_fcgid.c>
  AddHandler fcgid-script .fcgi
  SocketPath /var/lib/apache2/fcgid/sock
  IPCConnectTimeout 120
  IPCCommTimeout 120
</IfModule>

It was the IPCCommTimeout setting that did the trick, it defaults to 20 seconds.

What with such a large recordset I had to introduce paging to the web application so the results can be previewed in something more snappy than a 19,000 row table. I used the will_paginate plugin to implement this and it was unbelievably easy. With the plugin, the changes amount to changing the query for your recordset to something along the lines of:

@posts = Post.paginate( :all, :page => params[:page])

and adding the following to the bottom of the view to get the next/prev/page 1|2|3 business:

<%= will_paginate @posts %>

That's it, the first of 600 pages is thrown up in a few seconds.

I have put together two projects using Radiant and I really like it. Radiant is Content Management System (CMS) based on Ruby on Rails. It is really easy to put a site up and it has an admirable level of simplicity and power. It can be set up such that every aspect of the site template (layout and css) and content can be configured through the administration pages and these pages are simple enough that non-techys can conceivably be trained up to edit their own content (the impossible dream). The only thing that cannot be managed through this are images, although there is a gallery extension that I haven't tried out yet.

One of my projects needed an extension to be able to handle 'what's on' type events. I tried out one on a list of third party extensions but it wasn't to my taste so I had a go at writing my own extension. I worked through the tutorial on the radiant site and it was very easy: it took me less than two hours to put together what I wanted. Radiant comes with generators that create the skeleton code for extensions for you, you just fill in the blanks. Extensions give you the ability to define your own tags to extend Radiant's tag language. You then mix html tags with your own tags so your tags provide the content, html provides the layout and css provides the styling.

Ultimately Radiant gives you a simple framework to build a site on and the extension facility gives you the hooks to take it pretty much wherever you want. You have full access to Ruby, Ruby standard library, Rails. Very nice.

There must be a downside and for me that would be that Radiant/Rails development can be very slow: ten or twenty seconds waiting for a page to update. Also the Radiant extensions seem to require rebooting the server to see the changes whereas in conventional Rails development it is not necessary to reboot the server (it's wonderful: change the code, switch to browser, bang the refresh button). UPDATE: running under Aptana, using Webrick the server does not need rebooting when the extension tag code is modified.

Radiant seems to be a small project and the SVN is not exactly buzzing. It is in a nice state as it is, functional and bloat-free. Maybe this is a good thing, the API's are not going to change under my feet.

Loving it.

Hobo looks interesting. It is a RAD layer on top of Rails: RAD*2 or RAD^2 it remains to be seen. It promises to make developing rails applications even faster. Interesting features thus far:

• a more robust 'production ready' system for boilerplate user interfaces than rails native scaffold.
• It comes with support for user, roles and authentication built in (using acts_as_authenticated).
• DRYML template language to layout your pages. On no, not another template language. However, this one looks pretty good, it allows you to create your own tags which are effectively macros. You can create libraries of macro snippets to insert into your pages. rhtml can do this with partials but DRYML looks much easier to use, you can build web pages in a procedural fashion (user defined tag == subroutine? get me? no weird inheritance models to get your head around).
• wrappers for Active Record to make it easier to use. For example, they are working on more intelligent migrations having the ability to automatically generate the regressing 'down' code.

The Hobo web site is very nice, a very slick job of selling it including the obligatory screencasts, although the latter would probably only be understandable by people who have used rails. The screencasts show someone using emacs but I'll be tolerant. The written examples are very clear and lucid.

I'll consider this for my next project, now that I've already finished adding users, roles and authentication to my last project (grrr).

Ooh Err, has_and_belongs_to_many, how_can_a_keyword_be_so_long?

Anyway, I've just used it for the first time to implement a UI in rails for user role permissions and here are some notes so that I might be able to use it again without excessive code trawling.

has_and_belongs_to_many is used to define many to many relationships. In the system I am implementing I have many Users and many Roles for them to perform. A User can be assigned to many Roles and a Role may be assigned to many Users. This is a classic many to many relationship and is implemented by having a table that contains a list of mappings of user id to role id, one mapping for each assignment of a user to a role.

This is (essentially) the migration that is used to create the user that is created by acts_as_authenticated but here it is stripped to the meaty bits:

   1  class CreateUsers < ActiveRecord::Migration
   2    def self.up
   3      create_table "users", :force => true do |t|
   4        t.column :login, :string
   5      end
   6    end
   7  
   8    def self.down
   9      drop_table "users"
  10    end
  11  end
Toggle Line Numbers

This is the migration that the role_requirement plugin uses to create the roles tables:

   1  class CreateRoles < ActiveRecord::Migration
   2    def self.up
   3      create_table "roles" do |t|
   4        t.column :name, :string
   5      end
   6  
   7      # generate the join table
   8      create_table "roles_users", :id => false do |t|
   9        t.column "role_id", :integer
  10        t.column "user_id", :integer
  11      end
  12      add_index "roles_users", "role_id"
  13      add_index "roles_users", "user_id"
  14    end
  15  
  16    def self.down
  17      drop_table "roles"
  18      drop_table "roles_users"
  19    end
  20  end
Toggle Line Numbers

In summary, there are three tables in the database:

• A 'users' table with a field called 'login' that gives the user login name
• A 'roles' table with a field called 'name' that gives the name of the role
• A table called 'roles_users' that holds each assignment of a role to a user.

Models need to be created for the User and Role table but not the roles_users table as that one is handled automagicaly by rails.

Here are the meaty bits of the model for the User table:

class User < ActiveRecord::Base
  has_and_belongs_to_many :roles

  validates_length_of       :login,    :within => 1..40
  validates_uniqueness_of   :login, :case_sensitive => false
end

And the model for the Role table:

class Role < ActiveRecord::Base
  has_and_belongs_to_many :users

  validates_presence_of :name
  validates_uniqueness_of   :name, :case_sensitive => false
end

Both these have the magical has_and_belongs_to_many declaration to invoke the many-to-many goodness. They also perform some validation on what the user enters such as making sure they are giving their users login names.

Now for some code for a partial that can be used in a view to edit or create user records. This is used to generate a list of check boxes, one for each role. The check box will be checked according to whether or not the user has been assigned that role:

   1  <% form_for :user, :url => { :action => action, :id => @user} do |f| %>
   2  
   3      <p>Enter the login name for the new user:</p>
   4  
   5      <div style="margin-left: 50px; margin-bottom: 50px">
   6        <%= f.text_field  :login %>
   7      </div>
   8  
   9      <p>Select the Roles that this user can perform</p>
  10  
  11      <div style="margin-left: 50px">
  12          <table>
  13              <% for oRole in Role.find(:all, :order => :name) %>
  14                  <tr>
  15                      <td>
  16                          <%= check_box_tag "user[role_ids][]", oRole.id, @user.roles.include?(oRole) %>
  17                      </td>
  18                      <td>
  19                          <%= oRole.name %>
  20                      </td>
  21                  </tr>
  22              <% end %>
  23          </table>
  24      </div>
  25  
  26      <%= submit_tag submit_tag %>
  27      <%= submit_tag "Cancel" %>
  28  <% end %>
Toggle Line Numbers

Here we iterate through all the Roles in the role table, sorted into name order. For each role we generate a check box tag. The check_box_tag line is tricky but can be broken down as:

"user[role_ids][]"

this is the name for the check box tag field. Naming the check box like this ensures that when the form is posted back to the server, the values for each tag will be placed correctly in the params array

oRole.id

the id of the Role record for this checkbox. It is these ids that are stored in the roles_users table to map a user id to a role id.

@user.roles.include?(oRole)

from the current user record, search the list of roles mapped to that user and see if the list already contains a particular role. If it does then the checkbox will be checked when the form is displayed.

When the form is submitted we need to ensure that the roles mapped to a user are updated correctly. This is simple:

   1  def update
   2    strLogin = params[:user][:login]
   3  
   4    oUser = User.find( params[:id])
   5    oUser.login = strLogin
   6    oUser.role_ids = params[:user][:role_ids]
   7    oUser.save!
   8    flash[:notice] = "Updated User '#{strLogin}'"
   9  
  10    redirect_to :action => :list
  11  end
Toggle Line Numbers

This is the method in the user controller that updates an existing record. The juicy bit is the line

oUser.role_ids = params[:user][:role_ids]

which is ALL it takes to update all the role assignments! This saves a lot of work such as adding new role assignments and removing extraneous ones.

Another little thing to watch out for: when deleting a role or a user do NOT use the delete method:

User.delete( params[:id]) ## WRONG

delete apparently just sends the raw sql to the database engine to get it to delete the object. Instead you should call destroy:

User.destroy( params[:id]) # Delete user and database objects associated with him/her

destroy will invoke the rails Active Record magic that will cause the roles_users table to be updated, removing any entires for roles or users that are being deleted. Calling delete will not do this and will leave stray records in the database.

Trying to add authentication to my fledgeling rails application. Following these steps I get an error and all is broken:

   1  C:\Projects\Aptana IDE Beta\qa>ruby script/generate authenticated user account
   2        exists  app/models/
   3        exists  app/controllers/
   4        exists  app/helpers/
   5        create  app/views/account
   6        exists  test/functional/
   7        exists  test/unit/
   8        create  app/models/user.rb
   9        create  app/controllers/account_controller.rb
  10        create  lib/authenticated_system.rb
  11  No such file or directory - ./script/../config/../lib/authenticated_system.rb
Toggle Line Numbers

Mystifying but the solution is simple: it is expecting the existance of a /lib directory under my rails application root but there isn't one. A quick mkdir lib and all is dandy:

   1  C:\Projects\Aptana IDE Beta\qa>ruby script/generate authenticated user account
   2        exists  app/models/
   3        exists  app/controllers/
   4        exists  app/helpers/
   5        exists  app/views/account
   6        exists  test/functional/
   7        exists  test/unit/
   8     identical  app/models/user.rb
   9     identical  app/controllers/account_controller.rb
  10        create  lib/authenticated_system.rb
  11        create  lib/authenticated_test_helper.rb
  12        create  test/functional/account_controller_test.rb
  13        create  app/helpers/account_helper.rb
  14        create  test/unit/user_test.rb
  15        create  test/fixtures/users.yml
  16        create  app/views/account/index.rhtml
  17        create  app/views/account/login.rhtml
  18        create  app/views/account/signup.rhtml
  19        exists  db/migrate
  20        create  db/migrate/005_create_users.rb
Toggle Line Numbers

Now it's installed and running I find it isn't enough for my needs: my application needs roles, acts_as_authenticated only caters for basic access. I would also like to be able to list users and edit their privileges but acts_as_authenticated does not offer this out of the box.

I'm reminded of life with drupal modules: you have to install them and try them out and see if they do what you want, the descriptions never include enough information.

It looks like I need rolerequirement which extends acts_as_authenticated to give support for roles. It only does the role-checking logic, however this doesn't provide a front-end for administering roles Is scaffold up to it? Can I add authentication to scaffold generated stuff? Let's see.